CN110717992B - Method, apparatus, computer system and readable storage medium for scheduling model - Google Patents

Abstract

The present disclosure provides a method of a scheduling model applied to a scheduling platform, comprising: acquiring a scheduling task, wherein the scheduling task comprises a plurality of models to be scheduled and a plurality of dependency relations among the models to be scheduled, the models to be scheduled at least comprise two models written in different languages or at least comprise two models written in the same language, and the configuration information of the models written in different languages is different; selecting a first model conforming to the operation condition from a plurality of models to be scheduled according to the scheduling logic; determining a first operation platform for operating the first model according to the configuration information of the first model; and sending the execution file of the first model to the first operation platform so that the first operation platform executes the execution file of the first model. The present disclosure also provides an apparatus, a computer system, and a computer-readable storage medium for a scheduling model applied to a scheduling platform.

Description

Method, device, computer system and readable storage medium for scheduling model

Technical field

The present disclosure relates to the field of computer technology, and more specifically, to a method for applying a scheduling model to a dispatching platform, a device for applying a dispatching model to a dispatching platform, a computer system, and a computer-readable storage medium.

Background technique

Among related technologies, the modeling platform can realize functions such as project management, data management, data processing, and model management. Different customers can build models that meet their own business needs on the modeling platform application side, and the built models can be used to achieve business goals. For example, customers build prediction models based on big data to predict data trends. However, with the cross-business requirements among multiple customers, in order to achieve certain business goals, it is inevitable that the models built by different customers need to interact.

In the process of realizing the concept of the present disclosure, the inventor found that there are at least the following problems in related technologies: current modeling platforms can generally only schedule a single model and lack the ability to coordinately schedule multiple models, resulting in the inability to carry out business.

Contents of the invention

In view of this, the present disclosure provides a method for applying a scheduling model to a dispatching platform, a device for applying a scheduling model to a dispatching platform, a computer system, and a computer-readable storage medium.

One aspect of the present disclosure provides a method for applying to a scheduling model of a scheduling platform, including: obtaining a scheduling task, wherein the scheduling task includes multiple to-be-scheduled models and dependencies between the multiple to-be-scheduled models. , the multiple to-be-scheduled models include at least two models written in different languages, and the configuration information of the models written in different languages is different or at least include two models written in the same language; according to the scheduling logic, from all Select a first model that meets the operating conditions among the multiple to-be-scheduled models; determine a first operating platform for operating the first model according to the configuration information of the first model; and send the first operating platform to the first operating platform. The executable file of the first model is generated so that the first running platform executes the executable file of the first model.

According to an embodiment of the present disclosure, the method further includes: receiving status information from the first running platform for running the first model; in the case where the status information represents the completion of running of the first model, according to the Select a second model that meets the operating conditions from the plurality of to-be-scheduled models based on the dependency relationship between the multiple to-be-scheduled models; determine the configuration information for running the second model according to the configuration information of the second model. a second running platform; and sending the executable file of the second model to the second running platform, so that the second running platform executes the executable file of the second model.

According to an embodiment of the present disclosure, the method further includes: receiving a first output result and a first log file from an execution file of the first model executed by the first running platform; receiving an execution file from the second running platform a second output result and a second log file of the execution file of the second model; and storing the first output result, the first log file, the second output result and the second log file.

According to an embodiment of the present disclosure, the method further includes: in the process of the second running platform executing the execution file of the second model, providing the data stored by the scheduling platform to the second running platform to Realize data sharing in the process of running the multiple to-be-scheduled models on different operating platforms.

According to an embodiment of the present disclosure, selecting a second model that meets the operating conditions from the multiple to-be-scheduled models according to the dependency relationship between the multiple to-be-scheduled models includes: determining among the multiple to-be-scheduled models one or more unrun models; determine whether the pre-models each of the one or more un-run models depends on has been completed according to the dependency relationship between the multiple to-be-scheduled models; and run the dependent pre-models The completed unrun model is determined as the second model that meets the running conditions.

According to an embodiment of the present disclosure, the method further includes: before obtaining the scheduling task, obtaining a registration request for registering the multiple to-be-scheduled models; and in response to the registration request, registering the multiple to-be-scheduled models. The execution files corresponding to the scheduling model are stored in the model library.

Another aspect of the present disclosure provides an apparatus for a scheduling model applied to a scheduling platform, including: a first acquisition module for acquiring a scheduling task, wherein the scheduling task includes a plurality of models to be scheduled and the plurality of Dependencies between models to be scheduled. The multiple models to be scheduled include at least two models written in different languages or at least two models written in the same language. The configurations of the models written in different languages The information is different; the first selection module is used to select the first model that meets the operating conditions from the multiple to-be-scheduled models according to the scheduling logic; the first determination module is used to determine the first model according to the configuration information of the first model. a first running platform that runs the first model; and a first sending module for sending the execution file of the first model to the first running platform, so that the first running platform executes the first The executable file of the model.

According to an embodiment of the present disclosure, the device further includes: a first receiving module for receiving status information from the first running platform running the first model; a second selection module for receiving the status information from the first running platform. When the operation of the first model is completed, select a second model that meets the operating conditions from the plurality of models to be scheduled according to the dependency relationship between the multiple models to be scheduled; the second determination module, for determining a second running platform for running the second model according to the configuration information of the second model; and a second sending module for sending the execution file of the second model to the second running platform. , so that the second running platform executes the execution file of the second model.

According to an embodiment of the present disclosure, the device further includes: a second receiving module, configured to receive a first output result and a first log file from an execution file of the first model executed by the first running platform; a third a receiving module, configured to receive a second output result and a second log file from the execution file of the second model executed by the second running platform; and a first storage module, configured to store the first output result, the The first log file, the second output result and the second log file.

According to an embodiment of the present disclosure, the device further includes: a sharing module, configured to provide the scheduling platform to the second running platform during the execution of the execution file of the second model by the second running platform. The stored data is used to realize data sharing in the process of running the multiple to-be-scheduled models on different operating platforms.

According to an embodiment of the present disclosure, the second selection module is configured to: determine one or more unrunning models among the multiple to-be-scheduled models; determine the Whether the pre-models on which each of the one or more un-run models depends has been completed; and determining the un-run model on which the dependent pre-models have been completed as the second model that meets the running conditions.

According to an embodiment of the present disclosure, the device further includes: a second acquisition module, configured to acquire a registration request for registering the multiple to-be-scheduled models before acquiring the scheduling task; and a second storage module, using In response to the registration request, execution files corresponding to the multiple to-be-scheduled models are stored in the model library.

Another aspect of the present disclosure provides a computer system, including: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more When the processor executes, the one or more processors are caused to implement the method as described above.

Another aspect of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon, which when executed by a processor cause the processor to implement the method as described above.

Another aspect of the present disclosure provides a computer program comprising computer-executable instructions that, when executed, are used to implement the method as described above.

Description of the drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

Figure 1 schematically illustrates an exemplary system architecture that can apply methods and devices for scheduling models applied to a scheduling platform according to embodiments of the present disclosure;

Figure 2 schematically illustrates a flowchart of a method for a scheduling model applied to a scheduling platform according to an embodiment of the present disclosure;

Figure 3 schematically illustrates a flowchart of a method for a scheduling model applied to a scheduling platform according to another embodiment of the present disclosure;

Figure 4 schematically shows a schematic diagram of a method applied to a scheduling model of a scheduling platform according to another embodiment of the present disclosure;

Figure 5 schematically illustrates a flow chart of a method for selecting a second model that meets operating conditions from multiple to-be-scheduled models based on dependencies between the multiple to-be-scheduled models according to an embodiment of the present disclosure;

Figure 6 schematically illustrates a flow chart of a method applied to a scheduling model of a scheduling platform according to another embodiment of the present disclosure;

Figure 7 schematically shows a block diagram of a device for a scheduling model applied to a scheduling platform according to an embodiment of the present disclosure; and

Figure 8 schematically illustrates a block diagram of a computer system suitable for implementing the method described above, according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for convenience of explanation, numerous specific details are set forth to provide a comprehensive understanding of the disclosed embodiments. It will be apparent, however, that one or more embodiments may be practiced without these specific details. Furthermore, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. The terms "comprising," "comprising," and the like, as used herein, indicate the presence of stated features, steps, operations, and/or components but do not exclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used here should be interpreted to have meanings consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

Where an expression similar to "at least one of A, B, C, etc." is used, it should generally be interpreted in accordance with the meaning that a person skilled in the art generally understands the expression to mean (e.g., "having A, B and C "A system with at least one of" shall include, but is not limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or systems with A, B, C, etc. ). Where an expression similar to "at least one of A, B or C, etc." is used, it should generally be interpreted in accordance with the meaning that a person skilled in the art generally understands the expression to mean (for example, "having A, B or C "A system with at least one of" shall include, but is not limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or systems with A, B, C, etc. ).

Embodiments of the present disclosure provide a method for applying a scheduling model to a dispatching platform, a device for applying a scheduling model to a dispatching platform, a computer system, and a computer-readable storage medium. The method includes: obtaining a scheduling task, wherein the scheduling task includes multiple to-be-scheduled models and dependencies between the multiple to-be-scheduled models, and the multiple to-be-scheduled models include at least two models written in different languages or at least two models. Models written in the same language have different configuration information for models written in different languages; select the first model that meets the running conditions from multiple to-be-scheduled models according to the scheduling logic; determine the model for running based on the configuration information of the first model a first running platform of the first model; and sending the execution file of the first model to the first running platform, so that the first running platform executes the execution file of the first model.

FIG. 1 schematically illustrates an exemplary system architecture that can apply methods and devices for scheduling models applied to a scheduling platform according to embodiments of the present disclosure. It should be noted that Figure 1 is only an example of a system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be used in other applications. Device, system, environment or scenario.

As shown in Figure 1, the system architecture 100 according to this embodiment may include a terminal device 101, a network 102 and a network 104, a scheduling platform 103, an operating platform 105 and an operating platform 106. Networks 102 and 104 are used to provide media for communication links between the terminal device 101, the scheduling platform 103, and the operating platforms 105 and 106. Networks 102 and 104 may include various connection types, such as wired and/or wireless communication links, and the like.

The terminal device 101 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

The user can use the terminal device 101 to interact with the dispatch platform 103 through the network 102 to receive or send messages, etc. Various communication client applications may be installed on the terminal device 101, such as web browser applications, search applications, instant messaging tools, email clients and/or social platform software (only examples). Users can register the model they need to run through the browser application. During the process of registering the model, the model code and description file and other information can be uploaded to the scheduling platform 103 through the network 102 .

The scheduling platform 103 may be composed of one or more servers, and may be a background management server that provides support for websites browsed by users using the terminal device 101 (example only). The background management server can analyze and process the received user request and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.

The scheduling platform 103 can send the execution file of the model to the execution platform 105 and/or 106 through the network 104, so that the execution platform 105 and/or 106 can execute the execution file of the model, and the execution file can include the code of the model, thereby achieving the effect of running the model. .

The execution platform 105 and/or 106 may be composed of one or more servers and can support the execution of multi-language scripts.

It should be noted that the scheduling model method provided by the embodiment of the present disclosure can generally be executed by the scheduling platform 103. Correspondingly, the device of the scheduling model provided by the embodiment of the present disclosure may generally be provided in the scheduling platform 103.

It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Depending on implementation needs, there can be any number of end devices, networks, and servers.

FIG. 2 schematically shows a flow chart of a method applied to a scheduling model of a scheduling platform according to an embodiment of the present disclosure.

As shown in Figure 2, the method includes operations S210 to S240.

In operation S210, a scheduling task is obtained, where the scheduling task includes multiple to-be-scheduled models and dependencies between the multiple to-be-scheduled models. The multiple to-be-scheduled models include at least two models written in different languages or at least two models. Models written in the same language have different configuration information for models written in different languages.

According to embodiments of the present disclosure, the scheduling task may be generated based on the user's operation on the client, and the client may send the result obtained in response to the user's operation to the scheduling platform.

For example, users can connect multiple scheduled models ready for execution on the client's visual interface by dragging and connecting to generate scheduling tasks, and then send them to the scheduling platform.

According to embodiments of the present disclosure, before obtaining the scheduling task, a registration request for registering multiple models to be scheduled can be obtained first, and in response to the registration request, execution files corresponding to the multiple models to be scheduled are stored in the model library.

According to embodiments of the present disclosure, the execution file corresponding to the model to be scheduled may include information such as the code file and description file of the model, and the model may be managed in the model library.

According to embodiments of the present disclosure, users can write models to be scheduled on their own modeling platforms. Depending on the modeling platform, the writing languages that can be used may be different, and the resulting model script types may also be different. For example, SQL scripts can be obtained by writing in the SQL language, JavaScript scripts can be obtained by writing in the JavaScript language, and python scripts can be obtained by writing in the python language.

According to embodiments of the present disclosure, each model has corresponding configuration information. For example, the configuration information of each model includes but is not limited to model ID, name, description, parameters, running environment information, etc.

In operation S220, a first model that meets the operating conditions is selected from multiple to-be-scheduled models according to the scheduling logic.

According to embodiments of the present disclosure, the first model may be the first model that needs to be executed after the scheduling task is submitted, or it may be a model that does not rely on the output of other models as input in the scheduling task.

According to embodiments of the present disclosure, the first model may include multiple. In other words, the scheduling platform can schedule multiple models to the same or different running platforms at the same time for simultaneous running. When the model scripts are different and the required running environments are different, multiple models can be scheduled to run simultaneously on different running platforms. For example, the model of the SQL script is scheduled to the running platform that provides the sparkSQL running environment, the model of the JavaScript script is scheduled to the running platform that provides the JavaScript engine, and the model of the python script is scheduled to the running platform that provides the python container.

In operation S230, a first running platform for running the first model is determined according to the configuration information of the first model.

According to an embodiment of the present disclosure, the configuration information of the first model may be script type information of the model. Determine the running platform that can run the model based on the script type of the model.

According to embodiments of the present disclosure, models written in different languages can be run by different running platforms.

In operation S240, the execution file of the first model is sent to the first running platform, so that the first running platform executes the execution file of the first model.

Through the embodiments of the present disclosure, business goals can be achieved by scheduling multiple models through the scheduling platform. The multiple models can be models written in different languages. Different models can be scheduled to run on different running platforms, and cross-process and cross-platform models can be implemented. Scheduling and collaboration support the running of models in different languages, enabling multiple models to collaboratively complete business data processing. Therefore, it can solve the problem in related technologies that modeling platforms can generally only schedule a single model and lack the collaborative scheduling capability for multiple models, resulting in the inability to carry out business.

According to embodiments of the present disclosure, the scheduling platform may receive status information for running the first model from the first running platform. When the status information represents the completion of running the first model, the scheduling platform may obtain status information from multiple to-be-scheduled models based on the dependencies between the multiple to-be-scheduled models. Select a second model that meets the operating conditions among the to-be-scheduled models, determine a second operating platform for running the second model based on the configuration information of the second model, and send the execution file of the second model to the second operating platform so that the second model The second operating platform executes the execution file of the second model.

According to embodiments of the present disclosure, the scheduling platform can abstract the input, output, status update and other behaviors of the model, and provide standardized interfaces for the running platform to call when running the model, and also provide description specifications of the model dependency chain.

The method provided by the present disclosure will be further described below in conjunction with specific embodiments with reference to Figures 3 to 5 .

FIG. 3 schematically shows a flowchart of a method applied to a scheduling model of a scheduling platform according to another embodiment of the present disclosure.

As shown in Figure 3, the method includes operations S310 to S330.

In operation S310, a first output result of executing the execution file of the first model and a first log file are received from the first running platform.

In operation S320, a second output result of executing the execution file of the second model and a second log file are received from the second running platform.

In operation S330, the first output result, the first log file, the second output result and the second log file are stored.

According to embodiments of the present disclosure, when the second running platform executes the execution file of the second model, the data stored by the scheduling platform can be provided to the second running platform, so as to realize the process of running multiple to-be-scheduled models on different running platforms. data sharing.

Figure 4 schematically shows a schematic diagram of a method applied to a scheduling model of a scheduling platform according to another embodiment of the present disclosure.

As shown in Figure 4, the scheduling platform 402 has a dependent database 401. When the model is running, data can be read from the database with the help of basic capabilities. The scheduling platform 402 can provide a variety of services, such as basic services, data reading and writing services, log services, status services, and task scheduling services. Different running platforms 403 can obtain information from the scheduling platform 402. For example, the second running platform can obtain the status information of the first running platform from the dispatching platform 402, obtain the output results of the first running platform, and so on.

According to embodiments of the present disclosure, the basic service may be used to provide a data sharing service between multiple to-be-scheduled models.

According to embodiments of the present disclosure, the first to third operating platforms can feed back information such as output results, log files, status information, etc. to the scheduling platform 402. The scheduling platform 402 can provide the operating platform with an information sharing function. The scheduling platform 402 can schedule different models to different running platforms for execution.

FIG. 5 schematically shows a flowchart of a method for selecting a second model that meets operating conditions from multiple to-be-scheduled models based on dependencies between the multiple to-be-scheduled models according to an embodiment of the present disclosure.

As shown in Figure 5, the method includes operations S510 to S530.

In operation S510, one or more unrun models among the plurality of models to be scheduled are determined.

According to embodiments of the present disclosure, one or more unrun models among the multiple to-be-scheduled models may be determined after the first model has been run.

In operation S520, it is determined based on the dependency relationships between the multiple to-be-scheduled models whether the pre-models on which each of the one or more unrunning models depends is completed.

According to embodiments of the present disclosure, multiple to-be-scheduled models may logically form a directed acyclic graph, and the dependency relationship between multiple to-be-scheduled models may be, for example, that the input of the later model is the output of the previous model, and only if After the former model has been run, the latter model can start running.

In operation S530, the unrun model that has completed running of the dependent previous model is determined as a second model that meets the running conditions.

According to embodiments of the present disclosure, the number of second models may include multiple, and multiple second models may be sent to different running platforms at the same time for running.

FIG. 6 schematically illustrates a flowchart of an example method applied to a scheduling model of a scheduling platform according to an embodiment of the present disclosure.

As shown in Figure 6, the method includes operations S610 to S670.

In operation S610, the scheduling platform may obtain the scheduling task from the client.

In operation S620, the scheduling platform selects a model that meets the running conditions through scheduling logic, and determines whether there is an unrun model in the current task. If it does not exist, operation S630 is performed to mark the current task as running successfully and end the task. Otherwise, proceed to the next step S640.

In operation S640, the scheduling platform determines whether the unrun model has a pre-model or whether the pre-model has been successfully run. If so, operation S650 is performed to submit the current model.

In operation S650, after the model is submitted, the running platform can run in an asynchronous manner, and first obtains model parameters and input data from the scheduling platform.

In operation S660, the dispatching platform can encapsulate the basic capabilities, model parameters and other files of the scheduling platform into a task package that can be recognized by the running platform. The basic capabilities of the scheduling platform are provided to other running platforms through interfaces, and these interfaces are encapsulated into the SDK. , the model calls the basic capabilities of the scheduling platform through the interface provided in the SDK. According to the embodiment of the present disclosure, after the task is submitted to the running platform for execution, the task can be jumped to S620 for subsequent scheduling.

In operation S670, the running platform executes the encapsulated task package. The running platform can feedback status and record logs in real time, and finally store the data generated by the model. Specifically, the running platform can call the status service of the dispatching platform through the SDK to report the running status, call the log service of the dispatching platform to store the running log, and call the data service of the dispatching platform to obtain input data and save output data. During the model running process, information sharing and communication with other models can be achieved through the basic services of the scheduling platform.

According to embodiments of the present disclosure, the scheduling platform can be responsible for model interaction, status collection, log collection, persistence of model intermediate results, etc., and packages multiple models into tasks according to specifications and assigns them to the running platform for execution. Through the embodiments of the present disclosure, business goals can be achieved by scheduling multiple models through the scheduling platform. The multiple models can be models written in different languages. Different models can be scheduled to run on different running platforms, and cross-process and cross-platform models can be implemented. Scheduling and collaboration support the running of models in different languages, enabling multiple models to collaboratively complete business data processing. This solves the problem in related technologies that modeling platforms can generally only schedule a single model and lack collaborative scheduling capabilities for multiple models, resulting in inability to carry out business.

Figure 7 schematically shows a block diagram of a device for a scheduling model applied to a scheduling platform according to an embodiment of the present disclosure.

As shown in FIG. 7 , the device 700 applied to the scheduling model of the scheduling platform includes a first acquisition module 710 , a first selection module 720 , a first determination module 730 and a first sending module 740 .

The first acquisition module 710 is used to acquire scheduling tasks, where the scheduling tasks include multiple to-be-scheduled models and dependencies between the multiple to-be-scheduled models. The multiple to-be-scheduled models include at least two models written in different languages or Include at least two models written in the same language. Models written in different languages have different configuration information.

The first selection module 720 is used to select a first model that meets the operating conditions from multiple to-be-scheduled models according to the scheduling logic.

The first determining module 730 is configured to determine a first running platform for running the first model according to the configuration information of the first model.

The first sending module 740 is used to send the execution file of the first model to the first running platform, so that the first running platform executes the execution file of the first model.

Through the embodiments of the present disclosure, business goals can be achieved by scheduling multiple models through the scheduling platform. The multiple models can be models written in different languages. Different models can be scheduled to run on different running platforms, and cross-process and cross-platform models can be implemented. Scheduling and collaboration support the running of models in different languages, enabling multiple models to collaboratively complete business data processing. This solves the problem in related technologies that modeling platforms can generally only schedule a single model and lack collaborative scheduling capabilities for multiple models, resulting in inability to carry out business.

The device 700 applied to the scheduling model of the scheduling platform also includes a first receiving module, a second selecting module, a second determining module and a second sending module.

The first receiving module is used to receive status information from the first running platform running the first model.

The second selection module is configured to select a second model that meets the operating conditions from multiple to-be-scheduled models based on dependencies between the multiple to-be-scheduled models when the status information indicates that the first model is completed.

The second determination module is configured to determine a second running platform for running the second model according to the configuration information of the second model.

The second sending module is used to send the execution file of the second model to the second running platform, so that the second running platform executes the execution file of the second model.

According to an embodiment of the present disclosure, the apparatus 700 applied to the scheduling model of the scheduling platform further includes a second receiving module, a third receiving module and a first storage module.

The second receiving module is configured to receive the first output result and the first log file from the execution file of the first running platform executing the first model.

The third receiving module is configured to receive the second output result and the second log file from the execution file of the second model executed by the second running platform.

The first storage module is used to store the first output result, the first log file, the second output result and the second log file.

According to an embodiment of the present disclosure, the device 700 applied to the scheduling model of the scheduling platform further includes a sharing module for providing the second running platform with the information stored by the scheduling platform during the process of the second running platform executing the execution file of the second model. Data to achieve data sharing in the process of running multiple to-be-scheduled models on different operating platforms.

According to an embodiment of the present disclosure, the second selection module is used to determine one or more unrun models among the multiple to-be-scheduled models; determine the respective dependencies of the one or more un-run models according to the dependencies between the multiple to-be-scheduled models. whether the preceding model has been completed; and determine the unrun model whose dependent preceding model has been completed as the second model that meets the running conditions.

According to an embodiment of the present disclosure, the device 700 applied to the scheduling model of the scheduling platform further includes a second acquisition module and a second storage module.

The second acquisition module is used to acquire registration requests for registering multiple models to be scheduled before acquiring the scheduling task.

The second storage module is configured to respond to the registration request and store execution files corresponding to multiple to-be-scheduled models in the model library.

Any number of modules, sub-modules, units, sub-units according to embodiments of the present disclosure, or at least part of the functions of any number of them, may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present disclosure may be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLA), System-on-a-chip, system-on-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuits that can be implemented in hardware or firmware, or in a combination of software, hardware, and firmware Any one of these implementation methods or an appropriate combination of any of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a computer program module, and when the computer program module is executed, corresponding functions may be performed.

For example, any more of the first acquisition module 710, the first selection module 720, the first determination module 730 and the first sending module 740 can be combined and implemented in one module/unit/subunit, or any one module/ A unit/subunit can be split into multiple modules/units/subunits. Alternatively, at least part of the functionality of one or more of these modules/units/subunits may be combined with at least part of the functionality of other modules/units/subunits and combined in one module/unit/subunit realized in. According to an embodiment of the present disclosure, at least one of the first acquisition module 710 , the first selection module 720 , the first determination module 730 and the first sending module 740 may be at least partially implemented as a hardware circuit, such as a field programmable gate array. (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means by which circuits can be integrated or packaged It can be implemented by firmware, or it can be implemented by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the first obtaining module 710, the first selecting module 720, the first determining module 730 and the first sending module 740 may be at least partially implemented as a computer program module, and when the computer program module is executed, Perform the corresponding function.

It should be noted that the device part for utilizing the dispatch platform to dispatch the model in the embodiment of the present disclosure corresponds to the method part for utilizing the dispatch platform to dispatch the model in the embodiment of the present disclosure. The description of the device part for utilizing the dispatch platform to dispatch the model is specific. Please refer to the method section of scheduling model using the scheduling platform, which will not be described again here.

According to an embodiment of the present disclosure, a computer system is also provided, including: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more When executed by multiple processors, the one or more processors implement the method as described above.

According to an embodiment of the present disclosure, a computer-readable storage medium is also provided, on which executable instructions are stored. When the instructions are executed by a processor, they cause the processor to implement the method as described above.

Figure 8 schematically illustrates a block diagram of a computer system suitable for implementing the method described above, according to an embodiment of the present disclosure. The computer system shown in FIG. 8 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 8 , a computer system 800 according to an embodiment of the present disclosure includes a processor 801 that can be loaded into a random access memory (RAM) 803 according to a program stored in a read-only memory (ROM) 802 or from a storage portion 808 program to perform various appropriate actions and processes. Processor 801 may include, for example, a general purpose microprocessor (eg, a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (eg, an application specific integrated circuit (ASIC)), among others. Processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiments of the present disclosure.

In the RAM 803, various programs and data required for the operation of the system 800 are stored. The processor 801, ROM 802, and RAM 803 are connected to each other through a bus 804. The processor 801 performs various operations according to the method flow of the embodiment of the present disclosure by executing programs in the ROM 802 and/or RAM 803. It should be noted that the program may also be stored in one or more memories other than ROM 802 and RAM 803. The processor 801 may also perform various operations according to the method flow of embodiments of the present disclosure by executing programs stored in the one or more memories.

According to embodiments of the present disclosure, system 800 may also include an input/output (I/O) interface 805 that is also connected to bus 804 . System 800 may also include one or more of the following components connected to I/O interface 805: an input portion 806 including a keyboard, mouse, etc.; including a cathode ray tube (CRT), liquid crystal display (LCD), etc.; and speakers. an output section 807, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. Driver 810 is also connected to I/O interface 805 as needed. Removable media 811, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.

According to embodiments of the present disclosure, the method flow according to the embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method illustrated in the flowchart. In such embodiments, the computer program may be downloaded and installed from the network via communications portion 809 and/or installed from removable media 811 . When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to embodiments of the present disclosure, the systems, devices, devices, modules, units, etc. described above may be implemented by computer program modules.

The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium may be included in the device/device/system described in the above embodiments; it may also exist independently without being assembled into the device/system. in the device/system. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.

For example, according to embodiments of the present disclosure, the computer-readable storage medium may include one or more memories other than ROM 802 and/or RAM 803 and/or ROM 802 and RAM 803 described above.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagram or flowchart illustration, and combinations of blocks in the block diagram or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or may be implemented by special purpose hardware-based systems that perform the specified functions or operations. Achieved by a combination of specialized hardware and computer instructions. Those skilled in the art will understand that features recited in various embodiments and/or claims of the present disclosure may be combined and/or combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, various combinations and/or combinations of features recited in the various embodiments and/or claims of the disclosure may be made without departing from the spirit and teachings of the disclosure. All such combinations and/or combinations fall within the scope of this disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment is described separately above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims (10)

1. A method of scheduling model for a scheduling platform, comprising:

acquiring a scheduling task, wherein the scheduling task comprises a plurality of models to be scheduled and a dependency relationship among the models to be scheduled, the models to be scheduled at least comprise two models written in different languages or at least comprise two models written in the same language, and the configuration information of the models written in different languages is different;

selecting a first model conforming to the operation condition from the multiple models to be scheduled according to the scheduling logic;

determining a first operation platform for operating the first model according to the configuration information of the first model; and

Sending an execution file of the first model to the first operation platform so that the first operation platform executes the execution file of the first model;

wherein the first model comprises: and the first model to be executed in the plurality of models to be scheduled and/or the model which does not depend on the output of other models as input in the scheduling task.

2. The method of claim 1, further comprising:

receiving state information of running the first model from the first running platform;

under the condition that the state information characterizes that the first model is operated, selecting a second model conforming to the operation condition from the plurality of models to be scheduled according to the dependency relationship among the plurality of models to be scheduled;

determining a second operation platform for operating the second model according to the configuration information of the second model; and

and sending the execution file of the second model to the second operation platform so that the second operation platform executes the execution file of the second model.

3. The method of claim 2, further comprising:

receiving a first output result of an execution file executing the first model and a first log file from the first operation platform;

Receiving a second output result of an execution file executing the second model and a second log file from the second operation platform; and

and storing the first output result, the first log file, the second output result and the second log file.

4. A method according to claim 3, further comprising:

and in the process that the second operation platform executes the execution file of the second model, providing the data stored by the dispatching platform for the second operation platform so as to realize data sharing in the process that different operation platforms operate the plurality of models to be dispatched.

5. The method of claim 2, wherein selecting a second model from the plurality of models to be scheduled that meets the operating condition according to the dependency relationship between the plurality of models to be scheduled comprises:

determining one or more non-running models of the plurality of models to be scheduled;

determining whether the front model on which the one or more non-running models depend respectively is finished to run according to the dependency relationship among the plurality of models to be scheduled; and

and determining an unexposed model with the dependent front-end model operation completed as a second model conforming to the operation condition.

6. The method of claim 1, further comprising:

before acquiring the scheduling task, acquiring a registration request for registering the plurality of models to be scheduled; and

and responding to the registration request, and storing the execution files corresponding to the multiple models to be scheduled in a model library.

7. An apparatus for a scheduling model for a scheduling platform, comprising:

the system comprises a first acquisition module, a second acquisition module and a scheduling module, wherein the scheduling task comprises a plurality of models to be scheduled and a dependency relationship among the models to be scheduled, the models to be scheduled at least comprise two models written in different languages or at least comprise two models written in the same language, and the configuration information of the models written in different languages is different;

the first selection module is used for selecting a first model which meets the operation condition from the plurality of models to be scheduled according to the scheduling logic;

the first determining module is used for determining a first operation platform for operating the first model according to the configuration information of the first model; and

the first sending module is used for sending the execution file of the first model to the first operation platform so that the first operation platform executes the execution file of the first model;

Wherein the first model comprises: and the first model to be executed in the plurality of models to be scheduled and/or the model which does not depend on the output of other models as input in the scheduling task.

8. The apparatus of claim 7, further comprising:

the first receiving module is used for receiving state information of running the first model from the first running platform;

the second selection module is used for selecting a second model conforming to the operation condition from the plurality of models to be scheduled according to the dependency relationship among the plurality of models to be scheduled under the condition that the state information characterizes that the operation of the first model is completed;

the second determining module is used for determining a second operation platform for operating the second model according to the configuration information of the second model; and

and the second sending module is used for sending the execution file of the second model to the second operation platform so that the second operation platform executes the execution file of the second model.

9. A computer system, comprising:

one or more processors;

a memory for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 6.

10. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 6.