CN113114585B - Method, device and storage medium for joint optimization of task migration and network transmission - Google Patents

Abstract

The invention relates to the field of computer communication, in particular to a method, equipment and a storage medium for joint optimization of task migration and network transmission, which comprises the following steps: judging whether the called times are 0 or not, executing initialization operation, reasoning out a calculation task migration and TCP initial congestion window setting scheme, updating an experience replay buffer, judging whether the Seq2Seq model parameters need to be updated or not, and updating the Seq2Seq model parameters. The method for joint optimization of task migration and network transmission can provide a scheme for setting the TCP initial congestion window. Compared with the communication network physical layer resource allocation scheme, the implementation difficulty of the scheme is low, and the scheme can be independently completed by the mobile equipment. Secondly, the method uses a deep reinforcement learning technology to infer a calculation task migration and TCP initial congestion window setting scheme, and can be used for quickly making a calculation task completion time minimization decision.

Description

Method, device and storage medium for joint optimization of task migration and network transmission

technical field

The invention relates to the field of computer communication, in particular to a method, device and storage medium for joint optimization of task migration and network transmission.

Background technique

With the popularity of mobile devices such as smartphones and platform computers, mobile applications (i.e. applications running on mobile devices) have also experienced explosive growth in recent years. However, due to the limited computing power of mobile devices and high network transmission delay, it is difficult for some complex mobile applications (such as face recognition, augmented reality, interactive games, etc.) to obtain a satisfactory experience on mobile devices.

To this end, edge computing models have been proposed. This mode allows mobile devices to migrate complex computing tasks to the edge cloud for execution to improve the execution speed of complex computing tasks. However, the migration of computing tasks will inevitably lead to the network transmission of task input data, resulting in an increase in the completion time of computing tasks.

The prior art scheme mainly considers the allocation of resources at the physical layer of the communication network, and solves the optimization problem by using the numerical optimization method to obtain the optimal calculation task migration strategy and the communication network physical layer resource allocation scheme, so as to minimize the completion time of the calculation task. Purpose. In the prior art (Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In IEEE INFOCOM 2016-The 35thAnnual IEEE International Conference on Computer Communications), a computing task migration and physical layer based on Lagrange multiplier method is proposed A joint decision-making method for resource allocation. The method includes the following steps: first, initializing the Lagrangian multiplier; second, solving the computing task migration decision sub-problem; third, solving the mobile device CPU frequency control sub-problem, fourth, solving the mobile device transmission Power distribution sub-problem; fifth, update the Lagrange multiplier; sixth step, if the maximum number of iterations is reached, end the algorithm operation, otherwise turn to the second step. However, the prior art has the following disadvantages:

1. The existing technical solution requires multiple iterations to converge to the optimal solution, and the algorithm runs for a long time, making it difficult to quickly make optimal decisions;

2. The implementation of the communication network physical layer resource allocation scheme given by the prior art scheme is difficult to implement, requires the support of the communication network operator, and cannot be independently completed by the mobile device.

Therefore, how to use limited network resources to minimize the completion time of computing tasks has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a method, device and storage medium for joint optimization of task migration and network transmission, which solves the problems of long algorithm running time and difficult implementation of the allocation scheme.

The object of the present invention can be realized through the following technical solutions:

A method for joint optimization of task migration and network transmission, comprising the following steps:

Step 1: Determine whether the number of calls of the method is 0, if the number of calls t of the method is 0, go to step 2, otherwise go to step 3;

Step 2: Perform the initialization operation: initialize a basic Seq2Seq model with random numbers of size between 0 and 1, the input sequence length of the model is 6M, M is the number of mobile devices in the scene, the hidden layer length is k, and the output sequence length is M;

Among them, the value range of k is any positive integer, the value range of each element in the output sequence is any integer, and the value of M is the number of mobile devices;

Step 3: Infer the calculation task migration and the TCP initial congestion window setting scheme;

Step 4: Update the experience replay buffer;

Step 5: Determine whether it is necessary to update the Seq2Seq model parameters: Calculate the remainder of the number of calls t of the method divided by d, the value range of d is any positive integer, if the remainder is 0, go to Step 6, otherwise the method ends ;

Step 6: Update Seq2Seq model parameters.

Further, the initial value of the number of calls t in step 1 is 0, and each time the method is called once, the value of t will increase by 1, that is, t+1.

Further, the value range of k is preferably a positive integer between 64 and 512 .

Further, the specific operations of the scheme in the step 3 are as follows:

(1) Combine the state vectors of all mobile devices into a system state vector S, that is

S=[S ₁ , S ₂ ,...,S _i ,...,S _M ],

Where M is the number of mobile devices, S _i is the state vector of the i-th mobile device; S _i contains 6 mobile device states, namely

S _i =[IAT _i , IDT _i , IDS _i , LET _i , RTT _i , TP _i ],

where IAT _i is the time interval between the arrival of the first two computing task migration requests on the ith mobile device, IDT _i is the time interval between the completion of the first two computing tasks on the ith mobile device, and IDS _i is the request by the ith mobile device The input data size of the migrated computing task, LET _i is the local running time of the computing task requested to be migrated by the ith mobile device, RTT _i is the round-trip delay between the ith mobile device and the edge server, and TP _i is the ith mobile device the throughput of the device;

(2) Input the system state variable S into the Seq2Seq model, and obtain the inference result T, that is, the calculation task migration and the TCP initial congestion window setting scheme; T is a vector containing M elements, namely

T=[T ₁ , T ₂ , ..., T _i , ..., T _M ]

The meaning of each element is as follows: if T _i is equal to 0, it means that the ith mobile device is not allowed to migrate its computing tasks to the edge server; if T _i is greater than 0, it means that the ith mobile device is allowed to migrate its computing tasks to the edge server edge server, while setting the initial congestion window of the TCP connection used to transmit task input data to T _i .

Further, the concrete operation scheme of described step 4 is as follows:

(1) Judging whether there is free space in the experience replay buffer, if there is no free space, find the experience entry that was first added to the experience replay buffer, and delete it from the experience buffer;

(2) The system state S used in the current call, the inference result T and the current moment are added to the experience replay buffer as an experience entry.

Further, the experience replay buffer is used to store multiple experience entries, the maximum length is L and the value range of L is any positive integer, and each experience entry can be represented as a triple <S, T, E. >, where S is the system state, T is the inference result, and E is the moment when the experience entry is added to the experience buffer.

Further, the concrete operation scheme of described step 6 is as follows:

(1) Judging whether the experience entry in the experience replay buffer is greater than or equal to N, the value range of N is any positive integer less than or equal to the maximum length L of the experience replay buffer, if so, turn to step (2), Otherwise the method ends;

(2) randomly select N experience entries from the experience replay buffer;

(3) Extract the system states and inference results from the N experience items to form N binary groups <S,T>, where S is the system state and T is the inference result;

(4) Using the N binary groups as training datasets, train the Seq2Seq model to update the Seq2Seq model parameters.

A computer-readable storage medium stores instructions, and when the instructions are executed, the above-mentioned method for joint optimization of task migration and network transmission is realized.

A device for joint optimization of task migration and network transmission, comprising a processor, a memory, and a computer program stored in the memory and running on the processor, and when the processor executes the program, the above-mentioned joint optimization of task migration and network transmission is realized method to optimize.

Beneficial effects of the present invention:

1. Compared with using the numerical method to solve the optimization problem, the present invention uses the deep reinforcement learning technology to reason the calculation task migration and the TCP initial congestion window setting scheme faster, and can meet the needs of rapid decision-making;

2. Compared with the resource allocation scheme of the physical layer of the communication network, the implementation difficulty of the TCP initial congestion window setting scheme provided by the present invention is low, and can be independently completed by the mobile device.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. In other words, on the premise of no creative work, other drawings can also be obtained from these drawings;

Figure 1 is a schematic structural diagram of the Seq2Seq model of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The present invention provides a method for joint optimization of task migration and network transmission, comprising the following steps:

Step 1: Determine whether the number of calls of the method is 0, if the number of calls t of the method is 0, go to step 2, otherwise go to step 3;

It should be noted that the initial value of the number of calls t is 0. Whenever the method is called once, the value of t will increase by 1 (that is, it becomes t+1).

Step 2: Perform initialization operations. That is to initialize a basic Seq2Seq model with random numbers between 0 and 1 (the model structure is shown in Figure 1), the input sequence length of the model is 6M, M is the number of mobile devices in the scene, and the hidden layer length is k , the output sequence length is M;

The value range of k is any positive integer, preferably a positive integer between 64 and 512; the value range of each element in the output sequence is any integer; M is the number of mobile devices.

Seq2Seq is a recurrent neural network model that converts an input sequence into an output sequence, and is often used in machine translation, dialogue systems, automatic summarization, etc. In the present invention, the Seq2Seq model is used to establish the optimal mapping relationship between the system state vector and the calculation task migration and the TCP initial congestion window setting scheme, that is, the system state vector is regarded as an input sequence, and the calculation task migration and the TCP initial congestion window setting are set Schemes are seen as output sequences.

Step 3: Get the state vectors of all mobile devices;

For each mobile device, do the following:

(1) Calculate the time interval between the arrival of the first two computing task migration requests (denoted as IAT). If the number of current computing task migration requests is less than 3, let IAT;

(2) Calculate the time interval between the completion of the first two computing tasks (denoted as IDT), if the number of current computing task migration requests is less than 3, let IDT;

(3) Obtain the input data size of the computing task requesting migration (denoted as IDS);

(4) Calculate the local running time (denoted as LET) of the computing task requesting migration, and the calculation formula is as follows

where α is the number of CPU cycles required by the computing task requesting migration, and φ is the CPU frequency of the mobile device;

(5) Measure the round-trip delay (denoted as RTT) and throughput (denoted as TP) between the mobile device and the edge server;

(6) Combine the above mobile device states to form a state vector of the mobile device, that is, the state vector of the mobile device can be represented as a 5-tuple <IAT, IDT, IDS, LET, RTT, TP>.

Step 4: Infer computing task migration and TCP initial congestion window setting scheme;

The specific operation plan is as follows:

(1) Combine the state vectors of all mobile devices into a system state vector S, that is

S=[S ₁ , S ₂ ,...,S _i ,...,S _M ],

Where M is the number of mobile devices, S _i is the state vector of the i-th mobile device; S _i contains 6 mobile device states, namely

S _i =[IAT _i , IDT _i , IDS _i , LET _i , RTT _i , TP _i ],

where IAT _i is the time interval between the arrival of the first two computing task migration requests on the ith mobile device, IDT _i is the time interval between the completion of the first two computing tasks on the ith mobile device, and IDS _i is the request by the ith mobile device The input data size of the migrated computing task, LET _i is the local running time of the computing task requested to be migrated by the ith mobile device, RTT _i is the round-trip delay between the ith mobile device and the edge server, and TP _i is the ith mobile device Throughput of mobile devices.

(2) Input the system state variable S into the Seq2Seq model, and obtain the inference result T (that is, the calculation task migration and the TCP initial congestion window setting scheme); T is a vector containing M elements, namely

T=[T ₁ , T ₂ , ..., T _i , ..., T _M ]

The meaning of each element is as follows: if T _i is equal to 0, it means that the ith mobile device is not allowed to migrate its computing tasks to the edge server; if T _i is greater than 0, it means that the ith mobile device is allowed to migrate its computing tasks to the edge server edge server, while setting the initial congestion window of the TCP connection used to transmit task input data to T _i .

Step 5: Update the experience replay buffer;

The specific operation plan is as follows:

(1) Judging whether there is free space in the experience replay buffer, if there is no free space, find the experience entry that was first added to the experience replay buffer, and delete it from the experience buffer;

(2) The system state S used in the current call, the inference result T and the current moment are added to the experience replay buffer as an experience entry.

The experience replay buffer is used to store multiple experience entries. Each experience entry can be represented as a triple <S,T,E>, where S is the system state, T is the inference result, and E is the moment when the experience entry was added to the experience buffer. The definitions of S and T here are equivalent to those already given in step 4.

The maximum length of the experience replay buffer is L. The value range of L is any positive integer, preferably a positive integer between 1000 and 5000.

Step 6: Determine whether the Seq2Seq model parameters need to be updated. Calculate the remainder of the number of calls t of the method divided by d (the value range of d is any positive integer), if the remainder is 0, then turn to step 6, otherwise the method ends;

Step 7: Update Seq2Seq model parameters:

The specific operation plan is as follows:

(1) judge whether the experience entry in the experience replay buffer is greater than or equal to N, if so, turn to step (2), otherwise the method ends;

The value range of N is any positive integer less than or equal to the maximum length L of the experience playback buffer, preferably a positive integer between 64 and 256.

(2) randomly select N experience entries from the experience replay buffer;

(3) Extract the system states and inference results from the N experience items to form N binary groups <S,T>, where S is the system state and T is the inference result; the definitions of S and T here Equivalent to the definitions already given in step 4.

(4) Using the N binary groups as the training data set, use the stochastic gradient descent algorithm, the Adam algorithm or other deep neural network training algorithms to train the Seq2Seq model to update the Seq2Seq model parameters.

The training process uses the following loss function:

为条件概率运算符；c_n为上下文向量，是所有时刻隐状态的平均值，表示为where N is the training dataset size, T _{n, i} is the i-th component of the n-th inference result,

is the conditional probability operator; cn _is the context vector, which is the average value of the hidden state at all times, expressed as

where h _{n, i} is the hidden state at time i, h _{n, i} can be further expressed as

h _n,i =f(S _n,i ,h _n,i-1 )

where f is the recurrent neural network unit (you can choose one from simple RNN, LSTM, GRU and other recurrent neural network units), Sn _{, i} is the i-th component of the n-th system state vector, h _{n, 0} is the moment 0 hidden states (all hidden states are initialized with random numbers between 0 and 1 after training starts).

It should be noted that steps 1 and 3 are performed in the first computing unit, and steps 2, 4, 5, and 6 are performed in the second computing unit.

A method for joint optimization of task migration and network transmission proposed by the present invention can provide a TCP initial congestion window setting scheme. Compared with the resource allocation scheme of the physical layer of the communication network, the implementation of the scheme is less difficult and can be completed independently by the mobile device. Second, the method uses deep reinforcement learning technology to reason about computing task migration and TCP initial congestion window setting scheme, which can be used to quickly make computing task completion time minimization decisions.

A computer-readable storage medium stores instructions, and when the instructions are executed, the above-mentioned method for joint optimization of task migration and network transmission is realized.

A device for joint optimization of task migration and network transmission, comprising a processor, a memory, and a computer program stored in the memory and running on the processor, and when the processor executes the program, the above-mentioned joint optimization of task migration and network transmission is realized method to optimize.

The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above-mentioned embodiments. The above-mentioned embodiments and descriptions only illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will also have Various changes and modifications fall within the scope of the claimed invention.

Claims (7)

1. a method for joint optimization of task migration and network transmission, is characterized in that, comprises the following steps: Step 1: Perform initialization operation: initialize a basic Seq2Seq model with random numbers of size between 0 and 1, the model has an input sequence length of 6M, M is the number of mobile devices in the scene, the hidden layer length is k, and the output sequence length is M; Among them, the value range of k is any positive integer, the value range of each element in the output sequence is any integer, and the value of M is the number of mobile devices; Step 2: Infer computing task migration and TCP initial congestion window setting scheme; The specific operation of the scheme is as follows: 2.1: Combine the state vectors of all mobile devices into a system state vector S, that is S=[S ₁ , S ₂ ,...,S _i ,...,S _M ], Where M is the number of mobile devices, S _i is the state vector of the i-th mobile device; S _i contains 6 mobile device states, namely Si=[IAT _i , IDT _i , IDS _i , LET _i , RTT _i , TP _i ], where IAT _i is the time interval between the arrival of the first two computing task migration requests on the ith mobile device, IDT _i is the time interval between the completion of the first two computing tasks on the ith mobile device, and IDS _i is the request by the ith mobile device The input data size of the migrated computing task, LET _i is the local running time of the computing task requested to be migrated by the ith mobile device, RTT _i is the round-trip delay between the ith mobile device and the edge server, and TP _i is the ith mobile device the throughput of the device; 2.2: Input the system state vector S into the Seq2Seq model, and obtain the inference result T, that is, the calculation task migration and the TCP initial congestion window setting scheme; T is a vector containing M elements, namely T=[T ₁ , T ₂ , ..., T _i , ..., T _M ] The meaning of each element is as follows: if T _i is equal to 0, it means that the ith mobile device is not allowed to migrate its computing tasks to the edge server; if T _i is greater than 0, it means that the ith mobile device is allowed to migrate its computing tasks to the edge server the edge server, while setting the initial congestion window of the TCP connection used to transmit the task input data to be T _i ; Step 3: Update the experience replay buffer; The experience replay buffer is used to store multiple experience entries, the maximum length is L and the value range of L is any positive integer, and each experience entry can be represented as a triple <S, T, E>, where S is the system state, T is the inference result, and E is the moment when the experience entry is added to the experience buffer; Step 4: Determine whether it is necessary to update the Seq2Seq model parameters: Calculate the remainder of the number of calls t of the method divided by d, the value range of d is any positive integer, if the remainder is 0, go to Step 6, otherwise the method ends ; Step 5: Update Seq2Seq model parameters; Step 6: Determine whether the number of times the method is called is 0, if the number of times t of the method is called is 0, go to step 2, otherwise go to step 3. 2. The method for joint optimization of task migration and network transmission according to claim 1, wherein the initial value of the number of calls t in the step 1 is 0, and whenever the method is called once, the value of t is 0. The value will increase by 1, which is t+1. 3 . The method for joint optimization of task migration and network transmission according to claim 1 , wherein the value range of k in the step 2 is a positive integer between 64 and 512. 4 . 4. the method for joint optimization of a kind of task migration and network transmission according to claim 1, is characterized in that, the concrete operation scheme of described step 4 is as follows: (1) Judging whether there is free space in the experience replay buffer, if there is no free space, find the experience entry that was first added to the experience replay buffer, and delete it from the experience buffer; (2) The system state S used in the current call, the inference result T and the current moment are added to the experience replay buffer as an experience entry. 5. the method for joint optimization of a kind of task migration and network transmission according to claim 1, is characterized in that, the concrete operation scheme of described step 6 is as follows: (1) Judging whether the experience entry in the experience replay buffer is greater than or equal to N, the value range of N is any positive integer less than or equal to the maximum length L of the experience replay buffer, if so, turn to step (2), Otherwise the method ends; (2) randomly select N experience entries from the experience replay buffer; (3) Extract the system states and inference results from the N experience items to form N binary groups <S,T>, where S is the system state and T is the inference result; (4) Using the N binary groups as training datasets, train the Seq2Seq model to update the Seq2Seq model parameters. 6 . A computer-readable storage medium storing instructions, wherein when the instructions are executed by a processor, the method for joint optimization of task migration and network transmission according to any one of the preceding claims 1 to 5 is implemented. 7 . 7. A task migration and network transmission joint optimization device, comprising a processor, a memory and a computer program stored on the memory and running on the processor, wherein the processor implements the above-mentioned right when executing the program The method for joint optimization of task migration and network transmission described in any one of 1 to 5 is required to be optimized.

Images