CN114331290A - Distribution method, apparatus, electronic device, storage medium, and program product - Google Patents

Abstract

The embodiment of the application provides a distribution method, a distribution device, an electronic device, a storage medium and a program product, wherein the method achieves the purpose of distributing articles by public transportation by relaying the articles carried by distribution resources (such as passengers, riders and the like) taking the public transportation to reach a final destination. The embodiment of the application adopts public transport as a transportation means for distribution, and can realize that the public transport can reach a far destination in an urban area at a higher speed, namely, the public transport is used as a trunk road in a distribution system to realize wide-range city-wide distribution. And the distribution mode based on public transport can reduce the use of electric vehicles in distribution, and improves the safety of distribution resources to a certain extent. In addition, the distribution method provided by the embodiment of the application is very simple, the distribution resources only need to extract or store the articles at the starting point and the end point of the route of the distribution resources, and the route does not need to be specially changed, so that more common passengers can be promoted to participate as the distribution resources, and the distribution efficiency can be obviously improved.

Description

Distribution method, apparatus, electronic device, storage medium, and program product

technical field

The present application relates to the technical field of data processing, and in particular, to a distribution method, apparatus, electronic device, storage medium and program product.

Background technique

With the rise of online food ordering and other businesses, the distribution industry has developed rapidly, and the number of related employees has gradually increased. Among them is a nascent occupation—delivery riders, who often need to drive electric vehicles through heavy traffic to deliver meals or packages to customers on time.

It is not difficult to find that the existing distribution methods based on electric vehicles have some defects. For example, due to the limitations of electric vehicle speed and electricity, long-distance distribution is very inconvenient, and it is easy to affect timeliness and safety. How to develop a new distribution mode to improve the defects of electric vehicle distribution has become a research hotspot in the industry.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to solve the technical defects existing in the existing distribution methods, such as technical defects such as limited distribution range.

According to an aspect of the embodiments of the present application, a distribution method is provided, the method comprising:

In response to a dispatch request for any target item, perform at least one dispatch to any target item in the following manner until either target item reaches the final destination:

Get current environmental information and order information for each item in each locker;

Determine the next destination of each item in public transportation based on the current environment information and the order information of each item;

When detecting an item delivery instruction initiated by the delivery resource for any storage cabinet, obtain the destination of the delivery resource;

According to the next destination and the ride destination of each item in any storage cabinet, assign at least one item whose next destination is the same as the ride destination to the distribution resource, so that the distribution resource can carry at least one item to the ride destination storage cabinets.

In an optional implementation manner, based on the environmental information and the order information of each item, the next destination of each item in public transportation is determined, including:

Determine the model parameters of the pre-built profit model under the profit maximum constraint. The model parameters of the profit model include the distribution revenue of each item, the number of distributions for each item, the distribution cost per distribution for each item, the number of overtime orders, and At least one of the cost of order overtime;

Based on the profit maximization constraint, the current environment information, and the order information of each item, the next destination of each item in public transportation is determined.

In an optional implementation manner, based on the environmental information and the order information of each item, the next destination of each item in public transportation is determined, including:

Based on the environmental information and the order information of each item, the next destination of each item in public transportation is determined through the reinforcement learning scheduling model. The reinforcement learning scheduling model is obtained based on the reinforcement learning algorithm;

Among them, the decision-making behavior of the reinforcement learning algorithm includes determining the next destination for the item, the environmental state of the reinforcement learning algorithm includes the environmental information and the order information of the item, and the reward of the reinforcement learning algorithm includes the profit after the item reaches the final destination.

In an optional implementation, the reinforcement learning scheduling model is obtained based on a reinforcement learning algorithm, including:

Obtain multiple training samples, each of which includes environmental information at historical moments and order information for multiple items at historical moments;

Determine the historical environment state of the reinforcement learning algorithm according to each training sample, and determine the next destination of multiple items in public transportation at the historical moment based on each training sample, as the historical decision-making behavior of the reinforcement learning algorithm, according to the historical moment. The profit of multiple items reaching the final destination to determine the historical return of the reinforcement learning algorithm;

Reinforcement learning is performed based on historical environment states, historical decision-making behaviors and historical rewards, and a reinforcement learning scheduling model is obtained.

In an optional implementation manner, based on the environmental information and the order information of each item, the next destination of each item in public transportation is determined, including:

Determine the estimated time of arrival for each item using various routes;

The next destination of each item in public transportation is determined based on the environmental information, the order information of each item, and the estimated time of arrival corresponding to each item using various routes.

In an optional implementation manner, the estimated time of arrival corresponding to each item using various routes is determined, including:

According to at least one of the delivery starting point of each item, the delivery destination of each item, the preset participation rate, and the delivery time period of each item, through the preset estimated time of arrival model, it is obtained that each item adopts various Delivery time and waiting time corresponding to the route;

Based on the delivery time and waiting time corresponding to each item using various routes, determine the estimated arrival time corresponding to each item using various routes;

Among them, the estimated time of arrival model is obtained by Gaussian fitting of the historical delivery time and historical waiting time of multiple items.

In an optional implementation manner, the environmental information includes at least one of the following: weather information, time information, distribution resource supply information, and item distribution demand information;

The order information for each item includes at least one of the following: delivery origin, delivery destination, total time limit, remaining delivery time, allocation history, delivery revenue, delivery cost per allocation, and order overtime cost.

According to another aspect of the embodiments of the present application, a distribution method is provided, the method comprising:

When detecting an item delivery instruction initiated by the delivery resource for any storage cabinet, obtain the destination of the delivery resource;

Send the ride destination to the dispatch platform;

Receive item information of at least one item whose next destination and ride destination are the same as the next destination allocated by the dispatching platform to the distribution resource;

Display item information.

According to yet another aspect of the embodiments of the present application, a delivery device is provided, the device comprising:

The delivery request receiving module is used to receive a delivery request for any target item:

The order information acquisition module is used to acquire the current environment information and the order information of each item in each storage cabinet;

The item destination determination module is used to determine the next destination of each item in public transportation based on the current environment information and the order information of each item;

The ride destination acquisition module is used to obtain the ride destination of the distribution resource when it detects an item distribution instruction initiated by the distribution resource for any storage cabinet;

The allocation module, according to the next destination and the riding destination of each item in any storage cabinet, allocates at least one item whose next destination is the same as the riding destination to the distribution resource, so that the distribution resource can carry at least one item to the destination. Store in a locker at the ride destination.

According to yet another aspect of the embodiments of the present application, a delivery device is provided, the device comprising:

The obtaining module is used to obtain the destination of the delivery resource when it detects the delivery instruction initiated by the delivery resource for any storage cabinet;

The sending module is used to send the ride destination to the dispatching platform;

a receiving module, configured to receive the item information of at least one item whose next destination is the same as the riding destination allocated by the scheduling platform to the distribution resource;

The display module is used to display the item information.

According to an aspect of an embodiment of the present application, an electronic device is provided, the electronic device includes: a memory, a processor, and a computer program stored in the memory, where the processor executes the computer program to implement the delivery method provided by the embodiment of the present application. step.

According to an aspect of the embodiments of the present application, a computer-readable storage medium is provided, and a computer program is stored thereon, and when the computer program is executed by a processor, the steps of the distribution method provided by the embodiments of the present application are implemented.

According to an aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the distribution method provided by the embodiments of the present application are implemented.

The distribution method, device, electronic device, storage medium, and program product provided by the embodiments of the present application can take the distribution resources (such as passengers, riders, etc.) of public transportation to carry the goods to the final destination, and achieve the goal of using public transportation to distribute the goods. Purpose. In the embodiment of the present application, public transportation is used as the means of delivery, which can achieve a faster speed to a destination farther in the urban area, that is, as a trunk road in the delivery system, to achieve a wider range of city-wide delivery. And the distribution method based on public transportation can reduce the use of electric vehicles in distribution, and improve the safety of distribution resources to a certain extent. In addition, the distribution method provided by the embodiment of the present application is very simple, and the distribution resources only need to extract or store the items at the starting point and the end point of their own routes, and there is no need to transfer routes, so more ordinary passengers can be promoted as distribution resources. Participation can significantly improve distribution efficiency.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments of the present application.

1 is a schematic diagram of the system architecture of a crowdsourcing distribution system provided by an embodiment of the present application;

2 is a schematic flowchart of a distribution method provided by an embodiment of the present application;

FIG. 3 is an example diagram of an item distribution provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of reinforcement learning provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a reinforcement learning scheduling model provided by an embodiment of the present application;

6 is a schematic flowchart of another distribution method provided by an embodiment of the present application;

7 is a schematic structural diagram of a distribution device provided by an embodiment of the present application;

8 is a schematic structural diagram of another distribution device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed ways

Embodiments of the present application are described below with reference to the accompanying drawings in the present application. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.

It will be understood by those skilled in the art that the singular forms "a," "an," and "the" as used herein can include the plural forms as well, unless expressly stated otherwise. It should be further understood that the terms "comprising" and "comprising" used in the embodiments of the present application mean that corresponding features can be implemented as presented features, information, data, steps, operations, elements and/or components, but do not exclude Implementations support other features, information, data, steps, operations, elements, components, and/or combinations thereof, etc., as supported in the art. It will be understood that when we refer to an element as being "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or the one element and the other element may be intervening through intervening elements Establish a connection relationship. Furthermore, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates at least one of the items defined by the term, eg "A and/or B" indicates implementation as "A", or as "A", or as "A and B" ".

In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

First, introduce and explain several terms involved in the embodiments of the present application:

(1) Public transportation: a transportation system with fixed lines and stations, such as subways or buses.

(2) Distribution resources: personnel or robots that can deliver items to designated locations, where personnel may refer to professional dispatchers or riders, or ordinary passengers, and robots may be preset distribution robots.

(3) Crowdsourcing distribution: The distribution mode that outsources the distribution tasks performed by employees in the past to non-specific public in a free and voluntary form has the advantages of simple workflow and flexible working hours. For the embodiment of the present application, the crowdsourcing distribution includes the manner in which ordinary passengers participate in the distribution of items.

(4) Relay distribution: The process of delivering a single item successively by multiple distribution resources. For example, the first distribution resource transports the item from the starting site to a transit station through the subway system, and then the subsequent distribution resources continue to deliver the item to final site. Among them, the number of times the item is allocated to the distribution resource may also be called the number of relays or the number of hops.

(5) Order: Credentials for ordering item delivery. Each item that needs to be delivered has corresponding order information. Item delivery can also be called order delivery.

The embodiment of the present application provides a public transportation-based crowdsourcing distribution method, which can be implemented by the crowdsourcing distribution system as shown in FIG. 1 . Specifically, as shown in Figure 1, the crowdsourcing distribution system includes a user terminal (lower left), a distribution resource terminal (lower right), a storage cabinet (below), and a scheduling platform (above). in:

The scheduling platform can be a cloud server platform, which can interact with the user terminal, the distribution resource terminal, and the storage cabinet based on cloud services, to store order (item) information and distribution resource information, and to schedule and allocate orders.

The user terminal may include an ordering user terminal, which is used by the user to place an order and receive information, etc., and may also include an order receiving user terminal, which is used to receive the order and provide the order items. The client can interact with the storage cabinet directly or indirectly. The user terminal may specifically be a mobile terminal used by the user, or may be a client installed on the mobile terminal, such as application software or an application applet, but is not limited thereto. In practical applications, the mobile terminal may include, but is not limited to, such as a mobile phone, a smart phone, a tablet computer, a notebook computer, a smart watch, and the like.

The distribution resource terminal can interact with the storage cabinet to receive the allocated order (item) information, open the storage cabinet to extract or store the items, and so on. The distribution resource terminal may be a mobile terminal used by ordinary passengers or riders, or a client installed on the mobile terminal. For details of the type of the distribution resource terminal, please refer to the description of the user terminal, which will not be repeated here. In addition, the distribution resource terminal can also refer to a preset robot, which can directly interact with the storage cabinet.

Storage lockers (such as smart lockers) are set up at various stations of public transportation, interact with cloud servers through Wi-Fi network or cellular network to update item information, and are used for transit and temporary storage of order items.

The crowdsourcing delivery method provided by the embodiment of the present application can allocate suitable orders to the corresponding delivery resources according to the bus routes of the delivery resources, and at the same time, with the help of the storage cabinets deployed at each station of public transportation, one order can be delivered by multiple Relay distribution of resources to achieve the effects of improving distribution efficiency, increasing distribution scope, and improving distribution security.

Among them, the distribution network based on public transportation is a low-cost distribution network and can have millions of passengers as potential delivery personnel. In some metropolitan areas, public transport is more popular than private cars and taxis because it is faster and causes less congestion and pollution. There are therefore significant advantages to using public transport as a delivery tool.

In addition, the public transportation-based crowdsourcing distribution method provided by the embodiment of the present application is very simple. The distribution resources only need to extract or store the items at the starting point and the end point of their own routes, and they can help the distribution during the process of taking public transportation without the need for Deliberately changing routes will not affect the travel arrangements of distribution resources, minimize the extra time occupied by distribution resources, and thus reduce distribution costs.

Based on this, the crowdsourcing distribution method provided by the embodiments of the present application may also be called a free-riding distribution method, which can encourage more ordinary passengers to participate as distribution resources, realize real crowdsourcing distribution, and further improve distribution efficiency.

The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below by describing several exemplary embodiments. It should be noted that the following embodiments may refer to, learn from, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be described repeatedly.

A distribution method is provided in the embodiment of the present application, as shown in FIG. 2 , the method includes:

Step S201: In response to the delivery request for any target item, perform at least one distribution to any target item in the following manner, until any target item reaches the final destination:

Wherein, the items to be delivered may refer to meals or items in a take-out order, or parcels delivered instantly in the same city, and the embodiment of the present application does not limit the types of items herein. It is understandable that items and even organisms that need to be delivered by outsourcing within the specified time are all applicable to this application and should be included within the scope of protection of this application.

In this embodiment of the present application, any target item can be delivered to the final destination through the relay distribution of the distribution resources, that is, the following steps S202 to S205 are repeatedly executed to sequentially allocate any target item to at least one distribution resource, so as to pass the allocated resources. At least one delivery resource relays the item to the final destination. The reason for the use of relay delivery is that considering the large difference in the passenger flow of different sites, sometimes choosing a transit station can often send the order to the final destination faster.

In this embodiment of the present application, the timing of repeated execution of the following steps S202 to S205 is not specifically limited. In a feasible implementation manner, the execution may be repeated every predetermined time interval, for example, the predetermined time may be 5 minutes. In other embodiments, other timings may also be set for repeated execution.

Step S202: obtaining current environment information and order information of each item in each storage cabinet;

The environmental information refers to relevant information that will affect the delivery of items, including but not limited to at least one of the following: weather information, time information, delivery resource supply information, and item delivery demand information. Wherein, time information includes but is not limited to date, working day or holiday, time point, etc.; distribution resource supply information refers to the information of distribution resources that can be dispatched and can deliver items, such as information of distribution resources waiting at the site, The information of the distribution resources in the ride between the two sites, etc.; the item distribution demand information refers to the information of the items that need to be distributed, such as the information of the items stored in the storage cabinets of the site, the distribution resources between the two sites Information on items to be carried, etc.

Compared with some existing technologies that only use transportation data to study the problem of item distribution from the perspective of supply, the embodiment of the present application creatively combines the information of distribution resources and the information of items, and handles the scheduling of orders from the perspectives of supply and demand, It can improve the practical application effect of the program.

The order information of an item refers to the relevant information related to the order that will affect the delivery of the item, including but not limited to at least one of the following: delivery origin, delivery destination, total time limit, remaining delivery time, distribution history, delivery revenue, each time Allocated shipping costs, order overtime costs, etc.

Step S203: Determine the next destination of each item in public transportation based on the current environment information and the order information of each item;

Among them, the next destination is the next site, which may also be called the next hop. It can be understood that the next destination may be the final destination of the item, or may be a transit station.

In the embodiment of the present application, the scheduling problem of determining the next destination for an item is regarded as a routing decision problem for the item, that is, the selection of the delivery route of the item.

Step S204: when detecting an item delivery instruction initiated by the delivery resource for any storage cabinet, obtain the ride destination of the delivery resource;

When the distribution resource arrives at any of the storage cabinets, you can initiate a delivery instruction for any storage cabinet through a smartphone or click on the storage cabinet, and enter its destination for the system to obtain.

Step S205: According to the next destination and the ride destination of each item in any storage cabinet, assign at least one item whose next destination is the same as the ride destination to the distribution resource, so that the distribution resource can carry at least one item to the destination. Store in a locker at the ride destination.

That is, the system will decide which item or items the delivery resource will carry based on the destination of the delivery resource and the next destination of the existing items in any storage cabinet. If the delivery resource shares the same next hop with one or some items, the item or items will be allocated to the delivery resource, and the specific carrying quantity does not exceed the preset carrying capacity of the delivery resource. The carrying capacity of the distribution resource is not specifically limited in the embodiments of the present application, and those skilled in the art can set or evaluate in real time according to the actual situation.

As an example, the process of item distribution using the distribution method provided by the embodiment of the present application is shown in FIG. 3 . For an order, multiple distribution resources can be used for distribution, and FIG. 3 takes two as an example.

There are two main factors to consider in the dispatch scheduling of each item: time constraints and total relay hops. The goal of scheduling is to deliver the item from the delivery origin O to the delivery destination D with the minimum number of relay hops within the time limit of the order (such as 1 hour).

One of the easiest ways is to only wait for those delivery resources from point O to point D, but the problem is that there may be very few delivery resources from O to D, and if you only wait for such delivery resources, it may lead to delivery timeout.

Therefore, the method proposed in the embodiment of the present application is to predict the passenger flow of the site, and then predict the delivery time of the item under different path selections, and then perform path planning according to the starting site and target site of the order, and select the most suitable one. For example, adding a transfer station A between ODs may increase the cost of delivery, but it can avoid overtime.

Delivering through multiple hops saves time to meet time constraints compared to single-hop routes, but creates additional challenges for delivery margins and estimating total delivery time. Based on this, the embodiments of the present application select the path with the least relay process under a given delivery time limit in the process of implementing relay delivery at different sites, so as to reduce delivery costs. At the same time, by predicting the passenger flow and delivery time in the subway, the delivery cost can be controlled while the overtime rate is reduced, thereby maximizing the delivery profit.

Since items and distribution resources and environments in the real world change dynamically, all factors (ie time constraints, multi-hops, profits and costs, etc.) are comprehensively considered in the embodiments of this application to improve the applicability of crowdsourcing distribution based on public transportation. sex and performance.

It should be noted that the technical solutions provided in the embodiments of the present application focus on scheduling in public transportation, that is, the starting point and the ending point of the dispatched items and distribution resources refer to stations in the public transportation route.

In practical applications, for scenarios such as takeaway delivery, the item is the meal ordered by the user. On the basis of using public transportation as the main road route, there may be branch routes that require other means of delivery, such as the delivery of meals by riders driving electric vehicles from merchants to storage cabinets at the delivery point in public transportation, and in the food When reaching the storage locker at the delivery destination in the public transport route, the meal is delivered to the user by the rider driving the electric vehicle. That is, the client can interact with the storage cabinet indirectly through the rider. In this case, a separate scheduling algorithm may be used for scheduling the branch route, which is not limited in this embodiment of the present application. In other embodiments, it is also not excluded that the merchant directly stores the meal in the storage cabinet of the delivery origin in the public transportation, or the user directly picks up the meal in the storage cabinet of the delivery destination in the public transportation route. That is, the client can directly interact with the storage cabinet. In this case, the allocation method provided by the embodiment of the present application can be directly used for scheduling. Those skilled in the art should understand that this scenario is only an example, and appropriate changes based on this example can also be applied to the present application, and should also be included in the protection scope of the present application.

The delivery method provided by the embodiment of the present application can achieve the purpose of using public transportation to deliver the goods by taking the delivery resource relay of public transportation to carry the goods to the final destination, and can reach the destination farther in the urban area at a faster speed. , that is, as the main road in the distribution system to achieve a wider range of city-wide distribution. And the distribution method based on public transportation can reduce the use of electric vehicles in distribution, and improve the safety of distribution resources to a certain extent. In addition, the distribution method provided by the embodiment of the present application is very simple, and the distribution resources only need to extract or store the items at the starting point and the end point of their own routes, and there is no need to transfer routes, so more ordinary passengers can be promoted as distribution resources. Participation can significantly improve distribution efficiency.

Considering the distribution of items, the platform needs a profit model that is used to balance the delivery time limit and the number of relay deliveries of items to determine the most suitable distribution route to minimize costs and meet time constraints. The embodiment of the present application provides a possible implementation manner, and step S203 may specifically include:

Determine the model parameters of the pre-built profit model under the profit maximum constraint. The model parameters of the profit model include the distribution revenue of each item, the number of distributions for each item, the distribution cost per distribution for each item, the number of overtime orders, and At least one of the cost of order overtime;

Based on the profit maximization constraint, the current environment information, and the order information of each item, the next destination of each item in public transportation is determined.

Among them, the distribution revenue of each item is the distribution fee that can be charged for the distribution of the item; the product of the distribution times of each item and the distribution cost of each distribution is the distribution fee that needs to be paid to all distribution resources that deliver the item; overtime orders The product of the quantity and the order overtime cost is the delivery fee that needs to be paid for all overtime orders.

Then in the whole delivery process, profit = total delivery fee charged for all items - delivery fee paid to all delivery resources that deliver all items - delivery fee paid for all overtime orders.

In a feasible implementation, it is assumed that if the order is overtime, the delivery fee charged will be returned in full, then the profit model that can be adopted is as follows:

Among them, N is the total number of items; h is the number of hops (number of times) of relay delivery of an order item; CustPay is the delivery fee charged; HopCost pays the delivery fee for each delivery resource; M is the number of overtime orders;

Understandably, higher CustPay, lower HopCost, fewer dispenses per item, and fewer overtime orders are needed to maximize the platform’s profits. Therefore, this application is implemented in the scheduling of items based on the maximum profit constraint, focusing on how to reduce the number of allocations required for each item, while delivering more items within the time limit. Based on this, it is determined that each item is in the public domain. The next destination in traffic, allowing the platform to make as much profit as possible.

The embodiment of the present application provides a possible implementation. Step S203 may specifically include: determining the next destination of each item in public transportation based on the environmental information and the order information of each item, through a reinforcement learning scheduling model, The reinforcement learning scheduling model is obtained based on the reinforcement learning algorithm.

Reinforcement Learning (RL) is a machine learning algorithm commonly used to model sequential decision-making problems. In the RL setting, as shown in Figure 4, the decision executor (Agent) will make a decision action (Action) and get some reward (Reward) each time, which can also be called reward, and the decision action will change each time Environment state (State). The environment feeds the reward of the decision-making behavior and the changed state back to the decision-making body. The learning process of reinforcement learning, that is, the decision executor learns how to choose the next decision behavior according to the environment state and reward in the interaction with the environment, so as to maximize the next reward and the total reward. In other words, reinforcement learning can select the optimal strategy (decision-making behavior) according to the set objective function for different environmental states through the learning of historical data.

Specifically, in the embodiment of the present application, the scheduling of items is regarded as the routing of items through reinforcement learning, that is, the next hop of the package is selected. Then the process is naturally formulated as a Markov decision process.

In the embodiment of this application, the environment of the reinforcement learning algorithm refers to the dynamic changes of items and distribution resources, as well as the dynamic changes of some real environments, such as weather and time.

In the embodiment of the present application, the decision executor of the reinforcement learning algorithm is an agent that executes decisions for each item respectively. In the embodiment of the present application, the scheduling platform shown in FIG. 1 acts as the meta-agent, that is, the scheduling platform acts as the decision executor Make decisions for all items in a centralized manner. Therefore, the scheduling platform can obtain the routes of all items, which is convenient for sharing local information of all items, and then guides each routing decision for each item from a global perspective.

Further, in this embodiment of the present application, the decision-making behavior of the reinforcement learning algorithm includes determining the next destination for the item, that is, the next site the item should go to among all the sites.

In this embodiment of the present application, the environmental state of the reinforcement learning algorithm includes environmental information and order information of items. Specifically, the environmental state can be divided into two types: global state (based on environmental information) and local state (based on item order information). Among them, the global state can include:

(1) Demand (ie, item delivery demand) status, which can be expressed as: one-dimensional distribution of items waiting at all sites (ie, stored in storage cabinets), and two-dimensional distribution of items being distributed between all two sites Dimensional distribution (i.e. a matrix of sites × sites).

(2) The state of supply (that is, the supply of distribution resources) can be expressed as: the one-dimensional distribution of the distribution resources that are waiting to enter all the stations, and the two-dimensional distribution of the distribution resources that are in the ride between all the two stations (that is, the stations × matrix of sites).

(3) Context state, which can be expressed as: weather, date, working day or holiday, time point, etc.

The local state of a decision-making action can be expressed as at least one of the following information of the corresponding item: delivery origin, delivery destination, total time limit, remaining delivery time, distribution history, distribution revenue, distribution cost per distribution, order overtime cost, etc. , and may not be limited to this.

In practical applications, a multi-dimensional state composed of all state variables of a decision-making behavior can be described by the state space.

In the embodiment of the present application, the reward of the reinforcement learning algorithm includes the profit after the item reaches the final destination. Since the goal of the decision executor is to maximize the expected return of each state, the reward set in the embodiment of the present application is such that as long as the time limit is met, The algorithm chooses the route with the fewest hops to maximize profit. Specifically, before the item reaches the final destination, the reward is set to 0; otherwise, it is set to the profit after the item reaches the final destination. That is, the reward is not generated until the item reaches its final destination.

In a feasible implementation, it is assumed that if the order times out, the delivery fee will be refunded in full, then the return that can be set is as follows:

是付给配送物品i的所有配送资源的配送费。

是一个指标函数，若物品i配送未超时，则

若物品i配送超时，则

Among them, CustPay _i is the delivery fee charged for item i,

is the delivery fee paid to all delivery resources for delivery item i.

is an indicator function. If the delivery of item i does not time out, then

If the delivery of item i times out, then

γ表示折扣率，γ∈(0,1]，r表示回报；即Gt表示基于折现率γ得到的折扣回报的总和。In practical applications, the value function Q(s,a)=E[G _t | _{s t} = _s ,at =a] can be used to estimate the selection of various decision-making behaviors ₍ at = a) of the expected return E, so that the decision-making behavior with the highest expected return is selected for execution. where G _t the current discounted return, optionally,

γ represents the discount rate, γ∈(0,1], and r represents the return; that is, Gt represents the sum of discounted returns based on the discount rate γ.

In this embodiment of the present application, a neural network can be used to express the value function. For example, take an approximation function Q(s, a; θ) with parameters θ, where θ can be updated using various reinforcement learning algorithms.

In a feasible implementation, a deep Q network (Deep QNetwork, DQN) including an experience replay mechanism can be selected to express the approximate value function and update the parameters. High-dimensional perceptual input (environmental state) can be processed through DQN to deal with a large number of stations, items and distribution resources. The experience playback mechanism can determine the way of storing and extracting training data during the DQN training process, avoid the impact of time series correlation on the training process, improve the utilization of training data, and further improve the efficiency of reinforcement learning.

In this embodiment of the present application, in each iteration of reinforcement learning, DQN uses the following loss function to update parameters:

L(θ)=E _{(s,a,r,s')～U(D)} [(r+γmax _a' Q(s',a';θ ^- )-Q(s,a;θ)) ² ]

Among them, U(D) is the uniform distribution of the playback buffer D. In the empirical playback, the sampled training data e _t = (s _t , at , r _t , s _{t +1} ) are stored in the playback buffer D _t = e ₁ , ..., e _t .

In the embodiment of the present application, the reinforcement learning scheduling model is obtained based on the reinforcement learning algorithm, and may specifically include:

Obtain multiple training samples, each of which includes environmental information at historical moments and order information for multiple items at historical moments;

Determine the historical environment state of the reinforcement learning algorithm according to each training sample, and determine the next destination of multiple items in public transportation at the historical moment based on each training sample, as the historical decision-making behavior of the reinforcement learning algorithm, according to the historical moment. The profit of multiple items reaching the final destination to determine the historical return of the reinforcement learning algorithm;

Reinforcement learning is performed based on historical environment states, historical decision-making behaviors and historical rewards, and a reinforcement learning scheduling model is obtained.

That is to say, the embodiments of the present application use a deep RL method to learn the most suitable route for each item from massive historical traffic data and order data, and specifically adopt different fee settings (such as CustPay and HopCost, etc.) and environmental factors (weather, time, supply and demand, etc.) to learn.

Specifically, a large amount of historical public transportation data and order data can be collected from ubiquitous mobile devices and infrastructure, and the transit time between each pair of stations can be estimated through historical traffic data, combined with historical order data. Simulation is carried out, and then a scheduling model is established through reinforcement learning. The reinforcement learning scheduling model will consider both actual factors and subsequent decisions, that is, relevant relationships will be considered in subsequent scheduling decisions.

Since the distribution method provided by the embodiments of the present application has the property of "free-riding", the distribution process does not change the vehicle or the distribution route, that is, the distribution resources do not need to take a detour, so the distribution of items can be effectively simulated based on real historical data. process, and then learn scheduling policies from historical big data.

In the embodiment of the present application, the above-mentioned profit model may also be combined with the reinforcement learning algorithm to learn the decision-making behavior of the reinforcement learning algorithm with the ultimate goal of profit maximization.

Specifically, profit models can be used to guide the reward function design of each decision-making behavior of reinforcement learning algorithms. Corresponds to the reward of the reinforcement learning algorithm above including the profit after the item reaches its final destination.

At the same time, the scheduling platform acts as a meta-agent to predict the total return (i.e. platform profit) within a predetermined time period (such as a few hours) in the future every time a decision is made, guides each decision from a more global perspective, and ultimately The goal is to maximize profits for the platform.

The embodiment of the present application provides a possible implementation manner, and step S203 may specifically include:

Determine the estimated time of arrival for each item using various routes;

The next destination of each item in public transportation is determined based on the environmental information, the order information of each item, and the estimated time of arrival corresponding to each item using various routes.

Among them, Estimated Time of Arrival (ETA) is a commonly used concept in map navigation and logistics transportation, which refers to the time when people or vehicles are expected to arrive at a certain place. In this embodiment of the present application, ETA can be used to estimate the time required for item delivery between any two sites.

In the embodiment of the present application, the delivery time of an item is divided into two parts: waiting time and delivery time. Specifically, the waiting time refers to the time that the items are stored in each storage cabinet (including the storage cabinets of the transfer station), which may be the time between the items being placed and taken out; the delivery time refers to the time that the items are carried by the distribution resources, Specifically, it may be the time between the delivery resource taking out the item and placing the item at the next destination.

When an item is carried and distributed by multiple distribution resources, both the waiting time and the delivery time consist of multiple periods of time. Specifically, the calculation formula of the total delivery time _TD is as follows:

Among them, _TW is the waiting time before each distribution of the item, _TR is the delivery time of each distribution of the item, and n is the number of times the item is distributed.

The inventors of the present application found, based on experimental observations on historical data, that both the waiting time _TW and the transit time _TR follow a Gaussian distribution. It should be noted that the delivery time is estimated based on every two stations. In theory, choosing different routes between the two stations will lead to different delivery times. However, people's choice of routes is often fixed during a specific time period. For example, during peak hours, people generally choose a journey that takes less time. Therefore, for different time periods of the day, the route selection is almost fixed, so that the delivery time also follows a Gaussian distribution.

In a feasible implementation, the parameter Gaussian (N(μ,σ ² )) can be used to model the waiting time and the delivery time. Through the Gaussian, not only the mean information (μ) of the time distribution, but also the time can be obtained. Variance information (σ ² ) of the distribution. Among them, the variance information can provide a statistical guarantee for the total delivery time.

其中，

和

分别是

的均值信息和方差信息。为运送时间T_R构建的高斯分布模型为

其中，

和

分别是

的均值信息和方差信息。基于此，每个物品的运送时间和等待时间可以拟合高斯。Specifically, the Gaussian distribution model constructed for the waiting time _TW is

in,

and

respectively

mean and variance information. The Gaussian distribution model constructed for the transit time _TR is

in,

and

respectively

mean and variance information. Based on this, the shipping time and waiting time of each item can be fitted with a Gaussian.

Further, considering that when passengers are used as delivery resources, the willingness of passengers will affect the estimation of waiting time (because not all passengers are willing to participate in the delivery of items, that is, how many passengers participate will affect the delivery time of items), using the participation rate indicator ρ to simulate the global willingness (eg ρ=0.1 means that 10% of passengers are willing to participate in the delivery of items). Specifically, the participation rate index can be determined by the distribution cost and/or the distribution time period allocated each time, and a person skilled in the art can set a specific determination method according to the actual situation, which is not specifically limited here.

A Gaussian can be fitted to the shipping time and waiting time of each item for a fixed time period and participation rate. That is, in the embodiment of the present application, determining the estimated arrival time corresponding to each item using various routes may include: according to the distribution starting point of each item, the delivery destination of each item, the preset participation rate (ρ), each item For at least one item in the delivery time period of the item, through the preset estimated time of arrival model, the delivery time and waiting time corresponding to each item using various routes are obtained; the delivery time and waiting time corresponding to each item using various routes are obtained. time, determine the estimated arrival time corresponding to each item using various routes; _wherein , the estimated arrival time model ([μW, _σW , _μR , _σR ]) is the historical delivery time and historical waiting time of multiple items obtained by Gaussian fitting.

Specifically, the ETA result can be a four-dimensional tensor [μW, _σW , _μR , σR] = _H (ρ, _t , o, d), where _μW and _σW are Gaussian parameters for estimating latency , μ _R and σ _R are Gaussian parameters for estimating the delivery time, o and d represent the source and final destination of the item delivery to the two sites, ρ represents the participation rate, and t represents the delivery time period.

In this embodiment of the present application, the purpose of performing time estimation through the ETA model is to obtain more information about time allocation, that is, a route filter can be designed according to the ETA. Considering that there are hundreds of public transportation stops in a city, it is computationally expensive to determine the next destination in public transportation for each item. Through ETA, it is possible to filter the routes over time for each item to improve the route routing performance, guaranteeing Delivery time of the item.

In the embodiment of the present application, the reinforcement learning scheduling model can also be used to determine the next destination of each item in public transportation in combination with ETA awareness (Aware) to eliminate infeasible decision-making behaviors and avoid excessive decision-making data. The training convergence is slow and the impact on routing performance. Specifically, ETA information collected offline can be used as a priori information for filtering to speed up training and ensure that the training results can better achieve on-time delivery of items.

In practical applications, the Gaussian quantization function (quantile function, ie inverse CDF (CumulativeDistribution Function, cumulative distribution function)) can be used to ensure that the actual arrival time is less than or equal to the expected arrival time The probability is p (which can be called on-time. rate parameter).

The embodiment of the present application provides a feasible implementation manner. When performing the filtering operation, the probability of the potential decision-making behavior a _j may be:

是从当前站o_t到最终目的地d_i的预计到达时间，而r_i是物品i的剩余配送时间。

的计算公式可以为：in,

is the estimated time of arrival from the current station _{o t} to the final destination di, and ri is the remaining delivery time for item _i .

The calculation formula can be:

Among them, s _j is the next destination (transit station) determined by the decision-making behavior a _j of item i at the current time t. That is, the estimated delivery time is the sum of the waiting time of the current station, the transportation time from the current station to the transfer station, the waiting time of the transfer station, and the transportation time from the transfer station to the final destination. It can be understood that if the next destination determined by the decision behavior a _j happens to be the final destination, then

的计算公式中的每一项分别使用以下高斯分位数函数计算：According to ETA results [μ _W , σ _W , μ _R , σ _R ]=H(ρ, t, o, d),

Each term in the formula is calculated using the following Gaussian quantile function:

where erf() is the error function, 0<p<1, it is understandable that a larger value of p can guarantee more items to be delivered on time, but will filter some potentially low-cost routes. Therefore, the setting of the parameter p is a trade-off between the on-time delivery rate and the profit, and those skilled in the art can set p according to the actual situation, which is not limited in this embodiment of the present application. As an example, p=0.9, ie 90% of the actual arrival times are less than or equal to the expected arrival times. Using this mechanism maximizes the total time limit of the system.

In the embodiment of the present application, as shown in FIG. 5 , the execution process of determining the next destination of each item in public transportation by the ETA-aware RL scheduling algorithm may include:

Update the global and local states of all items based on the current environment information and the order information of all items.

(Optional step) Sort all items according to the remaining delivery time of each item (for example, in ascending order, but not limited to this). Items with more urgency (with less remaining delivery time) are allocated first. For example, if the remaining delivery time is less than or equal to 0, it means that the item is overdue and will be allocated to distribution resources first.

For each item i do the following:

Calculate the value Q(s,a _j ) of all decision-making actions a _j through DQN;

Through the ETA filter, all decision-making behaviors are filtered according to the input ETA result H(ρ, t, o, d) to obtain the filtered decision-making behavior;

Choose the decision action with the highest value among the filtered decision actions according to the epsilon greedy algorithm (but still give some chance to other low probability decision actions).

After all items have determined their next destination in public transportation, order items are allocated to delivery resources according to the route.

The distribution method provided by the embodiment of the present application can realize crowdsourcing distribution based on public transportation, and introduce a profit model with practical significance, so as to maximize the profit of the platform for the distribution of goods. In order to integrate the impact of practical factors such as time limit, multi-hop delivery, passenger participation willingness and profit on order scheduling, the embodiment of the present application builds a reinforcement learning scheduling model to learn scheduling strategies from massive historical traffic data and order data. Order scheduling is formulated as a routing decision problem, i.e. choosing the next destination for item delivery. The ETA module is designed to expedite the training process and provide an overall delivery time guarantee.

Through the statistics of the delivery data corresponding to this scheme, it is found that compared with the existing distribution algorithm, the use of this scheme increases the profit margin by 40% and the delivery rate by 29%. Improved margins and delivery rates by 9% and 8%, respectively, by using an estimated time of arrival model.

In addition, since the training sample data in the embodiments of the present application comes from massive transmission data and delivery data collected from ubiquitous mobile devices and infrastructure, in addition to crowdsourcing distribution, more applications can be supported.

The embodiment of the present application also provides a distribution method, and the execution body of the method is a distribution resource terminal. As shown in FIG. 6 , the method includes:

Step S601: when detecting an item delivery instruction initiated by the delivery resource for any storage cabinet, obtain the destination of the delivery resource;

When the delivery resource arrives at any storage cabinet, an item delivery instruction for any storage cabinet can be initiated by means of a smart phone or by clicking on the storage cabinet, and the travel destination can be input.

Step S602: sending the ride destination to the dispatch platform;

The dispatching platform will determine the next destination in the public transportation for each item in any of the storage cabinets. For the specific determination method, please refer to the introduction above, which will not be repeated here.

After receiving the ride destination of the distribution resource, the scheduling platform will decide which item or items the distribution resource will carry based on the ride destination and the next destination of the existing items in any storage cabinet.

Step S603: Receive item information of at least one item whose next destination and ride destination are the same as the next destination allocated by the scheduling platform to the distribution resource;

If the distribution resource shares the same next hop with one or some items, the scheduling platform will allocate one or more items to the distribution resource, and the specific quantity to be carried does not exceed the preset carrying capacity of the distribution resource. The carrying capacity of the distribution resource is not specifically limited in the embodiments of the present application, and those skilled in the art can set it or evaluate it in real time according to the actual situation.

Step S604: Display the item information.

After the scheduling platform allocates one or more items to the distribution resources, it sends the information of the allocated items to the distribution resources for viewing, so that the distribution resources can clearly identify which item or items to take out of the storage cabinet to carry.

The distribution method of the embodiment of the present application achieves the purpose of using public transportation to distribute the articles by carrying the articles on the distribution resources of the public transportation. In the embodiment of the present application, public transportation is used as the means of delivery, which can achieve a faster speed to a destination farther in the urban area, that is, as a trunk road in the delivery system, to achieve a wider range of city-wide delivery. And the distribution method based on public transportation can reduce the use of electric vehicles in distribution, and improve the safety of distribution resources to a certain extent. In addition, the distribution method provided by the embodiment of the present application is very simple, and the distribution resources only need to extract or store items at the starting point and the end point of their own routes, and there is no need to change routes, so more ordinary passengers can be promoted as distribution resources. Participation can significantly improve distribution efficiency.

The following takes the online ordering scenario as an example to describe the operation process of the crowdsourcing distribution system shown in Figure 1:

(1) After the user orders the food online, the merchant directly or indirectly through the rider places the takeaway in the pick-up cabinet (storage cabinet) of the delivery start site of a public transportation. The online order includes the destination to which the takeaway needs to be delivered. And takeaway delivery time guarantee services, such as half an hour, 1 hour, etc.

(2) According to the implementation information of all environments and orders, the scheduling platform decides the next destination of all items (including the takeaway) in public transportation.

(3) When the passengers (or riders) participating in the delivery come to the pick-up counter, they can input their destination through a smartphone or directly in the pick-up counter. The dispatch platform decides which item or items the passenger will carry based on the passenger's input and the predetermined route of the items in the pickup cabinet. If the passenger shares the same next hop with the takeaway, the passenger will be assigned the takeaway.

(4) When the passenger arrives at the destination (not necessarily the final destination of the takeaway), the takeaway is put into the pick-up cabinet of the destination, and the task completion command is triggered. Another passenger will come and repeat the process until the takeaway is delivered to its final destination in public transport. Passengers simply pick up and drop off their packages at the source and destination of their rides, no detours. Therefore, food delivery is purely free-riding, and passengers' routes or behaviors do not change.

(5) When the takeaway arrives at the final destination in public transportation, the user can directly open the pickup cabinet to receive the takeaway, or the rider can pick up the takeaway from the pickup cabinet and deliver it to the user. If the delivery exceeds the time limit, the platform will compensate the user.

In this method, passengers or riders who take public transportation relay takeaway food to the final destination, so as to achieve the purpose of using public transportation to deliver takeaway food. Order meals further away. And the distribution method based on public transportation can reduce the use of electric vehicles in distribution, and improve the safety of distribution resources to a certain extent. In addition, the distribution method provided by the embodiment of the present application is very simple, and the distribution resources only need to extract or store items at the starting point and the end point of their own routes, and there is no need to change routes, so more ordinary passengers can be promoted as distribution resources. Participation can significantly improve distribution efficiency.

An embodiment of the present application provides a distribution device. As shown in FIG. 7 , the distribution device 70 may include: a distribution request receiving module 701 , an order information acquisition module 702 , an item destination determination module 703 , a ride destination acquisition module 704 and Allocation module 705, wherein,

The delivery request receiving module 701 is configured to receive a delivery request for any target item;

The order information acquisition module 702 is used to acquire current environment information and order information of each item in each storage cabinet;

The item destination determination module 703 is configured to determine the next destination of each item in public transportation based on the current environment information and the order information of each item;

The ride destination obtaining module 704 is configured to obtain the ride destination of the distribution resource when detecting an item distribution instruction initiated by the distribution resource for any storage cabinet;

The allocation module 705 allocates at least one item whose next destination is the same as the riding destination to the distribution resource according to the next destination and the riding destination of each item in any storage cabinet, so that the distribution resource can carry the at least one item to the destination. Store in a locker at the ride destination.

In an optional implementation manner, when the item destination determination module 703 is used to determine the next destination of each item in public transportation based on the environmental information and the order information of each item, it is specifically used for:

Determine the model parameters of the pre-built profit model under the profit maximum constraint. The model parameters of the profit model include the distribution revenue of each item, the number of distributions for each item, the distribution cost per distribution for each item, the number of overtime orders, and At least one of the cost of order overtime;

Based on the profit maximization constraint, the current environment information, and the order information of each item, the next destination of each item in public transportation is determined.

In an optional implementation manner, when the item destination determination module 703 is used to determine the next destination of each item in public transportation based on the environmental information and the order information of each item, it is specifically used for:

Based on the environmental information and the order information of each item, the next destination of each item in public transportation is determined through the reinforcement learning scheduling model. The reinforcement learning scheduling model is obtained based on the reinforcement learning algorithm;

Among them, the decision-making behavior of the reinforcement learning algorithm includes determining the next destination for the item, the environmental state of the reinforcement learning algorithm includes the environmental information and the order information of the item, and the reward of the reinforcement learning algorithm includes the profit after the item reaches the final destination.

In an optional implementation, the reinforcement learning scheduling model is obtained based on a reinforcement learning algorithm, including:

Obtain multiple training samples, each of which includes environmental information at historical moments and order information for multiple items at historical moments;

Determine the historical environment state of the reinforcement learning algorithm according to each training sample, and determine the next destination of multiple items in public transportation at the historical moment based on each training sample, as the historical decision-making behavior of the reinforcement learning algorithm, according to the historical moment. The profit of multiple items reaching the final destination to determine the historical return of the reinforcement learning algorithm;

Reinforcement learning is performed based on historical environment states, historical decision-making behaviors and historical rewards, and a reinforcement learning scheduling model is obtained.

In an optional implementation manner, when the item destination determination module 703 is used to determine the next destination of each item in public transportation based on the environmental information and the order information of each item, it is specifically used for:

Determine the estimated time of arrival for each item using various routes;

The next destination of each item in public transportation is determined based on the environmental information, the order information of each item, and the estimated time of arrival corresponding to each item using various routes.

In an optional implementation manner, when the item destination determination module 703 is used to determine the estimated arrival time corresponding to each item using various routes, it is specifically used for:

According to at least one of the delivery starting point of each item, the delivery destination of each item, the preset participation rate, and the delivery time period of each item, through the preset estimated time of arrival model, it is obtained that each item adopts various Delivery time and waiting time corresponding to the route;

Based on the delivery time and waiting time corresponding to each item using various routes, determine the estimated arrival time corresponding to each item using various routes;

Among them, the estimated time of arrival model is obtained by Gaussian fitting of the historical delivery time and historical waiting time of multiple items.

In an optional implementation manner, the environmental information includes at least one of the following: weather information, time information, distribution resource supply information, and item distribution demand information;

The order information for each item includes at least one of the following: delivery origin, delivery destination, total time limit, remaining delivery time, allocation history, delivery revenue, delivery cost per allocation, and order overtime cost.

The apparatus of the embodiments of the present application can execute the methods provided by the embodiments of the present application, and the implementation principles thereof are similar. The actions performed by each module in the apparatus of the embodiments of the present application are the same as the steps in the methods of the embodiments of the present application. Correspondingly, for the detailed functional description and beneficial effects of each module of the apparatus, reference may be made to the description in the corresponding method shown above, and details are not repeated here.

An embodiment of the present application provides a distribution device. As shown in FIG. 8 , the distribution device 80 may include: an acquisition module 801 , a transmission module 802 , a reception module 803 and a display module 805 , wherein,

The obtaining module 801 is configured to obtain the riding destination of the distribution resource when detecting an item distribution instruction initiated by the distribution resource for any storage cabinet;

The sending module 802 is configured to send the ride destination to the dispatch platform;

The receiving module 803 is configured to receive the item information of at least one item whose next destination is the same as the riding destination allocated by the scheduling platform to the distribution resource;

The display module 804 is used to display the item information.

The apparatus of the embodiments of the present application can execute the methods provided by the embodiments of the present application, and the implementation principles thereof are similar. The actions performed by each module in the apparatus of the embodiments of the present application are the same as the steps in the methods of the embodiments of the present application. Correspondingly, for the detailed functional description and beneficial effects of each module of the apparatus, reference may be made to the description in the corresponding method shown above, and details are not repeated here.

An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, where the processor executes the above computer program to implement the steps of the foregoing method embodiments.

In an optional embodiment, an electronic device is provided. As shown in FIG. 9 , the electronic device 900 shown in FIG. 9 includes: a processor 901 and a memory 903 . The processor 901 is connected to the memory 903 , for example, through a bus 902 . Optionally, the electronic device 900 may further include a transceiver 904, and the transceiver 904 may be used for data interaction between the electronic device and other electronic devices, such as data transmission and/or data reception. It should be noted that, in practical applications, the transceiver 904 is not limited to one, and the structure of the electronic device 900 does not constitute a limitation to the embodiments of the present application.

The processor 901 may be a CPU (Central Processing Unit, central processing unit), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, an application-specific integrated circuit), an FPGA (Field Programmable Gate Array, Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure. The processor 901 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

The bus 902 may include a path to transfer information between the components described above. The bus 902 may be a PCI (Peripheral Component Interconnect, Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture, Extended Industry Standard Architecture) bus or the like. The bus 902 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 9, but it does not mean that there is only one bus or one type of bus.

The memory 903 may be a ROM (Read Only Memory, read only memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory, random access memory) or other types that can store information and instructions. A dynamic storage device can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory, a CD-ROM) or other CD-ROM storage, CD-ROM storage (including compressed CDs, Laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other media that can be used to carry or store computer programs and can be read by a computer, without limitation.

The memory 903 is used for storing a computer program for executing the embodiments of the present application, and the execution is controlled by the processor 901 . The processor 901 is configured to execute the computer program stored in the memory 903 to implement the steps shown in the foregoing method embodiments.

The electronic device may refer to the above-mentioned scheduling platform, or the above-mentioned distribution resource terminal, etc., but is not limited thereto.

Embodiments of the present application provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

Embodiments of the present application further provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

It should be understood that, although the respective operation steps are indicated by arrows in the flowcharts of the embodiments of the present application, the execution order of these steps is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart may be performed in other sequences as required. In addition, some or all of the steps in each flowchart are based on actual implementation scenarios, and may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages may be executed at the same time, and each of these sub-steps or stages may also be executed at different times respectively. In scenarios with different execution times, the execution order of these sub-steps or stages may be flexibly configured according to requirements, which is not limited in this embodiment of the present application.

The above are only optional implementations of some implementation scenarios of the present application. It should be pointed out that for those skilled in the art, without departing from the technical concept of the solution of the present application, other similar solutions based on the technical concept of the present application are adopted. The implementation means also belong to the protection scope of the embodiments of the present application.

Claims (10)

1. a distribution method, is characterized in that, comprises: In response to a dispatch request for any target item, at least one allocation of the any target item is performed in the following manner until the any target item reaches the final destination: Get current environment information and order information for each item in each locker; determining the next destination of each item in public transportation based on the current environment information and the order information for each item; When detecting an item delivery instruction initiated by the delivery resource for any storage cabinet, obtain the ride destination of the delivery resource; According to the next destination of each item in any of the storage cabinets and the ride destination, the delivery resource is allocated at least one item whose next destination is the same as the ride destination, so that the delivery The resource carries the at least one item to a storage locker at the ride destination for storage. 2 . The distribution method according to claim 1 , wherein the determining the next destination of each item in public transportation based on the environmental information and the order information of each item, comprising: 3 . : Determine the model parameters of the pre-built profit model under the maximum profit constraint, and the model parameters of the profit model include the distribution revenue of each item, the number of times of distribution of each item, and the amount of each distribution of each item. at least one of delivery cost, number of overtime orders, and order overtime cost; A next destination in public transportation for each item is determined based on the profit maximization constraint, the current environment information, and the order information for each item. 3 . The distribution method according to claim 1 , wherein determining the next destination of each item in public transportation based on the environmental information and the order information of each item, comprising: 4 . : Based on the environmental information and the order information of each item, the next destination of each item in public transportation is determined through a reinforcement learning scheduling model, and the reinforcement learning scheduling model is obtained based on a reinforcement learning algorithm ; Wherein, the decision-making behavior of the reinforcement learning algorithm includes determining the next destination for the item, the environmental state of the reinforcement learning algorithm includes environmental information and the order information of the item, and the reward of the reinforcement learning algorithm includes after the item reaches the final destination profit. 4. The distribution method according to claim 3, wherein the reinforcement learning scheduling model is obtained based on a reinforcement learning algorithm, comprising: Acquiring a plurality of training samples, each training sample includes environmental information at a historical moment and order information of multiple items at the historical moment; The historical environment state of the reinforcement learning algorithm is determined according to each training sample, and based on each training sample, the next destination in public transportation of a plurality of items at the historical moment is determined as the historical decision of the reinforcement learning algorithm behavior, determining the historical return of the reinforcement learning algorithm according to the profit after the multiple items at the historical moment reach the final destination; Reinforcement learning is performed based on the historical environment state, the historical decision-making behavior and the historical reward to obtain the reinforcement learning scheduling model. 5. A distribution method, characterized in that, comprising: When detecting an item delivery instruction initiated by the delivery resource for any storage cabinet, obtain the ride destination of the delivery resource; sending the ride destination to the dispatch platform; Receive item information of at least one item whose next destination allocated to the distribution resource by the scheduling platform is the same as the riding destination; Display the item information. 6. A distribution device, characterized in that, comprising: The delivery request receiving module is used to receive a delivery request for any target item; The order information acquisition module is used to acquire the current environment information and the order information of each item in each storage cabinet; an item destination determination module, configured to determine the next destination of each item in public transportation based on the current environment information and the order information of each item; A ride destination acquisition module, configured to acquire the ride destination of the distribution resource when an item distribution instruction initiated by the distribution resource for any storage cabinet is detected; The allocation module, according to the next destination of each item in any of the storage cabinets and the riding destination, allocates at least one item whose next destination is the same as the riding destination to the distribution resource, so that The delivery resource carries the at least one item to a storage locker at the ride destination for storage. 7. A distribution device, characterized in that, comprising: an acquisition module, configured to acquire the ride destination of the distribution resource when an item distribution instruction initiated by the distribution resource for any storage cabinet is detected; a sending module, configured to send the ride destination to the dispatch platform; a receiving module, configured to receive the item information of at least one item whose next destination allocated to the distribution resource by the scheduling platform is the same as the riding destination; The display module is used to display the item information. 8. An electronic device, comprising a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to realize any one of claims 1-4 or claim 5. steps of the method described. 9. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method of any one of claims 1-4 or claim 5 are implemented. 10. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method of any one of claims 1-4 or claim 5 are implemented.

Images