CN113296925B

CN113296925B - Storage resource allocation method, device, electronic device and readable storage medium

Info

Publication number: CN113296925B
Application number: CN202010431742.5A
Authority: CN
Inventors: 李俊伟; 耿嵩
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-05-20
Filing date: 2020-05-20
Publication date: 2025-01-03
Anticipated expiration: 2040-05-20
Also published as: CN113296925A

Abstract

The disclosed embodiment discloses a storage resource allocation method, device, electronic device and readable storage medium, the storage resource allocation method comprising obtaining a first set of data resources of a specified object that is stored or may have been stored by a specified network node within a first preset time period; obtaining a second set of data resources of the specified object that is requested within a second preset time period; determining a third set based on the intersection of the first set and the second set; and allocating storage resources of the specified network node to the specified object based on the number of times each data resource in the third set is requested within the second preset time period. This technical solution can reasonably allocate storage resource quotas on network nodes, ensure storage of hotspot resources with a high probability of being accessed again, improve the rationality of storage resource use and increase the hit rate of resource access on network nodes.

Description

Storage resource allocation method and device, electronic equipment and readable storage medium

Technical Field

The disclosure relates to the technical field of computer application, and in particular relates to a storage resource allocation method, a storage resource allocation device, electronic equipment and a readable storage medium.

Background

With the development of internet technology, in order to reduce client access delay and alleviate network congestion, data resources are typically stored on network nodes. In the prior art, the traditional storage resource allocation scheme decides the storage resource quota based on the resource amount accessed by the domain name for a period of time, but the resources accessed by part of historic history cannot be accessed again in the future, and the storage spaces required by different domain names are different, so that the problems of storage resource waste and unreasonable storage resource allocation are easily caused by the traditional technical scheme.

Disclosure of Invention

In order to solve the problems in the related art, embodiments of the present disclosure provide a storage resource allocation method, apparatus, electronic device, and readable storage medium.

In a first aspect, an embodiment of the present disclosure provides a storage resource allocation method.

Specifically, the storage resource allocation method includes:

Acquiring a first set of data resources of a specified object stored or possibly stored in a specified network node in a first preset time period;

acquiring a second set of data resources of the specified object which are requested within a second preset time period;

determining a third set from an intersection of the first set and the second set;

And distributing the storage resources of the designated network node for the designated object according to the number of times that each data resource in the third set is requested in the second preset time period.

In combination with the first aspect, in a first implementation manner of the first aspect, the specified network node represents a class of network nodes with the same attribute, and/or the specified network node represents any node in the content distribution network, a network node scheduled by the load balancing server, a network node scheduled by the task processing scheduling server, and a network node scheduled by the storage resource scheduling server.

With reference to the first aspect, in a second implementation manner of the first aspect, the specified object is any one of a specified domain name, a specified service, and a specified server.

With reference to the first aspect, in a third implementation manner of the first aspect, the obtaining the first set of the data resources of the specified object, where the specified network node may possibly store in a first preset time period, includes obtaining request data of the data resources of the specified object in the first preset time period, and determining the first set according to the probability that the specified network node receives the request of the data resources of the specified object and the request data.

With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the method further includes determining the probability according to a ratio of a network bandwidth allocated to the specified object by the specified network node to a total bandwidth of the specified object.

With reference to the third implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the disclosure obtains request data of the data resource of the specified object in the first preset time period, and includes determining, according to the request data of the data resource of the specified object received by each network node belonging to the same level as the specified network node in the first preset time period, the request data of the data resource of the specified object in the first preset time period.

With reference to the first aspect, in a sixth implementation manner of the first aspect, the obtaining the second set of the data resources of the specified object, where the second set is requested in a second preset time period, includes determining the second set according to request data for the data resources of the specified object, which are received by each network node that belongs to the same level as the specified network node in the second preset time period.

With reference to the first aspect, in a seventh implementation manner of the first aspect, the allocating the storage resources of the designated network node to the designated object according to the number of times each data resource in the third set is requested in the second preset time period includes selecting N data resources with the largest number of times that each data resource in the third set is requested in the second preset time period, so that a ratio of the total number of times that each data resource in the third set is requested to the total number of times that each data resource in the third set satisfies a preset condition in the second preset time period, and allocating the storage resources of the designated network node to the designated object according to the size of the N data resources.

With reference to the seventh implementation manner of the first aspect, in an eighth implementation manner of the first aspect, the disclosure makes a ratio of a total number of times the N data resources are requested to a total number of times the data resources in the third set satisfy a preset condition, and includes making the ratio not smaller than a preset hit rate of the data resources requesting the specified object to the specified node.

With reference to the eighth implementation manner of the first aspect, in a ninth implementation manner of the first aspect, the preset hit rate is determined according to at least one or a combination of a importance level of a service related to the specified object, a level selection of the service related to the specified object, and a storage resource usage of a network node represented by the specified network node.

With reference to the first aspect, in a tenth implementation manner of the first aspect, the method further includes determining, according to the storage resources allocated by the designated node to the designated object, the storage resources allocated by the network node represented by the designated node to the designated object.

With reference to the first aspect, in an eleventh implementation manner of the first aspect, the first preset time period is a first preset time period that is forward from a specified time point, and/or the second preset time period is a second preset time period that is backward from the specified time point.

In a second aspect, in an embodiment of the present disclosure, a storage resource allocation apparatus is provided.

Specifically, the storage resource allocation apparatus includes:

A first acquisition module configured to acquire a first set of data resources of a specified object stored or likely to have been stored by a specified network node within a first preset period of time;

A second acquisition module configured to acquire a second set of data resources of the specified object that are requested within a second preset time period;

A first determination module configured to determine a third set from an intersection of the first set and the second set;

And the allocation module is configured to allocate the storage resources of the designated network node for the designated object according to the number of times that each data resource in the third set is requested in the second preset time period.

With reference to the second aspect, in a first implementation manner of the second aspect, the designated network node represents a class of network nodes with the same attribute, and/or the designated network node represents any node in the content distribution network, a network node scheduled by a load balancing server, a network node scheduled by a task processing scheduling server, and a network node scheduled by a storage resource scheduling server.

With reference to the second aspect, in a second implementation manner of the second aspect, the specified object is any one of a specified domain name, a specified service, and a specified server.

With reference to the second aspect, in a third implementation manner of the second aspect, the first acquisition module includes an acquisition sub-module configured to acquire request data of a data resource for the specified object in the first preset period of time, and a determination sub-module configured to determine the first set according to a probability that the specified network node receives a request of the data resource for the specified object and the request data.

With reference to the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect, the apparatus further includes a portion for determining the probability according to a ratio of a network bandwidth allocated to the specified object by the specified network node to a total bandwidth of the specified object.

With reference to the third implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the first obtaining submodule is configured to determine request data of a data resource of the specified object in the first preset time period according to request data of a data resource of the specified object received by each network node belonging to the same level as the specified network node in the first preset time period.

With reference to the second aspect, in a sixth implementation manner of the second aspect, the second obtaining module is configured to determine the second set according to request data of data resources for the specified object, which are received by each network node that belongs to the same level as the specified network node in the second preset time period.

With reference to the second aspect, in a seventh implementation manner of the second aspect, the portion for allocating the storage resources of the designated network node to the designated object according to the number of times each data resource in the third set is requested in the second preset time period is configured to select N data resources with the largest number of times that each data resource in the third set is requested in the second preset time period, so that a ratio of the total number of times that each data resource in the third set is requested to the total number of times that each data resource in the third set in the second preset time period satisfies a preset condition, and allocate the storage resources of the designated network node to the designated object according to the size of the N data resources.

With reference to the seventh implementation manner of the second aspect, in an eighth implementation manner of the second aspect, the disclosure makes a ratio of a total number of times the N data resources are requested to a total number of times the data resources in the third set satisfy a preset condition, and includes making the ratio not smaller than a preset hit rate of the data resources requesting the specified object to the specified node.

With reference to the eighth implementation manner of the second aspect, in a ninth implementation manner of the second aspect, the preset hit rate is determined according to at least one or a combination of the following, namely, importance degree of the service related to the specified object, grade selection of the service related to the specified object, and storage resource usage of the network node represented by the specified network node.

With reference to the second aspect, in a tenth implementation manner of the second aspect, the apparatus further includes a second determining module configured to determine, according to the storage resources allocated by the designated node to the designated object, the storage resources allocated by the network node represented by the designated node to the designated object.

With reference to the second aspect, in an eleventh implementation manner of the second aspect, the first preset time period is a first preset time period that is forward from a specified time point, and/or the second preset time period is a second preset time period that is backward from the specified time point.

In a third aspect, an embodiment of the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to perform the method steps of:

With reference to the third aspect, in a first implementation manner of the third aspect, the designated network node represents a class of network nodes with the same attribute, and/or the designated network node represents any node in the content distribution network, a network node scheduled by a load balancing server, a network node scheduled by a task processing scheduling server, and a network node scheduled by a storage resource scheduling server.

With reference to the third aspect, in a second implementation manner of the third aspect, the specified object is any one of a specified domain name, a specified service, and a specified server.

With reference to the third aspect, in a third implementation manner of the third aspect, the obtaining the first set of the data resources of the specified object, where the specified network node may possibly store in the first preset time period, includes obtaining request data of the data resources of the specified object in the first preset time period, and determining the first set according to the probability that the specified network node receives the request of the data resources of the specified object and the request data.

With reference to the third implementation manner of the third aspect, in a fourth implementation manner of the third aspect, the method further includes determining the probability according to a ratio of a network bandwidth allocated to the specified object by the specified network node to a total bandwidth of the specified object.

With reference to the third implementation manner of the third aspect, in a fifth implementation manner of the third aspect, the disclosure obtains request data of the data resource of the specified object in the first preset time period, and includes determining, according to the request data of the data resource of the specified object received by each network node belonging to the same level as the specified network node in the first preset time period, the request data of the data resource of the specified object in the first preset time period.

With reference to the third aspect, in a sixth implementation manner of the third aspect, the obtaining the second set of the data resources of the specified object, where the second set is requested in a second preset time period, includes determining the second set according to request data for the data resources of the specified object, which are received by each network node that belongs to the same level as the specified network node in the second preset time period.

With reference to the third aspect, in a seventh implementation manner of the third aspect, the allocating the storage resources of the designated network node to the designated object according to the number of times each data resource in the third set is requested in the second preset time period includes selecting N data resources with the largest number of times that each data resource in the third set is requested in the second preset time period, so that a ratio of the total number of times that each data resource in the third set is requested to the total number of times that each data resource in the third set satisfies a preset condition in the second preset time period, and allocating the storage resources of the designated network node to the designated object according to the size of the N data resources.

With reference to the seventh implementation manner of the third aspect, in an eighth implementation manner of the third aspect, the disclosure makes a ratio of the total number of requested times of the N data resources to the total number of requested times of each data resource in the third set satisfy a preset condition, and includes making the ratio not smaller than a preset hit rate of the data resource requesting the specified object to the specified node.

With reference to the eighth implementation manner of the third aspect, in a ninth implementation manner of the third aspect, the preset hit rate is determined according to at least one or a combination of a importance level of a service related to the specified object, a level selection of the service related to the specified object, and a storage resource usage of a network node represented by the specified network node.

With reference to the third aspect, in a tenth implementation manner of the third aspect, the method further includes determining, according to the storage resources allocated by the designated node to the designated object, the storage resources allocated by the network node represented by the designated node to the designated object.

With reference to the third aspect, in an eleventh implementation manner of the third aspect, the first preset time period is a first preset time period that is forward from a specified time point, and/or the second preset time period is a second preset time period that is backward from the specified time point.

In a fourth aspect, in an embodiment of the present disclosure, there is provided a readable storage medium having stored thereon computer instructions which, when executed by a processor, implement a method according to any one of the first aspect, the first implementation manner to the eleventh implementation manner.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

According to the technical scheme, the storage resources on the network node are allocated for the specified object according to the storage condition and the estimated access condition of the resources of the specified object (such as the domain name) on the network node, so that the storage resource quota on the network node can be reasonably allocated, the hot spot resources with high access probability are ensured to be stored, the use rationality of the storage resources is improved, and the hit rate of the resource access on the network node is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

FIG. 1 illustrates a flow chart of a storage resource allocation method according to an embodiment of the present disclosure;

fig. 2 shows a flowchart of step S101 of a storage resource allocation method according to an embodiment of the present disclosure;

FIG. 3 illustrates a data resource schematic diagram of a storage resource allocation method according to an embodiment of the present disclosure;

Fig. 4 illustrates a schematic diagram of a preset time period of a storage resource allocation method according to an embodiment of the present disclosure;

FIG. 5 illustrates a block diagram of a storage resource allocation apparatus according to an embodiment of the present disclosure;

FIG. 6 shows a block diagram of a first acquisition module of a storage resource allocation device according to an embodiment of the present disclosure;

fig. 7 shows a block diagram of an electronic device according to an embodiment of the disclosure;

fig. 8 shows a schematic diagram of a computer system suitable for use in implementing a storage resource allocation method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, portions irrelevant to description of the exemplary embodiments are omitted in the drawings.

In this disclosure, it should be understood that terms such as "comprises" or "comprising," etc., are intended to indicate the presence of features, numbers, steps, acts, components, portions, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, acts, components, portions, or combinations thereof are present or added.

In addition, it should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In the prior art, the traditional storage resource allocation scheme decides the storage resource quota based on the resource amount accessed by the domain name for a period of time, but the resources accessed by part of historic history cannot be accessed again in the future, and the storage spaces required by different domain names are different, so that the problems of storage resource waste and unreasonable storage resource allocation are easily caused by the traditional technical scheme.

In view of the above drawbacks, the technical solution provided by the embodiments of the present disclosure includes obtaining a first set of data resources of a specified object that may be stored by a specified network node in a first preset time period, obtaining a second set of data resources of the specified object that are requested in a second preset time period, determining a third set according to an intersection of the first set and the second set, and allocating storage resources of the specified network node to the specified object according to a number of times each data resource in the third set is requested in the second preset time period.

Fig. 1 illustrates a flowchart of a storage resource allocation method according to an embodiment of the present disclosure. As shown in fig. 1, the storage resource allocation method includes the following steps S101 to S104:

in step S101, a first set of data resources of a specified object stored or possibly stored in a specified network node in a first preset time period is obtained;

in step S102, a second set of data resources of the specified object requested within a second preset time period is acquired;

In step S103, determining a third set from the intersection of the first set and the second set;

In step S104, according to the number of times each data resource in the third set is requested in the second preset time period, the storage resource of the designated network node is allocated to the designated object.

According to an embodiment of the present disclosure, the designated network node represents a class of network nodes having the same attribute and/or the designated network node represents any of the network nodes in the content distribution network, the network nodes scheduled by the load balancing server, the network nodes scheduled by the task processing scheduling server, and the network nodes scheduled by the storage resource scheduling server.

For some specified object (e.g., a specified domain name), the bandwidths allocated to it by different network nodes are different, e.g., a first class of network nodes allocates 10G bandwidth, a second class of network nodes allocates 20G bandwidth, a third class of network nodes allocates 30G bandwidth, then the specified network node may represent a network node of 10G bandwidth, or a network node of 20G bandwidth, or a network node of 30G bandwidth.

The designated node may also represent a CDN (Content Delivery Network ) network node that is a cache server employed by the content delivery network that is distributed in a relatively centralized region or network of users, that may cache resources from an origin server and send the cached resources to the users in response to user requests. When the resource requested by the user is not cached on the CDN node, the CDN node requests the resource from the source server, caches the resource from the source server and forwards the resource to the client.

In addition to the CDN network nodes mentioned above, the designated node may also represent a network node scheduled by a load balancing server, or a network node scheduled by a task processing scheduling server, or a network node scheduled by a storage resource scheduling server.

According to the embodiments of the present disclosure, in application scenarios such as load balancing, task processing scheduling, and storage resource scheduling, the problem of allocating storage resources for a specified object on different network nodes may be involved. In any application scenario described above, a specified network node may be used to represent a class of network nodes having the same attribute, so as to apply the technical solution according to the embodiments of the present disclosure. The technical scheme provided by the embodiment of the disclosure is used for load balancing application scenes, storage resources can be reasonably allocated for network nodes scheduled by a load balancing server, task processing scheduling application scenes can be used for reasonably allocating storage resources for network nodes scheduled by a task processing scheduling server, and storage resources can be reasonably allocated for network nodes scheduled by a storage resource scheduling server.

According to embodiments of the present disclosure, the designated node may also be a real network node.

According to the embodiment of the disclosure, the storage resources are allocated to the class of nodes represented by the designated node according to the storage resources allocated to the designated node. For example, if an X-byte storage resource is allocated to a designated node representing "a network node to which a certain domain name is allocated 10G bandwidth", an X-byte storage resource is also allocated accordingly to "a network node to which a certain domain name is allocated 10G bandwidth".

According to an embodiment of the present disclosure, the specified object is any one of a specified domain name, a specified service, a specified server. The bandwidth allocated to different designated objects varies according to the designated network node, the designated objects being of different magnitudes in the designated network node. For example, when the specified object is a specified domain name, the specified network node allocates a domain name 10G bandwidth to a domain name B and allocates a domain name 20G bandwidth to the specified network node, and the magnitude of the domain name B in the specified node is greater than the magnitude of the domain name a. In addition, the total bandwidth of the specified object is the sum of the bandwidths allocated to the specified object by all the network nodes, for example, the specified object is a specified domain name, two network nodes are allocated to the specified object, the first network node is allocated to the specified domain name 10G bandwidth, the second network node is allocated to the specified domain name 20G bandwidth, and the total bandwidth of the specified domain name is 30G.

According to the embodiment of the disclosure, the first preset time period is a first preset time period from a designated time point forward, and/or the second preset time period is a second preset time period from the designated time point backward. The specified time point is a time point before the current time, for example, the specified time point is 12:00 before 5 days, at this time, the first preset time period is a period of time before 12:00 before 5 days, and the second preset time period is a period of time after 12:00 before 5 days.

According to the embodiment of the disclosure, the future network access and storage conditions are predicted by specifying the network access and storage conditions at the time point. The first preset time period before the designated time point is used for simulating the situation before the time point to be predicted, and the second preset time period after the designated time point is used for simulating the situation after the time point to be predicted.

According to embodiments of the present disclosure, the data resources may include any one or more of video data, audio data, text data, image data, web page data, and the like. According to embodiments of the present disclosure, the set of data resources may include { video data a, video data B, audio data C, text data D, &..the first set of data resources.

According to an embodiment of the present disclosure, when a designated node is a real network node, a first set of data resources of a designated object stored by the designated node for a first preset period of time may be acquired in step S101 according to a log of the designated node.

According to an embodiment of the present disclosure, when the designated node is a virtual node representing a real network node, step S101 obtains a first set of data resources that a class of real network nodes represented by the designated node may have stored in a first preset period of time.

According to an embodiment of the present disclosure, a third set is determined from an intersection of the first set and the second set, the third set being a set of data resources in the first set that are requested within a second preset time period. The set is used to simulate data resources that will be requested in the future from among the data resources cached by the designated network node at the point in time to be predicted.

Ideally, the designated network node caches all data resources in the third set such that all data resources that are requested in the future are stored on the designated network node. But to more efficiently utilize the storage resources, only the data resources that are requested a large number of times may be cached to achieve a reasonable utilization of the storage resources.

According to the embodiment of the disclosure, the storage resources on the network node are allocated to the specified object according to the storage condition and the estimated access condition of the resources of the specified object (for example, domain name) on the network node, so that the storage resource quota on the network node can be reasonably allocated, the hot spot resources with high requested probability are ensured to be stored, the use rationality of the storage resources is improved, and the hit rate of the data resource access request on the network node is improved.

Fig. 2 shows a flowchart of step S101 of a storage resource allocation method according to an embodiment of the present disclosure. As shown in fig. 2, the obtaining a first set of data resources of a specified object that may be stored by the specified network node in a first preset period of time includes the following steps S201 to S202:

In step S201, request data of a data resource for the specified object in the first preset time period is acquired;

in step S202, the first set is determined according to the probability that the specified network node receives a request for the data resource of the specified object and the request data.

According to an embodiment of the disclosure, the probability is determined from a ratio of a network bandwidth allocated to the specified object by the specified network node to a total bandwidth of the specified object. In order to obtain the first set, firstly obtaining request data of all requests for data resources of a specified object received by a network where a specified node is located in a first preset time period, and then calculating the data resources possibly stored by the specified node according to the probability that the specified network node receives the request for the data resources of the specified object and the request data. For example, the total bandwidth of the designated object is 100G, and the bandwidth allocated to the object by the designated node is 10G, so that each access request is routed to the designated node with a probability of 1/10, and the access request to be routed to the designated node is calculated according to the probability and all the access requests for the resources of the designated object received by the network where the designated node is located in the first preset time period. After the network node receives the access request, if the requested network data is not stored, the network data is read back from the source and then stored to the node, so if the network node receives the request, the network node considers that the network node stores the corresponding data, and therefore the data resources possibly stored by the designated network node can be determined through the steps.

According to the embodiment of the disclosure, acquiring the request data of the data resource of the specified object in the first preset time period comprises determining the request data of the data resource of the specified object in the first preset time period according to the request data of the data resource of the specified object received by each network node belonging to the same level with the specified network node in the first preset time period. For example, if the specified object is a specified domain name and the specified network node is a primary node in the CDN network, the requests for the data resources of the specified domain name, which are received by all primary nodes in the CDN network, are counted in a first preset time period, and the requests for the data resources of the domain name in the first preset time period are obtained. For another example, assuming that the specified object is a specified domain name and the specified network node is a secondary node in the CDN network, the requests for the data resources of the specified domain name, which are received by all the secondary nodes in the CDN network, are counted in a first preset time period, and the requests for the data resources of the domain name in the first preset time period are the request data of the data resources of the domain name. In the CDN network, the secondary node is an upper node of the primary node, when the primary node does not store the data resource requested by the client, the primary node can request the data resource to the secondary node, and if the secondary node stores the data resource, the secondary node can return the data resource to the primary node. If the secondary node does not store the data resource either, the secondary node may request the data resource from its upper node or from the data source server, and after obtaining the data resource, cache the data resource locally and send it to the primary node.

According to an embodiment of the present disclosure, when determining the second set, if the designated network node is a primary node, the requests for the data resources of the designated object received by all primary nodes in the network are counted, and if the designated network node is a secondary node, the requests for the data resources of the designated object received by all secondary nodes in the network are counted, so as not to cause repeated counting.

According to the embodiment of the disclosure, the obtaining the second set of the data resources of the specified object, which are requested in the second preset time period, comprises determining the second set according to the request data of the data resources of the specified object, which are received by each network node belonging to the same level as the specified network node in the second preset time period.

By determining the second set according to the scheme, all request data aiming at the data resource of the appointed object in the network where the appointed node is located in a second preset time period can be obtained. For example, assuming that the specified object is a specified domain name, the specified node is a primary node of the CDN network, and the CDN has ten primary nodes for distributing data resources of the specified domain name, counting all requests, received by the ten primary nodes in a second preset time period, of data resources of the specified domain name, and taking a set of data resources corresponding to the all requests as a second set. For another example, assuming that the specified object is a specified domain name, the specified node is a secondary node of the CDN network, and the CDN has three secondary nodes for distributing data resources of the specified domain name, all requests, received by the three secondary nodes in a second preset time period, of the data resources of the specified domain name are counted, and a set of data resources corresponding to the all requests is taken as a second set.

According to the embodiment of the disclosure, the allocating the storage resources of the designated network node to the designated object according to the number of times each data resource in the third set is requested in the second preset time period includes selecting N data resources with the largest number of times of being requested according to the number of times each data resource in the third set is requested in the second preset time period, so that the ratio of the total number of times each data resource is requested to the total number of times each data resource is requested in the third set in the second preset time period meets a preset condition, and allocating the storage resources of the designated network node to the designated object according to the size of each data resource.

According to an embodiment of the present disclosure, the number of times each data resource in the third set is requested in the second preset time period refers to a total number of requests for each data resource received by all network nodes belonging to the peer network node with the designated node in the network where the designated node is located. For example, assuming that the designated network node is a primary node of the CDN network, where the CDN network has ten primary nodes for distributing data resources from the designated object, the number of times each data resource in the third set is requested in the second preset time period refers to a sum of requests for the data resource of the designated object received by the ten primary nodes in the second preset time period for each data resource in the third set.

According to an embodiment of the present disclosure, the ratio of the total number of requested times of the N data resources to the total number of requested times of each data resource in the third set satisfies a preset condition, including making the ratio not smaller than a preset hit rate of the data resource requesting the specified object to the specified node. The hit rate is the probability that the resource requested by the appointed object is exactly the resource actually stored by the appointed network node, and the hit rate directly determines the size of the historical resource source-returning quantity. In the application, the historical resource back-source refers to that when the resource accessed by the appointed object is not stored or stored once but is eliminated by the cache when accessed, the back-source re-reads the network data. If the hit rate increases, the historical resource source-returning amount decreases, representing that most of the re-accessed resources are successfully stored.

According to the embodiment of the disclosure, the preset hit rate is determined according to at least one or more of the following combination of importance degree of the service related to the specified object, grade selection of the service related to the specified object and storage resource use condition of the network node represented by the specified network node.

For example, when the importance level of the service related to the specified object is high, a high preset hit rate may be set, whereas when the importance level of the service related to the specified object is low, a low preset hit rate may be set.

The level selection of the service associated with the specified object may be, for example, a cost selection of the user. For example, when the user selects a service related to a specified object with a higher fee, a higher preset hit rate may be set, whereas when the user selects a service related to a specified object with a lower fee, a lower preset hit rate may be set.

For example, a higher preset hit rate may be set when the storage resources of the network node represented by the designated network node are more abundant, whereas a lower preset hit rate may be set when the storage resources of the network node represented by the designated network node are less abundant.

According to embodiments of the present disclosure, the preset hit rate may be set by manual adjustment or in an adaptive adjustment manner.

According to an embodiment of the disclosure, the method further includes determining, according to the storage resources allocated by the designated node to the designated object, the storage resources allocated by the network node represented by the designated node to the designated object.

When the storage resources are allocated, firstly acquiring the number of times that the data resources in the third set are requested in the second preset time period, and then acquiring N data resources with the largest number of times that the data resources are requested, so that the ratio of the total number of times that the N data resources are requested to the total number of times that all the data resources in the third set are requested is greater than or equal to the preset hit rate. The N data resources with the largest request are the data resources which are needed to be stored for the appointed object by the network node represented by the appointed node.

According to an embodiment of the present disclosure, after determining the N data resources, the storage resources on the designated node may be allocated to the designated object according to the size of the N data resources.

Fig. 3 illustrates a data resource schematic diagram of a storage resource allocation method according to an embodiment of the present disclosure.

Fig. 4 illustrates a schematic diagram of a preset time period of a storage resource allocation method according to an embodiment of the present disclosure.

As shown in fig. 3, an ellipse a represents a first set of data resources of a specified object stored or likely to be stored in a specified network node in a first preset time period, an ellipse B represents a second set of data resources of a specified object requested in a second preset time period, and a region C where the ellipse a coincides with the ellipse B is a third set, that is, an intersection of the first set and the second set. According to the embodiment of the disclosure, for the case that the designated object is a virtual node representing a class of nodes, request data of data resources for the designated object in a first preset time period is firstly obtained, and then the probability that the designated network node receives the data resource request for the designated object is calculated according to the total bandwidth of the designated object and the bandwidth allocated to the object by the designated node, so that the data resources possibly stored by the designated node in the first preset time period are determined as a first set. Or for the case that the designated network node is a real node, acquiring the real storage condition of the designated network node on the data resource of the designated object in the first preset time period, and correspondingly determining the first set as the set of the data resource of the designated object stored on the designated network node in the first preset time period. And then, acquiring request data of the whole network for the data resource of the specified object in a second preset time period as a second set. And obtaining a third set by taking the intersection of the first set and the second set, wherein the third set represents the data resources which are requested in the second set in the second preset time period.

The hatched portion of the area C in fig. 3 shows that when the designated node requests a preset hit rate of 99% of the data resources of the designated object, the network node allocates the storage resources for the designated object. When the storage resources are allocated, firstly, the number of times of data resources in a third set in a second preset time period is acquired, then N data resources with the largest number of times of requests are acquired, so that the ratio of the total number of times of the N data resources being requested to the total number of times of all data resources in the third set is greater than or equal to a preset hit rate (for example, 99%), and at this time, the N data resources with the largest number of requests are the storage resources allocated for the appointed object by the network node represented by the appointed node.

According to the technical scheme of the embodiment of the invention, the storage resources required by the appointed object on one node are accurately calculated, so that most of hot spot resources can be ensured to be stored, meanwhile, the waste of the storage resources is reduced, and the use rationality of the storage resources is improved.

Fig. 5 illustrates a block diagram of a storage resource allocation apparatus according to an embodiment of the present disclosure. The apparatus may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 5, the apparatus 500 for a terminal device includes a first acquisition module 510, a second acquisition module 520, a first determination module 530, and an allocation module 540.

The first obtaining module 510 is configured to obtain a first set of data resources of a specified object that is stored or may have been stored in a specified network node during a first preset period of time;

the second obtaining module 520 is configured to obtain a second set of data resources of the specified object that are requested within a second preset time period;

the first determining module 530 is configured to determine a third set from an intersection of the first set and the second set;

The allocation module 540 is configured to allocate storage resources of the specified network node for the specified object according to the number of times each data resource in the third set is requested within the second preset time period.

According to an embodiment of the present disclosure, the designated network node represents a class of network nodes having the same attribute and/or the designated network node represents any of the network nodes in the content distribution network, the network nodes scheduled by the load balancing server, the network nodes scheduled by the task processing scheduling server, and the network nodes scheduled by the storage resource scheduling server. For some specified object (e.g., a specified domain name), the bandwidths allocated to it by different network nodes are different, e.g., a first class of network nodes allocates 10G bandwidth, a second class of network nodes allocates 20G bandwidth, a third class of network nodes allocates 30G bandwidth, then the specified network node may represent a network node of 10G bandwidth, or a network node of 20G bandwidth, or a network node of 30G bandwidth. In addition, the designated node may also represent a CDN (Content Delivery Network ) network node, which is a cache server employed by the content delivery network and distributed in a relatively centralized region or network of users, that may cache resources from an origin server and send the cached resources to the users in response to user requests. When the resource requested by the user is not cached on the CDN node, the CDN node requests the resource from the source server, caches the resource from the source server and forwards the resource to the client.

According to an embodiment of the present disclosure, when the designated node is a real network node, a first set of data resources of a designated object stored by the designated node during a first preset period of time may be acquired at the first acquisition module 510 according to a log of the designated node.

According to an embodiment of the present disclosure, when the designated node is a virtual node representing a real network node, the first obtaining module 510 is configured to obtain a first set of data resources that may have been stored by a class of real network nodes represented by the designated node during a first preset time period.

Fig. 6 shows a block diagram of the first acquisition module 510 of the storage resource allocation apparatus according to an embodiment of the present disclosure. As shown in fig. 6, the first obtaining module 510 includes an obtaining sub-module 610 and a determining sub-module 620:

The acquiring submodule 610 is configured to acquire request data of a data resource for the specified object in the first preset time period;

The determination sub-module 620 is configured to determine the first set based on the probability that the specified network node received a request for the data resource of the specified object and the request data.

According to an embodiment of the present disclosure, the obtaining sub-module 610 is configured to determine, according to request data for a data resource of the specified object received by each network node belonging to the same level as the specified network node in the first preset time period, request data for the data resource of the specified object in the first preset time period. For example, if the specified object is a specified domain name and the specified network node is a primary node in the CDN network, the requests for the data resources of the specified domain name, which are received by all primary nodes in the CDN network, are counted in a first preset time period, and the requests for the data resources of the domain name in the first preset time period are obtained. For another example, assuming that the specified object is a specified domain name and the specified network node is a secondary node in the CDN network, the requests for the data resources of the specified domain name, which are received by all the secondary nodes in the CDN network, are counted in a first preset time period, and the requests for the data resources of the domain name in the first preset time period are the request data of the data resources of the domain name. In the CDN network, the secondary node is an upper node of the primary node, when the primary node does not store the data resource requested by the client, the primary node can request the data resource to the secondary node, and if the secondary node stores the data resource, the secondary node can return the data resource to the primary node. If the secondary node does not store the data resource either, the secondary node may request the data resource from its upper node or from the data source server, and after obtaining the data resource, cache the data resource locally and send it to the primary node.

According to an embodiment of the present disclosure, the second obtaining module 520 is configured to determine the second set according to request data for the data resource of the specified object received by each network node belonging to the same level as the specified network node in the second preset time period.

According to an embodiment of the present disclosure, the allocation module 540 is configured to select N data resources with the largest number of times of being requested according to the number of times each data resource in the third set is requested in the second preset time period, so that a ratio of the total number of times the N data resources are requested to the total number of times each data resource in the third set satisfies a preset condition in the second preset time period, and allocate the storage resources of the designated network node to the designated object according to the size of the N data resources.

According to embodiments of the present disclosure, the preset hit rate may be set by manual adjustment or in an adaptive adjustment manner. According to an embodiment of the disclosure, the apparatus further includes a second determining module configured to determine, according to the storage resources allocated by the designated node for the designated object, the storage resources allocated by the network node represented by the designated node for the designated object.

When the storage resources are allocated, firstly acquiring the number of times that the data resources in the third set are requested in the second preset time period, and then acquiring N data resources with the largest number of times that the data resources are requested, so that the ratio of the total number of times that the N data resources are requested to the total number of times that all the data resources in the third set are requested is greater than or equal to the preset hit rate. And finally, the N data resources with the largest request are storage resources allocated for the appointed object by the network node represented by the appointed node.

The present disclosure also discloses an electronic device, and fig. 7 shows a block diagram of the electronic device according to an embodiment of the present disclosure.

As shown in fig. 7, the electronic device 700 includes a memory 701 and a processor 702, wherein,

The memory 701 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 702 to implement the method steps of:

According to an embodiment of the disclosure, the designated network node represents a class of network nodes having the same attribute, the designated network node represents any of the network nodes in the content distribution network, the network nodes scheduled by the load balancing server, the network nodes scheduled by the task processing scheduling server, and the network nodes scheduled by the storage resource scheduling server.

According to an embodiment of the present disclosure, the specified object is any one of a specified domain name, a specified service, a specified server.

According to an embodiment of the disclosure, the one or more computer instructions are further executable by the processor to perform method steps of obtaining a first set of data resources of a specified object that a specified network node may have stored during a first preset time period, comprising obtaining request data for the data resources of the specified object during the first preset time period, determining the first set according to a probability that the specified network node received a request for the data resources of the specified object and the request data.

According to an embodiment of the present disclosure, the one or more computer instructions are further executable by the processor to perform the method steps of determining the probability based on a ratio of a network bandwidth allocated to the specified object by the specified network node to a total bandwidth of the specified object.

According to an embodiment of the present disclosure, the one or more computer instructions are further executable by the processor to perform the method steps of determining request data for the data resource of the specified object within the first preset time period based on request data for the data resource of the specified object received by each network node belonging to the same level as the specified network node within the first preset time period.

According to an embodiment of the present disclosure, the ratio of the total number of requested times of the N data resources to the total number of requested times of each data resource in the third set satisfies a preset condition, including making the ratio not smaller than a preset hit rate of the data resource requesting the specified object to the specified node.

According to an embodiment of the present disclosure, the one or more computer instructions are further executable by the processor to perform method steps of determining, based on storage resources allocated by the designated node to the designated object, storage resources allocated by a network node represented by the designated node to the designated object.

According to the embodiment of the disclosure, the first preset time period is a first preset time period from a designated time point forward, and/or the second preset time period is a second preset time period from the designated time point backward.

As shown in fig. 8, the computer system 800 includes a processing unit 801 that can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the system 800 are also stored. The processing unit 801, the ROM802, and the RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

Connected to the I/O interface 805 are an input section 806 including a keyboard, a mouse, and the like, an output section 807 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like, a storage section 808 including a hard disk, and the like, and a communication section 809 including a network interface card such as a LAN card, a modem, and the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed. The processing unit 801 may be implemented as a processing unit CPU, GPU, TPU, FPGA, NPU or the like.

In particular, according to embodiments of the present disclosure, the methods described above may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method described above. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules referred to in the embodiments of the present disclosure may be implemented in software or in programmable hardware. The units or modules described may also be provided in a processor, the names of which in some cases do not constitute a limitation of the unit or module itself.

As another aspect, the present disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or the computer system in the above-described embodiment, or may be a computer-readable storage medium that exists alone and is not assembled into the device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention referred to in this disclosure is not limited to the specific combination of features described above, but encompasses other embodiments in which any combination of features described above or their equivalents is contemplated without departing from the inventive concepts described. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Claims

1. A storage resource allocation method, comprising:

according to the number of times that each data resource in the third set is requested in the second preset time period, allocating the storage resource of the designated network node for the designated object, including:

According to the number of times that each data resource in the third set is requested in the second preset time period, selecting N data resources with the largest number of times that each data resource is requested, so that the ratio of the total number of times that each data resource in the third set is requested to the total number of times that each data resource in the third set meets a preset condition in the second preset time period;

And distributing the storage resources of the designated network node for the designated object according to the size of the N data resources.

2. The method according to claim 1, characterized in that:

the designated network node represents a class of network nodes having the same attribute, and/or

The designated network node represents any of the network nodes in the content distribution network, the network nodes scheduled by the load balancing server, the network nodes scheduled by the task processing scheduling server, and the network nodes scheduled by the storage resource scheduling server.

3. The method of claim 1, wherein the specified object is any one of a specified domain name, a specified service, a specified server.

4. The method of claim 1, wherein the obtaining a first set of data resources of the specified object that the specified network node may have stored within the first preset time period comprises:

acquiring request data of a data resource aiming at the specified object in the first preset time period;

The first set is determined based on the probability that the specified network node receives a request for the data resource of the specified object and the request data.

5. The method as recited in claim 4, further comprising:

and determining the probability according to the ratio of the network bandwidth allocated to the specified object by the specified network node to the total bandwidth of the specified object.

6. The method of claim 4, wherein obtaining the request data for the data resource of the specified object for the first preset time period comprises:

and determining the request data of the data resource of the specified object in the first preset time period according to the request data of the data resource of the specified object received by each network node belonging to the same level with the specified network node in the first preset time period.

7. The method of claim 1, wherein the obtaining the second set of data resources of the specified object that are requested within a second preset time period comprises:

And determining the second set according to the request data of the data resource aiming at the appointed object, which is received by each network node belonging to the same level with the appointed network node in the second preset time period.

8. The method of claim 1, wherein the ratio of the total number of requested times of the N data resources to the total number of requested times of each data resource in the third set satisfies a preset condition, comprising making the ratio not less than a preset hit rate for the data resource requesting the specified object for the specified network node.

9. The method of claim 8, wherein the predetermined hit rate is determined based on at least one or a combination of importance of the service associated with the specified object, a ranking selection of the service associated with the specified object, and storage resource usage of the network node represented by the specified network node.

10. The method as recited in claim 1, further comprising:

And determining the storage resources allocated for the specified object by the network node represented by the specified network node according to the storage resources allocated for the specified object by the specified network node.

11. The method according to claim 1, characterized in that:

The first preset time period is a first preset time period from a designated time point onwards, and/or

The second preset time period is a second preset time period rearward from the specified time point.

12. A storage resource allocation apparatus, comprising:

The system comprises a first acquisition module, a second acquisition module, a first storage module and a second storage module, wherein the first acquisition module is configured to acquire a first set of data resources of a specified object stored or possibly stored in a specified network node in a first preset time period;

An allocation module configured to allocate storage resources of the specified network node to the specified object according to the number of times each data resource in the third set is requested within the second preset time period, including:

13. The apparatus as recited in claim 12, further comprising:

And the second determining module is configured to determine the storage resources allocated for the specified object by the network node represented by the specified network node according to the storage resources allocated for the specified object by the specified network node.

14. An electronic device comprising a memory and a processor, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to perform the method steps of any of claims 1-11.

15. A readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the method steps of any of claims 1-11.