CN106850817A - A kind of download management equipment, method and data downloading system - Google Patents

Abstract

The present invention discloses a download management device, method and data download system, wherein the download management device includes: a buffer configured to cache each file obtained from a data source node and the content abstract of each file, the content of the file The abstract is the data generated by specific data conversion on the content of the file; the parser is configured to obtain the file download request from the client device, and parse out the content abstract of the requested file according to the download address of the file download request; the finder, configured to look in the cache based on the content summary of the file and, if found, transmit the requested file to the client device; The data source node obtains the requested file, transmits it to the client device, and provides the requested file to the cache for caching.

Description

A download management device, method and data download system

technical field

The present invention relates to the technical field of data download, in particular to a download management device used in edge nodes, a data download system, and a download management method used in the data download system.

Background technique

Existing CDN (Content Delivery Network, Content Distribution Network) improves data access speed by distributing data to each edge node and shortening the distance with the client. The basic idea is to avoid bottlenecks and links on the Internet that may affect the speed and stability of data transmission as much as possible, so as to make content transmission faster and more stable. The CDN system can redirect the user's request to the service node closest to the user in real time according to comprehensive information such as network traffic, connection of each node, load status, distance to the user, and response time.

However, since each existing CDN node caches all the files in the data source as much as possible, it will bring a large storage cost. Moreover, the content of some files has not changed, only the download address. If the URL of the file has changed, according to the existing CDN caching method, multiple files with the same content and different URLs will be cached in one node, that is, duplicate Data caching further leads to higher storage costs.

Contents of the invention

In view of the above problems, the present invention is proposed to provide a download management device used in an edge node, a data download system, and a download management method used in a data download system that overcome the above problems or at least partially solve the above problems.

According to one aspect of the present invention, a download management device used in an edge node is provided, including: a buffer configured to cache each file obtained from a data source node and a content summary of each file, and the content summary of the file is The data generated by specific data conversion is applied to the content of the file; the parser is configured to obtain the file download request from the client device, and parses out the content summary of the requested file according to the download address of the file download request; the finder is configured Searching for the requested file in the cache according to the content summary of the requested file parsed by the parser, and if found, transmitting the requested file to the client device; When the server does not find the requested file in the cache, it obtains the requested file from the data source node related to the requested file, transmits it to the client device, and provides the requested file to the cache for caching.

Optionally, the origin returner is adapted to obtain each part of the requested file from the data source node part by part, and transmit the obtained part of the requested file to the client device at the same time, until the requested file is completely obtained until.

Optionally, the back-to-source device includes: a data source query module configured to query the data source nodes related to the requested file according to the known back-to-source table and the download address of the requested file; the direct back-to-source module is configured to When the edge node is the edge node of the first operator network, and the requested file-related data source nodes include the data source node of the first operator network, directly from the data source node of the first operator network The node obtains the requested file; the proxy back-to-source module is configured to be the edge node of the first operator network when the edge node is the edge node of the first operator network, and the data source node related to the requested file is the data of the second operator network In the case of the source node, the requested file is obtained from the data source node related to the second operator network through the proxy cluster from the first operator network to the second operator network; Or after the proxy back-to-source module obtains the requested file, it notifies the cache to cache the requested file.

Optionally, the content summary of the file includes: one type of data in the sha series of secure hash calculations, or one type of data in the MD series of information summary calculations.

Optionally, the download management device is a varniash cache server.

According to yet another aspect of the present invention, a data download system is provided, including an edge node, a proxy cluster that provides information interaction services between different operator networks, and a data source node, wherein the edge node includes the above-mentioned download management equipment.

Optionally, it also includes: the above-mentioned management device for providing a file download address, the download address of the file download request received by the edge node is provided by the management device for providing the file download address.

Optionally, the agent clusters that provide information exchange services between different operator networks include agent clusters from the first operator network to the second operator network, and agent clusters from the second operator network to the first operator network, Wherein, the proxy clusters from the first operator network to the second operator network include: an edge-side first operator network proxy cluster configured to receive a file download request from an edge node of the first operator network, and send a file download request to the first The edge node of the operator network returns the requested file; the transmission channel is configured to transmit information from the first operator network agent cluster on the edge side to the second operator network agent cluster on the source side, and from the second operator network agent cluster on the source node side The network proxy cluster transmits information to the first operator network proxy cluster on the edge side; and the second operator network proxy cluster on the source node side is configured to download the file received from the edge side first operator network proxy cluster through the transmission channel Request, send a file download request to the data source node related to the second operator network, and receive the requested file returned by the data source node related to the second operator network, and transmit it to the edge-side first operator network through the transmission channel agent cluster;

The proxy cluster from the second operator network to the first operator network includes: an edge-side second operator network proxy cluster configured to receive a file download request from an edge node of the second operator network, and send a file download request to the second operator network The edge node of the network returns the requested file; the transmission channel is configured to transmit information from the second operator network agent cluster on the edge side to the first operator network agent cluster on the source node side, and from the first operator network agent cluster on the source node side The proxy cluster transmits information to the proxy cluster of the second operator's network on the edge side; and the proxy cluster of the first operator's network on the source node side is configured to receive a file download request from the proxy cluster of the second operator's network on the edge side through the transmission channel , send a file download request to the data source node related to the first operator network, and receive the requested file returned by the data source node related to the first operator network, and transmit it to the second operator network agent on the edge side through the transmission channel cluster.

According to another aspect of the present invention, a download management method used in a data download system is provided. The data download system includes at least an edge node and a data source node. The download management method includes: the edge node obtains a file from a client device Download the request, and analyze the content abstract of the requested file according to the download address of the file download request; the edge node searches the cache for the requested file according to the content abstract of the requested file, and if it exists, the requested file be transmitted to the client device; and if the requested file is not found in the cache, the edge node obtains the requested file from a data source node related to the requested file and transmits it to the client device; the edge node caches the data from the The file obtained by the source node and the content summary of the file. The content summary of the file is the data generated by applying specific data conversion to the content of the file.

According to the embodiment of the present invention, the file content digest parsed from the download address of the requested file can be used as the basis for querying the cached file, and whether the requested file has been stored in the cache is queried instead of using the entire URL of the file as an index query , in the same way, when storing files in the cache, it is also judged whether they are the same file according to whether the content abstract of the file is the same. Therefore, if the URLs of two files are different, but the content data of the files are substantially the same, then the content abstracts of the files are the same. Furthermore, if the files have been cached in the edge node before, then even if the client device Then send a different file download address URL, as long as the content summary of the file is consistent with the one in the cache, then the edge node will not go to the data source node to download the file repeatedly, but directly download the file from the cache according to the content summary of the file This file is found in and served to the client device. Thus, on the one hand, the duplicate data in the cache is reduced, and on the other hand, the efficiency of downloading files for the client device is also improved.

Further, the edge node obtains the file from the data source node through the proxy cluster. When the file is large, it can obtain each part of the requested file from the data source node one by one, and at the same time transmit the obtained parts of the requested file to the client device. Part, that is, back-to-source in non-blocking mode, can be cached and downloaded by the client device at the same time. It avoids the problem of waiting for the entire file to be fully cached before downloading when using a CDN, and the client device does not have to wait for the entire process.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

Fig. 1 shows a schematic diagram of a data downloading system according to an embodiment of the present invention;

Fig. 2 shows a schematic diagram of a management device for providing a file download address according to an embodiment of the present invention;

FIG. 3 shows a schematic diagram of cache logic in a cache according to an embodiment of the present invention;

Fig. 4 shows a schematic diagram of a proxy cluster of Telecom to Netcom according to an embodiment of the present invention;

Fig. 5 shows a schematic diagram of a data downloading system according to an embodiment of the present invention;

FIG. 6 shows a flowchart of a management method for providing a file download address according to an embodiment of the present invention; and

Fig. 7 shows a flowchart of a download management method used in a data download system according to an embodiment of the present invention.

detailed description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

Please refer to FIG. 1 , which is a schematic diagram of a data downloading system according to an embodiment of the present invention. The download system includes an edge node 100 of the first operator network, an edge node 200 of the second operator network, a proxy cluster 300 from the first operator network to the second operator network, and a proxy cluster 300 from the second operator network to the first operator network. The proxy cluster 400 of the network, the data source node 500 of the first operator network, and the data source node 600 of the second operator network. In addition, the figure also shows the client device 700 of the first operator's network and the client device 800 of the second operator's network that have data interaction with the data downloading system. It should be noted that there may be one or more edge nodes, data source nodes, proxy clusters, etc. included in the system.

Both the edge node 100 of the first operator network and the edge node 200 of the second operator network include a download management device 110, and the download management device 110 specifically includes a cache 102, a parser 104, a finder 106, and a source returner 108 . The proxy clusters 300 from the first operator network to the second operator network specifically include a first operator network proxy cluster 302 at an edge side, a first transmission channel 304 , and a second operator network proxy cluster 306 at a source node side. The proxy cluster 400 from the second operator network to the first operator network specifically includes a second operator network proxy cluster 402 at the edge side, a second transmission channel 404 , and a first operator network proxy cluster 406 at the source node side. The data source node 500 of the first operator network includes multiple data source nodes in the first operator network, such as data source node 1, data source node 2, and data source node n. The data source node 600 of the second operator network includes multiple data source nodes in the second operator network, such as data source node 1, data source node 2, and data source node n.

Since the download management device 110 included in the edge node 100 of the first operator network and the edge node 200 of the second operator network have the same structure, the data processing flow is also exactly the same, only the operator networks where the two edge nodes are located are different. , responsible for connecting client devices of different operator networks, so the following mainly takes the working principle of the edge node 100 of the first operator network as an example for illustration, and the internal structure and working principle of the edge node 200 of the second operator network refer to Section The contents of the relevant embodiments of the edge node 100 of an operator network are sufficient, and will not be repeated here.

In one embodiment of the present invention, a certain client device 700 connects to the edge node 100 to request to download a file, specifically, the client device 700 sends a file download request to the edge node 100, and the download request includes the download address of the requested file After obtaining the file download request, the parser 104 parses out the content summary of the requested file according to the download address of the file download request. The download request includes the download address of the requested file, and the most common download address is the URL (Uniform Resource Locator, Uniform Resource Locator) of the requested file. In one embodiment of the present invention, the URL of the requested file sent by the client device 700 to the edge node 100 of the first operator network is different from the URL in the prior art. The URL in the prior art generally includes the protocol type, The host domain name, path and file name where the file is stored. The general format of the URL is (the ones with square brackets [] are optional): protocol://hostname[:port]/path/[;parameters][? query query]#fragment fragment; and in the embodiment of the present invention, the URL of the requested file not only includes the content of the above conventional URL, but also includes a summary of the content of the requested file.

For example, traditionally, the existing general URL to request a file is:

http://wsdl11.yunpan.cn/share.php? method=Share.download&xqid=XXXX&nid=XXXX&cqid=XXXX&fname=1.txt&e=XXXX&st=XXXXX. Among them, XXXX represents the specific information of the file path and file name.

In the embodiment of the present invention, the URL of the requested file is:

http://wsdl11.yunpan.cn/share.php? method=Share.download&fhash=4c9a055de0a290341abf7fff6a4d8c0f2af7f155&xqid=XXXX&nid=XXXX&cqid=XXXX&fname=1.txt&e=XXXX&st=XXXXX.

By comparing the above two URLs, we can see that the second URL has more content than the first URL, namely:

"fhash=4c9a055de0a290341abf7fff6a4d8c0f2af7f155", this part is the content abstract of the requested file. The content summary of a file mainly refers to the use of a specific data conversion method for the content of the file to generate data that can identify the content of the file. The content summary calculation algorithm is, for example, the secure hash algorithm sha series of algorithms (sha1, sha224, and sha256, etc.) One of the information summary calculation MD series algorithms (such as md3, md4 and md5, etc.), and other algorithms known in the art. As long as the content abstract can be used as the unique identifier of the file content, files with the same content correspond to the same content abstract, and files with different content correspond to different content abstracts, so methods that can generate such content abstracts are within the protection scope of the present invention. Generally speaking, the unique identification of the content summary generated after specific data conversion through the entire content of the file is relatively strong, but in some cases, the unique identification of the content summary generated only based on part of the key content of the file is not excluded Sex can also fulfill a practical need, which is also possible in this case.

The URL of the file is generally generated when the file is uploaded to the data source node. The following describes how to obtain the URL of the requested file in the download request sent by the client device 100 to the edge node 100 of the first operator network with an example. Please also refer to FIG. 2 , which shows a schematic diagram of a management device for providing a file download address according to an embodiment of the present invention. The management device 210 includes a resource locator 212 , a summary generator 214 and a download address generator 216 . The management device 210 may be set in a corresponding data source node, such as the data source node 500 of the first operator network or the data source node 600 of the second operator network, or it may be set independently of the data source node. When the management device 210 is set independently from the data source nodes, it means that each data source node is basically only used as a file storage server, and the management device 210 is used as a management device responsible for file-related management and maintenance other than file storage, but the management The data source nodes of the device 210 can communicate with each other, and can obtain various information required by each other.

Specifically, on the one hand, after a file is uploaded to a data source node, the resource locator 212 in the management device 210 can obtain the storage path of the file on the data source node, and then generate The resource location information of the file is equivalent to generating the content of the path and file name in the URL of the file, for example, generating the content of the previous URL example: wsdl11.yunpan.cn/share.php? method=Share.download&xqid=XXXX&nid=XXXX X&cqid=XXXX&fname=1.txt&e=XXXX&st=XXXXX.

On the other hand, the summary generator 214 in the management device 210 can know the specific content of the file through the data source node, and then use various content summary calculation methods to calculate the content of the file to generate the content summary of the file. Various specific implementation schemes of the content abstract will not be repeated here. For example, generate this part of the previous URL example:

fhash = 4c9a055de0a290341abf7fff6a4d8c0f2af7f155.

Thereafter, the download address generator 216 in the management device 210 generates a complete download address of the file, such as a URL, according to the resource location information of the file provided by the resource locator 212 and the content abstract of the file provided by the abstract generator 214. The URL includes at least the resource location information of the file and the content summary of the file. For example, to generate the full URL in the previous URL instance:

http://wsdl11.yunpan.cn/share.php? method=Share.download&fhash=4c9a055de0a290341abf7fff6a4d8c0f2af7f155&xqid=XXXX&nid=XXXX&cqid=XXXX&fname=1.txt&e=XXXX&st=XXXXX.

It can be seen from this that the management device 210 can generate corresponding download addresses, such as URLs, for various files that have been uploaded to a certain data source node. Furthermore, when the client device 100 needs to download a certain file, it can obtain the download address of the file by clicking on the file on the website or checking the URL of the file, etc., and then send the download address including the download address to the edge node. download request. Therefore, edge nodes such as the edge node 100 of the first operator network obtain the URL of the file requested by the client device 100, and the URL not only includes conventional URL parameters, but also includes a content summary of the requested file. It can be seen that, essentially, the URL of the file requested by the client device 700 obtained by the edge node 100 is provided by the management device 210 .

In one embodiment of the present invention, in order to better ensure the security of data transmission, while the management device 210 generates URLs for files, it can also Generate a signature, such as generating a signature based on key information such as a summary of the file content in the URL, a file storage path, etc. When a client device 700 needs to download the file, the file URL and the corresponding signature are provided to the client device 700, and then the client When the end device 700 sends the download request to the edge node 100, it includes the file URL and the corresponding signature. The edge node 100 maintains the same signature generation method as the management device 210. After the edge node 100 receives the file download request from the client device 700, the resolver 104 generates the signature by itself according to the requested file URL and the preset signature generation method. For the signature corresponding to the file URL, compare the self-generated signature with the signature carried in the file download request sent by the client device 700. If they are consistent, it is judged that the URL has not been modified, and subsequent relevant processing of the downloaded file is performed; if inconsistent, then It indicates that the content of the URL of the file sent by the client device 700 has been modified, for example, the summary of the file content in the URL may be maliciously tampered with, and then an error or illegal prompt will be returned to the client device 700, and no subsequent related file download processing will be performed. .

As mentioned above, after receiving the download request from the client device 700, the parser 104 of the edge node 100 is further configured to parse out the content abstract of the requested file according to the download address of the file download request. Furthermore, the finder 106 in the edge node 100 searches in the cache 102 according to the content summary of the requested file parsed by the parser 104, and if the requested file is found, the requested file is directly transmitted to the client device 700, there is no need to go to the data source node to obtain the file.

The cache 102 caches each file obtained from each data source node (such as each data source node 500 of the first operator network and the data source node 600 of the second operator network) and content abstracts of each file. In one embodiment of the present invention, if the file requested by the client device 700 is in the cache 102, the file in the cache is directly provided to the client device 700; Go to the relevant data source node to download, after obtaining the requested file from the data source node, on the one hand, it is cached in the cache 102 for subsequent use, and on the other hand, it is provided to the client device 700 .

It can be seen from the above process that each file stored in the cache 102 is obtained from the data source node. Moreover, since the parser 104 can parse out the content abstract of the file, if there is no file corresponding to the content abstract in the cache 102, it will be obtained from the relevant data source node according to the file download request, and the download address of the file download request will be contains the content abstract of the file, so after the cache 102 obtains the file from the data source node, it already knows what the content abstract corresponding to the file is, and then the cache 102 will cache the file and the content abstract of the file. As mentioned earlier, there are many specific implementation schemes for the content summary of the file, for example, it can be the sha1 of the file, or it can be MD5. In addition, since sha1 is generally longer, but has better uniqueness, MD5 is shorter and more convenient for storage. Therefore, sha1 and MD5 can be used in combination. For example, the file content abstract recorded in the URL of the requested file is sha1, but the cache 102 can perform data conversion on sha1 again to generate a shorter hash value, namely MD5, and then store the file and the file in the cache Corresponding to MD5, use this MD5 as an index keyword key for searching file data in the cache 102 , as shown in FIG. 3 , a schematic diagram of cache logic in the cache according to an embodiment of the present invention.

Optionally, the cache 102 can also set a cache valid time for each file, because the cache capacity is limited, by setting the cache valid time, the occupancy of the cache by long-term unused files can be cleared in time. In addition, it can also be determined whether the file needs to be cached. If some files are not suitable for caching, even if the required file is obtained from the data source node, the cache 108 will not cache it. If there is a client device 700 to request the file later , the edge node 100 will obtain the file from the data source node and provide it to the client device 700 . In addition, the cache 102 can also maintain a download prohibition list, if the content abstract of the file requested by the client device 700 is in the download prohibition list, the edge node 100 will not search the file from the cache 102 and send it to the client. The end device 700 will not send the file to the client device 700 by obtaining the file from the data source node, but inform the client device 700 that the file is prohibited from being downloaded.

Since cache 102 has the aforementioned caching logic, finder 106 can search in cache 102 according to the content abstract of the requested file, whether there is a file corresponding to the file content abstract, and if found, it will directly cache The file in is sent to the client device 700.

It should be noted that generally each operator network will have multiple edge nodes, and there will be multiple operator networks. The cache logic of the buffer 102 can be maintained by all edge nodes of each operator network, for example, all The telecommunications edge nodes jointly maintain a cache logic, that is, the files obtained by each telecommunications edge node are stored in the same logical cache 102, and the files cached in the cache 102 can be used by all telecommunications edge nodes, that is, the finder 106 may search for the required file in the cache 102 jointly maintained by all telecom edge nodes. In addition, it can also be based on a region, that is, multiple edge nodes in a certain area of the same operator jointly maintain a buffer 102, for example, multiple telecom edge nodes in Beijing jointly maintain a buffer, and multiple telecom edge nodes in Shanghai The edge nodes jointly maintain another cache. The telecom edge nodes in Beijing can check whether there is a required file in the cache in Beijing. The node can check whether there is the required file in the cache in Shanghai. If not, it can only go to the data source node to obtain it. Or, it is also possible that each edge node maintains a separate cache logic, that is, each maintains its own cache 102, and each edge node can only search its own cache. If it is not found, it needs to go to the data source node to obtain document. In short, the cache logic of the cache 102 can be set differently according to actual needs, and these are all within the protection scope of the present invention.

If the finder 106 does not find the corresponding file in the cache 102 according to the content summary of the file, it will notify the source-returning device 108, and then the source-returning device 108 will download from the relevant data source according to the source-returning table and the download address of the requested file. The node obtains the requested file, transmits it to the client device 700, and provides the requested file to the cache 102 for caching. Specifically, the source-returning device 108 maintains a source-returning table, which records the content abstract of the file and/or the number of the data source node, and the corresponding relationship between the address of the data source node.

The so-called data source node number can be understood as the number of the data source node server or server cluster, still using the previous URL example as an example:

http://wsdl11.yunpan.cn/share.php? method=Share.download&fhash=4c9a055de0a290341abf7fff6a4d8c0f2af7f155&xqid=XXXX&nid=XXXX&cqid=XXXX&fname=1.txt&e=XXXX&st=XXXXX. The "11" after "wsdl" in the URL is the number of the data source node. Through this label, the host address corresponding to the number can be found in the back-to-source table, such as the specific IP address of the data source node. If the label corresponds to a cluster of data source nodes, the number can be used to find the host addresses of multiple data source nodes in the back-to-source table, and then the back-to-source device 108 can obtain the required information from any one of the hosts. document.

As mentioned earlier, since the back-to-source table can also save the correspondence between the content abstract of the file and the address of the data source node storing the file, the address of the data source node can also be found by using the content abstract of the requested file. In the back-to-source table, the content abstract of the same file may also correspond to multiple data source node addresses, and the back-to-source device 108 selects one address to obtain the required file from the corresponding data source node.

Whether searching for the data source node storing the file based on the data source node number in the URL of the requested file, or searching for the data source node storing the file based on the content summary of the requested file, if only one data source node is found For the requested file, the file data is obtained directly according to the data source node address provided by the back-to-source table. If multiple data source nodes are found, there are multiple selection methods according to actual needs. For example, the required files can be preferentially obtained from the data source nodes of the same operator. If there are multiple data source nodes that need files, there are both data source nodes of the first operator network and data source nodes of the second operator network, then the origin return device 108 preferentially selects the data source node of the first operator network to obtain all required documents. For another example, the load status of multiple data source nodes storing the required files may be considered, that is, according to the principle of load balancing, a data source node with the lightest load is selected to obtain the required files.

The content in the source-returning table maintained by the source-returning device 108 may be generated according to information obtained from each data source node. For example, whenever a file is uploaded to a data source node, a storage record will be updated to the back-to-source table, which records the content summary of the file and/or the number of the data source node storing the file, and the corresponding The address information of each file and the data source node storing the file can be recorded in the back-to-source table. The back-to-source table is not limited to be stored in the edge node 100, and can also be stored in other specialized management devices, as long as the data source node can communicate with the edge node 100 or the special management device, so that the information in the back-to-source table can be based on The data source node updates the stored file, and the source return device 108 in the edge node 100 can obtain the information in the source return table in time. As mentioned above, the source return device 108 can only query the relevant data source node address according to the data source node number in the URL of the requested file, or can only query the relevant data source node address according to the content abstract of the requested file, so the corresponding Yes, the back-to-source table can also only record the association relationship between the data source node number and the data source node address, or only record the association relationship between the content abstract of the file and the data source node address. Of course, you can also use the data source node number first to query the address of the relevant data source node. If you can’t find a suitable one, such as the same operator’s network, continue to check whether there are other storages of the same operator’s network through the content summary of the file. The data source node of the file, in this case, corresponds to the corresponding relationship between the data source node number, the content summary of the file, and the data source node address in the back-to-source table.

The origin-returning device 108 may include a data source query module, a direct-return-to-source module, a proxy-return-to-source module, and a cache notification module. Specifically, if the data source query module finds the storage file according to the known back-to-source table and the download address of the file download request, specifically according to the back-to-source table and the data source node number or the content summary of the file obtained from the download address The data source node of the file and the edge node 100 are in the same operator network, such as the data source node 500 of the first operator network, then the direct back-to-source module forwards the file download request to the found data source node 500, from The data source node 500 obtains the required files. Since both the edge node 100 and the data source node 500 are nodes of the first operator's network, the speed of direct communication between them is very fast without going through the proxy cluster, so they can be directly connected and downloaded. At the same time, the cache notification module notifies the cache 102 to cache the requested file obtained through the direct back-to-source module.

If the data source query module in the edge node 100 of the first operator network cannot find the requested file stored in the data source node belonging to the same operator network as the edge node 100 in the back-to-source table, only the second operator network can be found. If the data source node 600 of the provider network has stored the requested file, then the proxy back-to-source module obtains the requested file from the data source node 600 related to the second carrier network through the proxy cluster 300 from the first carrier network to the second carrier network. document. At the same time, the cache notification module notifies the cache 102 to cache the requested file acquired through the proxy back-to-source module.

In order to better describe how the agent back-to-source module obtains the requested file from the data source node 600 of the second operator network through the agent cluster 300 from the first operator network to the second operator network, the agent cluster 300 is first introduced below specific structure. The proxy cluster 300 includes a first carrier network proxy cluster 302 at the edge side, a second carrier network proxy cluster 306 at the source node side, and a first transmission channel 304 . For example, FIG. 4 is a schematic diagram of a Telecom to Netcom agent cluster according to an embodiment of the present invention. The proxy cluster includes a telecom proxy cluster, a Netcom proxy cluster, and an optical fiber transmission channel between the two proxy clusters. The telecom proxy cluster 412 is equivalent to the first carrier network proxy cluster 302 at the edge side, and the Netcom proxy cluster 416 is equivalent to the second carrier network proxy cluster 306 at the source node side. The telecom proxy cluster 412 communicates with the telecom download node (ie, the telecom edge node) through the telecom outlet, and the network proxy cluster 416 communicates with the Netcom data source node through the Netcom egress.

First, the origin return device 108 of the edge node 100 of the first operator network sends a request to the edge side first operator network in the proxy cluster 300 according to the file download request of the client device 700 and the found address of the data source node 600. The proxy cluster 302 sends a request to download a file from the data source node 600, and then the first carrier network proxy cluster 302 on the edge side transmits the file download request to the second source node side through the first transmission channel 304, which is generally an optical fiber. Carrier network proxy cluster 306 . Furthermore, the proxy cluster 306 of the second operator network on the source node side sends a file download request to the relevant data source node 600 of the second operator network, and obtains the requested file from the data source node 600 . After obtaining the requested file from the data source node 600, the second carrier network proxy cluster 306 on the source node side transmits the requested file to the first carrier network proxy cluster 302 on the edge side through the first transmission channel 304, and then Then, the agent cluster 302 of the first operator network at the edge side transmits the file to the proxy back-to-source module of the edge node 100 , and finally the proxy back-to-source module 108 of the edge node 100 returns the file to the client device 100 . So far, the client device 700 of the first operator network successfully downloads the the requested file.

The structure of the proxy cluster 400 from the second operator network to the first operator network is the same as the structure of the proxy cluster from the first operator network to the second operator network, so it will not be described in detail, but simply introduced. The agent cluster 400 includes an edge-side second operator network agent cluster 402, which receives a file download request from the edge node 200 of the second operator network, and returns the requested file to the edge node 200 of the second operator network. The second transmission channel 404 transmits information from the second carrier network proxy cluster 402 at the edge side to the first carrier network proxy cluster 406 at the source node side, and transmits information from the first carrier network proxy cluster 406 at the source node side to the second operator network proxy cluster at the edge side The provider network proxy cluster 402 transmits information; the source node side first carrier network proxy cluster 406 sends the file download request to the first carrier network according to the file download request received from the edge side second carrier network proxy cluster 402 through the second transmission channel 404 The relevant data source node 500 sends a file download request, and receives the requested file returned by the relevant data source node 500 of the first operator network, and transmits it to the edge-side second operator network proxy cluster 200 through the second transmission channel 404 .

From the above description, it can be seen that the origin return device 108 in the edge node 100 of the first operator network can receive data from the data source node (such as the data source node 500 of the first operator network or the data source node 600 of the second operator network) The files required by the client device 700 are obtained and transmitted to the client device 700 . In one embodiment of the present invention, the source return device 108 can obtain each part of the requested file from the data source node one by one, and transmit the obtained part of the file to the client device 700 at the same time until the file is completely obtained this document. Specifically, after the data source node returns the header information of the file header to the source returner 108 in the edge node 100, the edge node 100 immediately returns the header information to the client device 700, and the client device 100 obtains the header information and then Knowing the size of the file to be downloaded, it will keep the link with the edge node 100, wait for and accept the data returned by the edge node, until the entire file is downloaded. After the edge node 100 is connected to the data source node, it will continue to obtain the content data of the file from the data source node, and at the same time, continue to directly send the obtained content data to the client device 700 until the transmission of the file is completed. After the edge node 100 obtains the entire file from the data source node, it will put the header information and file content into the cache.

It can be seen from this that the edge node 100 can obtain the requested file from the data source node, and at the same time transmit the obtained partial content of the file to the client device 700, without waiting for the edge node 100 to completely obtain the requested file from the data source node. After downloading the entire file content, start to transmit the file to the client device 700 again. In this case, the source return device 108 in the edge node 100 can be regarded as a non-blocking source return device.

In an embodiment of the present invention, the download management device 110 may be implemented by a varniash cache server, or all or part of the edge nodes may be implemented by a varniash cache server. Through the varniash cache server, it is possible to transfer files to the client device synchronously during the process of downloading files from the data source node, that is, to have a non-blocking source return device. Please refer to FIG. 5 for details, which shows a schematic diagram of a data download system according to an embodiment of the present invention, wherein the telecommunications download node 512 is a specific example of the edge node 100 of the first operator network in FIG. 1, and the telecommunications download node 512 Comprise a plurality of varniash nodes; Netcom download node 514 is a specific example of the edge node 200 of the second operator network in Fig. A specific example of the agent cluster 300 from the operator's network to the second operator's network, which can include multiple agent clusters such as agent cluster 1 and agent cluster 2, which can be mutual backups; Netcom—>telecom agent cluster 518 is a diagram A specific example of the proxy cluster 400 from the second operator network to the first operator network in 1, which also includes multiple proxy clusters such as proxy cluster 1 and proxy cluster 2, which can serve as mutual backups; the telecom data source 520 is shown in Figure 1 A specific example of the data source node 500 of the first operator network in the network, which may include multiple data source nodes; the network data source 522 is a specific example of the data source node 600 of the second operator network in Figure 1, which may include Multiple data source nodes. Each varniash node corresponds to a group of proxy clusters, so by using the failover (failure backup) mechanism of the Varnish node itself, it can automatically switch to other proxy clusters when a certain proxy cluster fails, thereby reducing the risk of service unavailability .

Please refer to FIG. 6 , which shows a flowchart of a management method for providing a file download address according to an embodiment of the present invention.

The management method begins with step S610. In step S610, resource location information of the file is first generated according to the storage path of the file in the data source node. When a file is uploaded to a data source node, the data source node or other management devices that can communicate with the data source node can obtain the storage path of the file on the data source node, and then according to the storage path can be The resource location information of the generated file is equivalent to generating the content of the path and file name in the URL of the file. This step can be performed by the resource locator 212 in FIG. 2 . For related technologies, reference can be made to the description of the resource locator 212 in the foregoing embodiments, which will not be repeated here.

On the one hand, the resource location information of the file is generated through step S610; on the other hand, in step S620, the content of the file is converted using specific data to generate a content summary of the file. The content abstract of a file is essentially the identification information of the file content, and the content abstract of the file corresponds to the file data one by one, and the content abstract is different for files with different contents. This step can be implemented by the digest generator 214 in FIG. 2 , and related technologies can refer to the descriptions of the digest generator 214 in the above-mentioned embodiments, which will not be repeated here.

It should be noted that the order between step S610 and step S620 can be adjusted, for example, step S610 can be performed before, after or simultaneously with step S620.

After obtaining the resource location information of the file in step S610 and the content summary of the file in step S620, in step S630, at least according to the resource location information of the file and the content summary of the file, a download address of the file, such as a URL, is generated. The download address at least includes the resource location information of the file and the content summary of the file. So far, the download address of the file is generated, and thereafter, it can be provided to the client device that needs to download the file for use. For example, when the client device clicks on a file or file list on the website and needs to download the file, it can obtain the URL of the file, and then the subsequent client device can send a file download request to the edge node according to the download address, so that The edge node obtains the URL of the file, and the URL not only includes resource location information such as a storage path of the file, but also includes a content summary of the file. This step can be implemented by the download address generator 216 in FIG. 2 . For related technologies, reference can be made to the descriptions of the download address generator 216 in the foregoing embodiments, which will not be repeated here.

Next please refer to FIG. 7 , which shows a flowchart of a download management method used in a data download system according to an embodiment of the present invention. The data downloading system may include the edge node and the data source node described in the foregoing embodiments, and may further include the proxy cluster described in the foregoing embodiments.

The download management method begins with step S710. In step S710, the edge node obtains the file download request from the client device, and parses out the content summary of the requested file according to the download address of the file download request. As mentioned above in the flow of the embodiment shown in Figure 6, the client device can obtain the URL of the file to be downloaded, and the URL also contains the content summary of the file, so when the client device sends a download request to the edge node, The download address URL in the download request will contain a summary of the content of the requested file.

After the edge node obtains the download request of the client device through step S710, and parses out the content summary of the requested file according to the download request, in step S720, the edge node checks whether the requested file exists in the cache according to the content summary of the requested file. document. If yes, go to step S730: obtain the requested file from the cache and transmit it to the client device. If not, that is, there is no required file in the cache, go to step S740: the edge node obtains the requested file from the data source node related to the requested file, and transmits it to the client device.

In step S740 of one embodiment, the edge node may obtain each part of the requested file from the data source node part by part, and transmit the obtained part of the requested file to the client device at the same time, until all parts of the requested file are completely obtained. requested documents. Through this non-blocking back-to-source method, the edge node does not have to wait to obtain the entire file from the data source node before transferring it to the client device. Device Transfer, which further increases file download speeds.

In step S740 of one embodiment, in order to further improve the download efficiency, if the edge node and the data source node are not nodes of the same operator network, then the edge node can also obtain the requested file from the data source node through the proxy cluster, for example, The data downloading system further includes proxy clusters from the first operator network to the second operator network. Step S740 specifically includes: first, the edge node queries the data source node related to the requested file according to the known back-to-source table and the download address of the requested file; When the data source node related to the requested file includes the data source node of the first operator network, directly obtain the requested file from the data source node of the first operator network; when the edge node is a data source node of the first operator network edge node, and it is found that the data source node related to the requested file is the data source node of the second operator network, then the proxy cluster from the first operator network to the second operator network is related to the second operator network The datasource node fetches the requested file.

For the edge node to obtain the requested file directly from the data source node of the same operator network, the process is relatively simple, and will not be described in detail. The following mainly introduces the process of obtaining the requested file from the data source node of the non-same operator network through the proxy cluster. . Specifically, the proxy clusters from the first operator network to the second operator network include the proxy cluster of the first operator network on the edge side, the transmission channel, and the proxy cluster of the second operator network on the source node side. First, the first operator network on the edge side The provider network proxy cluster receives the file download request from the edge node of the first carrier network, and transmits it to the source node side second carrier network proxy cluster through the transmission channel; the source node side second carrier network proxy cluster receives the file After the download request, send a file download request to the data source node related to the second operator network accordingly, and receive the requested file returned by the data source node related to the second operator network, and transmit the file through the transmission channel to the proxy cluster of the first operator network on the edge side; furthermore, the proxy cluster of the first operator network on the edge side transmits the file to the edge node of the first operator network.

When the edge node obtains the file requested by the client through step S740 or after, in step S750, the content abstract of the file and the file content obtained from the data source node are cached, so that there is a subsequent download of the file When requesting, the edge node does not need to go to the data source node to obtain the file, and can directly transfer the file from its own cache to the client device through step S730. Through the continuous accumulation of step S750, more and more files are stored in the edge node cache, and the requested files can be directly provided to the client device to a large extent without going to the data source node for downloading, thereby improving the file size. It also reduces the number of communications with the data source node and reduces the load on the data source node.

In the above-mentioned data processing process, step S710 can be executed by the parser 104 in FIG. 1, steps S720 and S730 can be executed by the finder 106 in FIG. 1, and step S740 can be executed by the source return device 108 in FIG. S750 can be executed by the cache 102 in FIG. 1 , and the above steps executed by the edge node can be executed by the varniash cache server. For implementation of related technologies, reference may be made to the descriptions of corresponding components in the foregoing embodiments, and details are not repeated here.

It can be seen from the above embodiments of the present invention that the edge node queries whether the requested file has been stored in the cache according to the file content summary parsed from the requested file download address as an index keyword, rather than according to the file content. The entire URL is used as an index query. Similarly, when storing files in the cache, it is also judged whether they are the same file according to whether the content abstract of the file is the same, that is, the content abstract of the file is the identification of the file content data, as long as the content abstract of the file is different. are different in themselves, on the contrary, as long as the content abstract of the file is the same, it means that the file data itself is the same. Therefore, if the URLs of two files are different, but the content data of the files are substantially the same, then the content abstracts of the files are the same. Furthermore, if the files have been cached in the edge node before, then even if the client device Then send a different file download address URL, as long as the content summary of the file in the URL is consistent with the one in the cache, then the edge node will not go to the data source node to repeatedly download the file and cache it, but directly based on the content of the file The content digest finds the file from the cache and serves it to the client device. It can be seen that, on the one hand, it reduces the duplicate data in the cache, on the other hand, it also improves the efficiency of downloading files for client devices, and at the same time reduces the number of communications between edge nodes and data source nodes, thereby reducing the number of data source nodes. load and bandwidth consumption of the proxy cluster.

Furthermore, in some cases, the URL of the file has not changed, but the content data of the actual file has been updated. If the existing technology is only based on the URL that does not carry the summary information of the file content as the index keyword for querying the cached file, it is likely that the file whose content data has changed but the URL has not changed is sent to the client device, causing the client to The end device cannot download the latest and most correct file. And adopt the scheme of the embodiment of the present invention, owing to judge whether there is this file in the cache according to the summary of the file content in the file URL, if URL does not change, if the substantial data content changes, then when querying the cache by the summary of the file content, already found If the corresponding file is no longer available, the latest file will be automatically obtained from the data source node and provided to the client device, that is, the file in the cache will be updated in time. In other words, the update of the cache in the embodiment of the present invention is actively initiated by the edge node, rather than initiated by the data source node in the existing CDN network, thus avoiding the problem of low efficiency of using the existing CDN data cache update .

Furthermore, the edge node obtains the file from the data source node through the proxy cluster. When the file is large, the Stream non-blocking mode of the edge node server (such as Varnish) can be used to allow the client device to download while caching. It avoids the problem of waiting for the entire file to be fully cached before downloading when using a CDN, and the client device does not have to wait for the entire process.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the download management device, the management device, and the data download system according to the embodiments of the present invention. Or full functionality. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

It can be seen that the present invention at least provides the following solutions:

A1. A download management device used in an edge node, comprising:

A cache configured to cache each file obtained from the data source node and the content summary of each file, where the content summary of the file is data generated by applying specific data conversion to the content of the file;

A parser configured to obtain a file download request from a client device, and parse out a content abstract of the requested file according to a download address of the file download request;

A finder configured to search the cache for the requested file according to the content summary of the requested file parsed by the parser, and if found, transmit the requested file to the client device ;as well as

A source returner configured to obtain the requested file from a data source node related to the requested file when the finder does not find the requested file in the cache, and transmit it to the client device, and provide the requested file to the cache for caching.

A2. The download management device as described in A1, the source return device is adapted to obtain each part of the requested file from the data source node one by one, and transmit the requested file to the client device at the same time Fetched portions until the requested file is completely fetched.

A3. The download management device as described in A1 or A2, the source returning device includes:

The data source query module is configured to query the data source node related to the requested file according to the known back-to-source table and the download address of the requested file;

The direct back-to-source module is configured so that when the edge node is an edge node of the first operator network, and the data source nodes related to the requested file include the data source nodes of the first operator network , directly obtaining the requested file from the data source node of the first operator network;

The proxy back-to-source module is configured to: when the edge node is an edge node of the first operator network, and the data source node related to the requested file is a data source node of the second operator network, through obtaining the requested file from a data source node associated with the second operator network; and

The cache notification module is configured to notify the cache to cache the requested file after the requested file is acquired through the direct back-to-source module or proxy back-to-source module.

A4. The download management device described in any one of A1 to A3, wherein the content summary of the file includes: one of the secure hash calculation sha series data, or one of the information summary calculation MD series data.

A5. The download management device according to any one of claims A1 to A4, wherein the download management device is a varniash cache server.

B6. A data download system, comprising an edge node, a proxy cluster providing information interaction services between different operator networks, and a data source node, wherein the edge node includes any one of claims 1 to 5 Download and manage devices.

B7. The system according to B6, further comprising: a management device for providing a file download address, the download address of the file download request received by the edge node is provided by the management device for providing a file download address; in,

The management device for providing a file download address includes:

The resource locator is configured to generate the resource location information of the file according to the storage path of the file in the data source node;

a digest generator configured to apply specific data transformations to the content of the file to generate a content digest of the file; and

A download address generator configured to generate the download address of the file at least according to the resource location information of the file provided by the resource locator and the content summary of the file provided by the summary generator, the download The address includes at least the resource location information of the file and the content summary of the file.

B8. According to the system described in B7, the download address of the file generated by the download address generator is the Uniform Resource Locator URL of the file.

B9. According to the system described in any one of B6 to B8, the agent clusters that provide information interaction services between different operator networks include agent clusters from the first operator network to the second operator network, and the second A proxy cluster from the operator network to the first operator network, wherein,

The proxy clusters from the first operator network to the second operator network include:

The first operator network proxy cluster on the edge side is configured to receive a file download request from an edge node of the first operator network, and return the requested file to the edge node of the first operator network;

The transmission channel is configured to transmit information from the first operator network proxy cluster at the edge side to the second operator network proxy cluster at the source side, and to transmit information from the second operator network proxy cluster at the source node side to the The first operator's network on the edge side transmits information through the proxy cluster; and

The second operator network proxy cluster on the source node side is configured to send data related to the second operator network to the second operator network according to the file download request received from the edge side first operator network proxy cluster through the transmission channel. The source node sends a file download request, and receives the requested file returned by the data source node related to the second operator network, and transmits it to the edge-side first operator network proxy cluster through the transmission channel;

The proxy clusters from the second operator network to the first operator network include:

The second operator network proxy cluster on the edge side is configured to receive a file download request from an edge node of the second operator network, and return the requested file to the edge node of the second operator network;

The transmission channel is configured to transmit information from the second operator network proxy cluster on the edge side to the first operator network proxy cluster on the source node side, and to transmit information from the first operator network proxy cluster on the source node side to the edge The second operator's network proxy cluster transmits information on the side; and

The first operator network proxy cluster on the source node side is configured to send data related to the first operator network to the first operator network according to the file download request received through the transmission channel from the second operator network proxy cluster on the edge side. The source node sends a file download request, and receives the requested file returned by the data source node related to the first operator network, and transmits it to the second operator network proxy cluster at the edge side through the transmission channel.

C10. A download management method used in a data download system, the data download system at least includes an edge node and a data source node, and the download management method includes:

The edge node obtains the file download request from the client device, and parses out the content summary of the requested file according to the download address of the file download request;

The edge node looks up whether the requested file exists in the cache according to the content digest of the requested file, and if so, transmits the requested file to the client device; and

If the requested file is not found in the cache, the edge node obtains the requested file from a data source node related to the requested file, and transmits it to the client device;

The edge node caches the file obtained from the data source node and the content summary of the file, and the content summary of the file is data generated by applying specific data conversion to the content of the file.

C11. The download management method as described in C10, the step of obtaining the requested file from the data source node related to the requested file includes:

Portions of the requested file are fetched from the data source node portion by portion, and the fetched portions of the requested file are simultaneously transmitted to the client device until the requested file is completely fetched.

C12. The download management method as described in C10 or 11, the data download system further includes a proxy cluster from the first operator network to the second operator network, and the requested file is obtained from the data source node related to the requested file. The steps for the file include:

Query the data source node related to the requested file according to the known back-to-source table and the download address of the requested file;

When the edge node is an edge node of the first operator network, and the requested file-related data source nodes include the data source node of the first operator network, directly from the first operator network The data source node of the network obtains the requested file;

When the edge node is an edge node of the first operator network, and the requested file related data source node is a data source node of the second operator network, through the first operator network to the second operator network The proxy cluster of the second operator network obtains the requested file from the data source node related to the second operator network.

C13. The download management method described in any one of C10 to C12, wherein the agent clusters from the first operator network to the second operator network include the first operator network agent cluster on the edge side, the transmission channel and the source node side The second operator network proxy cluster, the step of the edge node obtaining the requested file from the data source node related to the second operator network through the proxy cluster from the first operator network to the second operator network includes: :

The first operator's network agent cluster on the edge side receives the file download request from the edge node of the first operator's network, and transmits it to the second operator's network agent cluster on the source node side through a transmission channel;

The second operator network proxy cluster on the source node side sends a file download request to the data source node related to the second operator network according to the file download request from the first operator network proxy cluster on the edge side, and receives the file download request. The requested file returned by the data source node related to the second operator network;

The second operator network proxy cluster on the source node side transmits the requested file to the first operator network proxy cluster on the edge side through the transmission channel;

The proxy cluster of the first operator network on the edge side transmits the requested file to the edge node of the first operator network.

C14. The download management method described in any one of C10 to C13, wherein each step is executed by a varniash cache server.

Claims (10)

1. A download management device used in an edge node, comprising: A cache configured to cache each file obtained from the data source node and the content summary of each file, the content summary of the file is data generated by applying specific data conversion to the content of the file; A parser configured to obtain a file download request from the client device, and parse out a content summary of the requested file according to the download address of the file download request; A finder configured to search the cache for the requested file according to the content summary of the requested file parsed by the parser, and if found, transmit the requested file to the client device ;as well as A source returner configured to obtain the requested file from a data source node related to the requested file when the finder does not find the requested file in the cache, and transmit it to the client device, and provide the requested file to the cache for caching. 2. The download management device according to claim 1, the source return device is adapted to obtain each part of the requested file from the data source node part by part, and transmit the requested file to the client device at the same time. The portion of the file that is fetched until the requested file is completely fetched. 3. The download management device according to claim 1 or 2, wherein the source returning device comprises: The data source query module is configured to query the data source node related to the requested file according to the known back-to-source table and the download address of the requested file; The direct back-to-source module is configured so that when the edge node is an edge node of the first operator network, and the data source nodes related to the requested file include the data source nodes of the first operator network , directly obtaining the requested file from the data source node of the first operator network; The proxy back-to-source module is configured to: when the edge node is an edge node of the first operator network, and the data source node related to the requested file is a data source node of the second operator network, through obtaining the requested file from a data source node associated with the second operator network; and The cache notification module is configured to notify the cache to cache the requested file after the requested file is acquired through the direct back-to-source module or proxy back-to-source module. 4. The download management device according to any one of claims 1 to 3, wherein the content summary of the file includes: one of the secure hash calculation sha series data, or one of the information summary calculation MD series data kind. 5. The download management device according to any one of claims 1 to 4, which is a varniash cache server. 6. A data download system, comprising an edge node, a proxy cluster providing information interaction services between different operator networks, and a data source node, wherein the edge node includes the Download and manage devices. 7. The system according to claim 6, further comprising: a management device for providing a file download address, the download address of the file download request received by the edge node is determined by the management device for providing a file download address provided; among them, The management device for providing a file download address includes: The resource locator is configured to generate the resource location information of the file according to the storage path of the file in the data source node; a digest generator configured to apply specific data transformations to the content of the file to generate a content digest of the file; and The download address generator is configured to generate the download address of the file at least according to the resource location information of the file provided by the resource locator and the content summary of the file provided by the summary generator, and the download The address includes at least the resource location information of the file and the content summary of the file. 8. The system according to claim 7, wherein the download address of the file generated by the download address generator is a Uniform Resource Locator (URL) of the file. 9. The system according to any one of claims 6 to 8, wherein the agent clusters that provide information interaction services between different operator networks include agent clusters from the first operator network to the second operator network, and A proxy cluster from the second operator network to the first operator network, wherein, The proxy clusters from the first operator network to the second operator network include: The first operator network proxy cluster on the edge side is configured to receive a file download request from an edge node of the first operator network, and return the requested file to the edge node of the first operator network; The transmission channel is configured to transmit information from the first operator network proxy cluster at the edge side to the second operator network proxy cluster at the source side, and to transmit information from the second operator network proxy cluster at the source node side to the The first operator's network on the edge side transmits information through the proxy cluster; and The second operator network proxy cluster on the source node side is configured to send data related to the second operator network to the second operator network according to the file download request received from the edge side first operator network proxy cluster through the transmission channel. The source node sends a file download request, and receives the requested file returned by the data source node related to the second operator network, and transmits it to the edge-side first operator network proxy cluster through the transmission channel; The proxy clusters from the second operator network to the first operator network include: The second operator network proxy cluster on the edge side is configured to receive a file download request from an edge node of the second operator network, and return the requested file to the edge node of the second operator network; The transmission channel is configured to transmit information from the second operator network proxy cluster on the edge side to the first operator network proxy cluster on the source node side, and to transmit information from the first operator network proxy cluster on the source node side to the edge The second operator's network proxy cluster transmits information on the side; and The first operator network proxy cluster on the source node side is configured to send data related to the first operator network to the first operator network according to the file download request received through the transmission channel from the second operator network proxy cluster on the edge side. The source node sends a file download request, receives the requested file returned by the data source node related to the first operator network, and transmits it to the second operator network proxy cluster at the edge side through the transmission channel. 10. A download management method used in a data download system, the data download system at least including an edge node and a data source node, the download management method comprising: The edge node obtains the file download request from the client device, and parses out the content summary of the requested file according to the download address of the file download request; The edge node looks up whether the requested file exists in the cache according to the content digest of the requested file, and if so, transmits the requested file to the client device; and If the requested file is not found in the cache, the edge node obtains the requested file from a data source node related to the requested file, and transmits it to the client device; The edge node caches the file obtained from the data source node and the content summary of the file, and the content summary of the file is data generated by applying specific data conversion to the content of the file.