CN111309254B - Data processing method, device, readable medium and electronic device - Google Patents

Abstract

The disclosure relates to a data processing method, a data processing device, a readable medium and electronic equipment. The method comprises the following steps: if a data storage request carrying first target data is received, determining a data type of the first target data, wherein the data type comprises a first type and a second type, and the first type of data has higher heat than the second type of data; and if the data type of the first target data is the first type, storing the first target data, and sending the first target data to other storage nodes of the data storage network so that the other storage nodes store the first target data, wherein the other storage nodes at least comprise a central storage node. Therefore, when the data has higher heat, the data is not only stored locally at the edge storage node, but also pushed to the central storage node of the data storage network for storage, and the access response speed of the data when accessed is improved.

Description

Data processing method, device, readable medium and electronic device

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to a data processing method, device, readable medium and electronic device.

Background Art

At present, there are generally two ways to store data. One is to transfer data to the center for storage, which occupies center resources and consumes a lot of costs. In addition, for some data with low access volume, it will cause a great waste of center resources. The other is to store data at the edge. If an edge node needs to access resources other than the edge node, it needs to access the data through the center. In this way, when the data has a high popularity, each edge node needs to access the center, resulting in an increased data processing burden for the center and poor response speed.

Summary of the invention

This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

In a first aspect, the present disclosure provides a data processing method, which is applied to an edge storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the method comprises:

If a data storage request carrying first target data is received, determining a data type of the first target data, the data type includes a first type and a second type, and the first type of data has a higher heat than the second type of data;

If the data type of the first target data is the first type, the first target data is stored, and the first target data is sent to other storage nodes of the data storage network so that the other storage nodes store the first target data, and the other storage nodes include at least a central storage node.

In a second aspect, the present disclosure provides a data processing method, which is applied to a central storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the method comprises:

If a data access request for accessing second target data sent by the target edge storage node is received, determining whether the second target data is stored locally;

If the second target data is stored locally, sending the second target data to the target edge storage node;

If the second target data is not stored locally, second storage information for indicating the storage location of the second target data is determined from the data storage information, and the second storage information is sent to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network.

In a third aspect, the present disclosure provides a data processing device, applied to an edge storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the device includes:

A first determination module is configured to determine a data type of the first target data if a data storage request carrying first target data is received, the data type comprising a first type and a second type, and the first type of data has a higher heat than the second type of data;

A first storage module is used to store the first target data if the data type of the first target data is the first type, and to send the first target data to other storage nodes of the data storage network so that the other storage nodes store the first target data, and the other storage nodes include at least a central storage node.

In a fourth aspect, the present disclosure provides a data processing device, applied to a central storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the device includes:

A second determination module, configured to determine whether the second target data is stored locally upon receiving a data access request for accessing second target data sent by a target edge storage node;

a fourth sending module, configured to send the second target data to the target edge storage node if the second target data is stored locally;

The fifth sending module is used to determine the second storage information for indicating the storage location of the second target data from the data storage information if the second target data is not stored locally, and send the second storage information to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network.

In a fifth aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect of the present disclosure; or, when executed by a processing device, implements the steps of the method described in the second aspect of the present disclosure.

In a sixth aspect, the present disclosure provides an electronic device, including:

a storage device having a computer program stored thereon;

A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure, or to implement the steps of the method described in the second aspect of the present disclosure.

Through the above technical solution, if a data storage request carrying first target data is received, the data type of the first target data is determined, and if the data type of the first target data is the first data type, the first target data is stored, and the first target data is sent to other storage nodes of the data storage network, so that other storage nodes store the first target data. Among them, other storage nodes include at least central storage nodes. In this way, when data is uploaded to the edge storage node, the popularity of the data is first judged, and when the uploaded data has a high popularity, the data is not only stored locally in the edge storage node, but also pushed to the central storage node of the data storage network for storage, which can improve the access response speed when the data is accessed.

Other features and advantages of the present disclosure will be described in detail in the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

In the attached picture:

FIG1 is a flow chart of a data processing method provided according to an embodiment of the present disclosure;

FIG2 is an exemplary flowchart of a step of determining a data type of first target data in a data processing method provided by the present disclosure;

FIG3 is a flow chart of a data processing method provided according to an embodiment of the present disclosure;

FIG4 is a block diagram of a data processing device provided according to an embodiment of the present disclosure;

FIG5 is a block diagram of a data processing device provided according to an embodiment of the present disclosure;

FIG. 6 is a block diagram of an electronic device provided by the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

Figure 1 is a flow chart of a data processing method provided according to an embodiment of the present disclosure. The method can be applied to edge storage nodes of a data storage network, and the data processing method provided by the present disclosure can be applied to each edge storage node in the data storage network.

Among them, the data storage network is composed of central storage nodes and edge storage nodes. The central storage node serves as the data storage center of the data storage network, and the edge storage node serves as the edge of the network and is distributed in various places.

As shown in FIG. 1 , the method may include the following steps.

In step 11, if a data storage request carrying first target data is received, the data type of the first target data is determined.

The data types include a first type and a second type, and the data of the second type has a higher heat than the data of the second type. The higher the heat of the data, the more times the data may be accessed. In a more popular sense, the data belonging to the first type can be understood as hot data, which has a higher access volume in the data storage network, and the data belonging to the second type can be understood as cold data, which has a lower access volume than the hot data.

If a data storage request carrying first target data is received, the data type of the first target data is first determined to determine whether the first target data to be stored belongs to the first type or the second type.

In step 12, if the data type of the first target data is the first type, the first target data is stored, and the first target data is sent to other storage nodes of the data storage network so that the other storage nodes store the first target data.

If the data type of the first target data is the first type, the first target data is stored, and the storing of the first target data described here is to store the first target data locally in the edge storage node. In addition, if the data type of the first target data is the first type, it means that the first target data to be stored has a higher popularity. Therefore, while storing the first target data locally, the first target data is also sent to other storage nodes of the data storage network so that other storage nodes store the first target data. Among them, the other storage nodes include at least a central storage node. In this way, when the first target data has a higher popularity, it is not only stored locally in the edge storage node that receives the first target data, but also pushed to the central storage node of the data storage network, thereby improving the subsequent access response speed to the first target data. In addition, other storage nodes may also include other edge storage nodes.

Through the above technical solution, if a data storage request carrying first target data is received, the data type of the first target data is determined, and if the data type of the first target data is the first data type, the first target data is stored, and the first target data is sent to other storage nodes of the data storage network, so that other storage nodes store the first target data. Among them, other storage nodes include at least central storage nodes. In this way, when data is uploaded to the edge storage node, the popularity of the data is first judged, and when the uploaded data has a high popularity, the data is not only stored locally in the edge storage node, but also pushed to the central storage node of the data storage network for storage, which can improve the access response speed when the data is accessed.

In a possible implementation, in step 11, determining the data type of the first target data may include the following steps, as shown in FIG. 2 .

In step 21, a first heat value of the first target data is determined.

The heat value is positively correlated with the heat of the data, that is, the higher the heat of the data, the higher the heat value of the data. And the heat value can be calculated by at least one of the following: data title, data label, data attribute, data size.

The data title is the title corresponding to the data. The data tag is the tag carried by the data when it is uploaded. The data attribute is the file type corresponding to the data, for example, video, audio, picture, document, etc. The data size is the size of the storage space occupied by the data. For example, if the first target data is a video titled "Food A" and its size is 14M, and the first target data is uploaded with the tag "Food", then the data title corresponding to the first target data is "Food A", the corresponding data tag is "Food", the corresponding data attribute is "Video", and the corresponding data size is "14M".

The following example uses the data title, data label, data attribute and data size to calculate the heat value to illustrate how to calculate the heat value. For example, the heat value can be calculated using the following formula:

Heat value = (first weight * data title heat) + (second weight * data label heat) + (third weight * data attribute heat) + (fourth weight * data size heat)

Among them, the first weight to the fourth weight can be set according to the experience value. The heat of the data title can be determined by the text contained in the title. For example, a hot word list can be pre-set, including multiple hot words with high access volume, and the number of such hot words contained in the data title can be determined, and the percentage of the hot words contained in the data title to all the hot words in the hot word list is determined as the heat of the data title. The heat of the data tag can be determined by the correspondence between the pre-set label and the heat. For example, the corresponding heat is set for each label in advance, so that the heat of the data tag can be determined according to the data label. The heat of the data attribute can be determined by the correspondence between the pre-set attribute and the heat. For example, the corresponding heat is set for each attribute in advance, so that the heat of the data attribute is determined according to the data attribute. The heat of the data size can be determined by the correspondence between the pre-set size and the heat. For example, the size of the general data is divided into several gears and the correspondence between each gear and the heat is determined, so that the heat of the data size is determined according to the gear to which the data size belongs. In addition, for calculating the heat value of other combinations of data titles, data tags, data attributes, and data sizes, the above formula can be referred to.

Therefore, referring to the above method, the first heat value of the first target data can be determined.

In step 22, the data type of the first target data is determined according to the first heat value.

According to the calculated first heat value, the data type of the first target data can be determined, that is, whether the first target data is cold data or hot data can be determined.

In a possible embodiment, step 22 may include the following steps:

Determine a heat value ranking in descending order of heat values according to the first heat value and the second heat values corresponding to each data stored in the data storage network;

If the first popularity value is in the top N positions of the popularity value ranking, it is determined that the first target data belongs to the first type;

If the first popularity value is not in the top N positions of the popularity value ranking, it is determined that the first target data belongs to the second type.

The data stored in the data storage network have their respective corresponding second heat values. Therefore, according to each second heat value and the first heat value corresponding to the first target data, after sorting in order from large to small, the heat value ranking corresponding to the first target data can be obtained, so that the position of the first heat value in the heat value ranking can be determined. The position of the first heat value in the heat value ranking can reflect the heat of the first target data. Therefore, if the first heat value is in the top N positions of the heat value ranking, it means that the heat of the first target data is relatively high, and it can be determined that the first target data belongs to the first type, and if the first heat value is not in the top N positions of the heat value ranking, it means that the heat of the first target data is not high, and it can be determined that the first target data belongs to the second type. Among them, N is a positive integer, which can be determined according to an empirical value. It should be noted that the first heat value and the second heat value are calculated in the same way to ensure the standard uniformity of the heat value.

Using the above method, the first heat value of the first target data is first calculated, and based on the first heat value of the first target data, its data type is determined by heat value comparison, so that the obtained result is more objective and can better reflect the heat situation of the first target data in the entire data storage network.

In a possible embodiment, the method provided by the present disclosure further includes:

If the first target data carries the mark information corresponding to the first type, it is determined that the first target data belongs to the first type, otherwise, step 21 and step 22 are executed.

Among them, the tag information corresponding to the first type is pre-defined, and the data with the above tag information will be automatically defaulted to belong to the first type. Therefore, if the first target data carries the tag information corresponding to the first type, it can be directly determined that the first target data belongs to the first type. In the case where the first target data does not carry the tag information corresponding to the first type, the data type of the first target data is determined by calculating the heat value, wherein the method of determining the data type of the first target data by calculating the heat value has been introduced in detail above and will not be repeated here.

By adopting the above method, if the first target data carries tag information corresponding to the first type, it is directly determined that the first target data belongs to the first type without calculating the heat value of the first target data. The method is simple and fast.

In a possible implementation, when the edge storage node sends the first target data to the central storage node, there may be multiple transmission lines, and the edge storage node may select a transmission line from among them for transmitting the first target data. The edge storage node may select the transmission line with the least routing consumption among the multiple transmission lines for data transmission.

For example, the routing cost can be calculated by the following formula:

Routing cost = bandwidth cost * network delay / average delay per unit time

The unit time may be 1 hour.

In a possible implementation, based on the steps shown in FIG. 1 , the method provided by the present disclosure may further include the following steps:

If the data type of the first target data is the second type, storing the first target data;

Generate first storage information and send the first storage information to the central storage node, wherein the first storage information is used to indicate the storage location of the first target data.

If the data type of the first target data is determined to be the second type in step 11, it means that the popularity of the first target data is not high, so the first target data can be directly stored locally in the edge storage node. At the same time, first storage information can also be generated based on the first target data and its storage location, and the first storage information can be sent to the central storage node for storage by the central storage node.

In the above manner, if the popularity of the data uploaded to the edge storage node is not high, the data will only be stored in the edge storage node, and the data will not be pushed to the central storage node of the data storage network, so as to save the storage space of the central storage node. At the same time, the storage information about the data storage will be sent to the central storage node, so that if there is an access request for the data, the central storage node can provide the access method for the data.

In a possible implementation, based on the steps shown in FIG. 1 , the method provided by the present disclosure may further include the following steps:

If a data access request for the second target data is received from the data access party and the second target data is not stored locally, the data access request for the second target data is sent to the central storage node to obtain a feedback result from the central storage node;

If the feedback result of the central storage node indicates the second storage information of the storage location of the second target data, the second target data is obtained from the edge storage node indicated by the second storage information, and the second target data is sent to the data access party;

If the second target data is stored locally, or if the feedback result of the central storage node is the second target data, the second target data is sent to the data access party.

If a data access request for second target data is received from a data access party, it is first determined whether the edge storage node locally stores the second target data. If the edge storage node locally stores the second target data, the second target data can be directly sent to the target access party.

If it is determined that the edge storage node does not store the second target data locally, it means that the edge storage node cannot find the second target data locally. At this time, you can ask the central storage node for help, that is, send a data access request for the second target data to the central storage node to obtain feedback from the central storage node.

The central storage node may store the second target data or may not store the second target data. Therefore, the central storage node may return two feedback results. If the central storage node stores the second target data, the feedback result is the second target data; if the central storage node does not store the second target data, the feedback result is not the second target data, but the storage location where the second target data is located.

If the feedback result of the central storage node is the second target data, the second target data may be directly sent to the target access party.

If the feedback result of the central storage node indicates the second storage information of the storage location of the second target data, the second target data can be obtained from the edge storage node indicated by the second storage information, and the second target data can be sent to the data access party.

Using the above method, when a data access request is received, it is first determined whether the edge storage node has the data requested by the data access request locally. If the data requested by the data access request is not available locally, the central storage node is further requested. If the central storage node also does not have the data requested by the data access request, the central storage node will return the storage location of the data requested by the data access request in the data storage network. In this way, the edge storage node can directly obtain the data requested by the data access request from other edge storage nodes, and the data response speed is improved.

FIG3 is a flow chart of a data processing method provided according to an embodiment of the present disclosure. The method can be applied to a central storage node of a data storage network, and the data processing method provided by the present disclosure can be applied to each central storage node in the data storage network. As shown in FIG3 , the method can include the following steps.

In step 31, if a data access request for accessing second target data sent by a target edge storage node is received, it is determined whether the second target data is stored locally.

In step 32, if the second target data is stored locally, the second target data is sent to the target edge storage node.

In step 33, if the second target data is not stored locally, second storage information for indicating the storage location of the second target data is determined from the data storage information, and the second storage information is sent to the target edge storage node.

The data storage information is used to indicate the storage location of each data in the data storage network. As described above for the edge storage node, whenever data is stored in the data storage network, the central storage node will obtain the storage location of the stored data in the data storage network. The central storage node stores the storage location of each data in the data storage network, which is equivalent to storing the index of the data.

Through the above technical solution, a data access request for accessing the second target data sent by the target edge storage node is received, and it is determined whether the second target data is stored locally. If the second target data is stored locally, the second target data is sent to the target edge storage node. If the second target data is not stored locally, the second storage information indicating the storage location of the second target data is determined from the data storage information, and the second storage information is sent to the target edge storage node. Thus, the central storage node can always provide the target edge storage node with information about the second target data, so that the target edge storage node can quickly obtain the second target data.

In a possible implementation manner, based on the steps shown in FIG3 , the present disclosure may further include the following steps:

Obtaining historical access information of each data stored in the data storage network;

According to the historical access information, determine the high-popularity data with the top M access times;

If there is third target data that is not stored locally in the high-hot data, the storage location of the third target data is determined according to the data storage information;

The third target data is acquired according to the storage location of the third target data, and the third target data is stored locally.

The historical access information of the data is used to indicate the number of access times of the data access requests received for the data.

According to the historical access information, the high popularity data with the top M access times can be determined, where M is a positive integer and can be determined according to an empirical value.

These high-heat data have a high number of accesses, and therefore should be stored in the central storage node to cope with possible multiple access situations. Therefore, if there is third target data in the high-heat data that is not stored locally in the central storage node, such third target data can be copied to the central storage node. The method adopted can be: determine the storage location of the third target data according to the data storage information, obtain the third target data according to the storage location of the third target data, and store the third target data locally. For example, if the storage location of the third target data is the edge storage node K, the central storage node can obtain the third target data from the edge storage node K and store it in the central storage node.

In the above manner, if data has high popularity but is not stored in the central storage node, the central storage node will obtain such data so that it can respond more quickly when such data is accessed later.

FIG4 is a block diagram of a data processing device provided according to an embodiment of the present disclosure. An edge storage node applied to a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node. As shown in FIG4 , the device 40 may include:

A first determination module 41 is configured to, if a data storage request carrying first target data is received, determine a data type of the first target data, the data type including a first type and a second type, and the first type of data has a higher heat than the second type of data;

The first storage module 42 is used to store the first target data if the data type of the first target data is the first type, and send the first target data to other storage nodes of the data storage network so that the other storage nodes store the first target data, and the other storage nodes include at least a central storage node.

Optionally, the first determining module 41 includes:

A first determination submodule is used to determine a first heat value of the first target data, wherein the heat value is positively correlated with the heat of the data, and the heat value is calculated by at least one of the following: data title, data label, data attribute, data size;

A second determining submodule is configured to determine a data type of the first target data according to the first heat value.

Optionally, the first determination submodule is used to determine that the first target data belongs to the first type if the first target data carries tag information corresponding to the first type, otherwise determine the first heat value of the first target data, and determine the data type of the first target data based on the first heat value.

Optionally, the second determination submodule is used to determine the heat value ranking in descending order according to the first heat value and the second heat value corresponding to each data stored in the data storage network; if the first heat value is in the top N positions of the heat value ranking, it is determined that the first target data belongs to the first type; if the first heat value is not in the top N positions of the heat value ranking, it is determined that the first target data belongs to the second type.

Optionally, the device 40 further includes:

a second storage module, configured to store the first target data if the data type of the first target data is the second type;

The information generation module is used to generate first storage information and send the first storage information to a central storage node, where the first storage information is used to indicate a storage location of the first target data.

Optionally, the device 40 further includes:

A first sending module is used for, if a data access request for second target data is received from a data access party and the second target data is not stored locally, sending the data access request for the second target data to a central storage node to obtain a feedback result from the central storage node;

A second sending module is used to obtain the second target data from the edge storage node indicated by the second storage information if the feedback result of the central storage node indicates the second storage information of the storage location of the second target data, and send the second target data to the data access party;

The third sending module is used to send the second target data to the data access party if the second target data is stored locally or if the feedback result of the central storage node is the second target data.

FIG5 is a block diagram of a data processing device provided according to an embodiment of the present disclosure. The device 50 is applied to a central storage node of a data storage network, and the data storage network is composed of a central storage node and an edge storage node. As shown in FIG5 , the device 50 includes:

A second determination module 51 is configured to determine whether the second target data is stored locally upon receiving a data access request for accessing the second target data sent by the target edge storage node;

A fourth sending module 52, configured to send the second target data to the target edge storage node if the second target data is stored locally;

The fifth sending module 53 is used to determine the second storage information for indicating the storage location of the second target data from the data storage information if the second target data is not stored locally, and send the second storage information to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network.

Optionally, the device 50 further includes:

A first acquisition module, configured to acquire historical access information of each data stored in the data storage network, wherein the historical access information of the data is used to indicate the number of access times of data access requests received for the data;

A third determination module is used to determine the high popularity data with the top M access times according to the historical access information;

A fourth determination module is used to determine the storage location of the third target data according to the data storage information if there is third target data that is not stored locally in the high-heat data;

A third storage module is used to obtain the third target data according to the storage location of the third target data, and store the third target data locally.

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

Referring to FIG6 below, a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure is shown. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG6 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

As shown in FIG6 , the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604.

Typically, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 6 shows an electronic device 600 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: if receiving a data storage request carrying first target data, determines the data type of the first target data, the data type includes a first type and a second type, and the first type of data has a higher popularity than the second type of data; if the data type of the first target data is the first type, stores the first target data, and sends the first target data to other storage nodes of the data storage network so that the other storage nodes store the first target data, and the other storage nodes include at least a central storage node.

Alternatively, the computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: if a data access request for accessing second target data sent by a target edge storage node is received, determines whether the second target data is stored locally; if the second target data is stored locally, sends the second target data to the target edge storage node; if the second target data is not stored locally, determines second storage information for indicating the storage location of the second target data from the data storage information, and sends the second storage information to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, the first determination module may also be described as a "module for determining the data type of the first target data".

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, a data processing method is provided, which is applied to an edge storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the method includes:

If a data storage request carrying first target data is received, determining a data type of the first target data, the data type includes a first type and a second type, and the first type of data has a higher heat than the second type of data;

If the data type of the first target data is the first type, the first target data is stored, and the first target data is sent to other storage nodes of the data storage network so that the other storage nodes store the first target data, and the other storage nodes include at least a central storage node.

According to one or more embodiments of the present disclosure, a data processing method is provided, wherein determining the data type of the first target data includes:

Determine a first heat value of the first target data, wherein the heat value is positively correlated with the heat of the data, and the heat value is calculated by at least one of the following: data title, data label, data attribute, data size;

The data type of the first target data is determined according to the first heat value.

According to one or more embodiments of the present disclosure, a data processing method is provided, the method further comprising:

If the first target data carries tag information corresponding to the first type, it is determined that the first target data belongs to the first type; otherwise, the steps of determining the first heat value of the first target data and determining the data type of the first target data based on the first heat value are performed.

According to one or more embodiments of the present disclosure, a data processing method is provided, wherein determining the data type of the first target data according to the first heat value includes:

Determining a heat value ranking in descending order of heat values according to the first heat value and the second heat values corresponding to each data stored in the data storage network;

If the first popularity value is in the top N positions of the popularity value ranking, it is determined that the first target data belongs to the first type;

If the first popularity value is not in the top N positions of the popularity value ranking, it is determined that the first target data belongs to the second type.

According to one or more embodiments of the present disclosure, a data processing method is provided, the method further comprising:

If the data type of the first target data is the second type, storing the first target data;

Generate first storage information and send the first storage information to a central storage node, where the first storage information is used to indicate a storage location of the first target data.

According to one or more embodiments of the present disclosure, a data processing method is provided, the method further comprising:

If a data access request for the second target data is received from the data access party, and the second target data is not stored locally, the data access request for the second target data is sent to the central storage node to obtain a feedback result from the central storage node;

If the feedback result of the central storage node indicates second storage information of the storage location of the second target data, obtain the second target data from the edge storage node indicated by the second storage information, and send the second target data to the data access party;

If the second target data is stored locally, or if the feedback result of the central storage node is the second target data, the second target data is sent to the data access party.

According to one or more embodiments of the present disclosure, a data processing method is provided, which is applied to a central storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the method includes:

If a data access request for accessing second target data sent by the target edge storage node is received, determining whether the second target data is stored locally;

If the second target data is stored locally, sending the second target data to the target edge storage node;

If the second target data is not stored locally, second storage information for indicating the storage location of the second target data is determined from the data storage information, and the second storage information is sent to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network.

According to one or more embodiments of the present disclosure, a data processing method is provided, the method further comprising:

Acquire historical access information of each data stored in the data storage network, wherein the historical access information of the data is used to indicate the number of access times of data access requests received for the data;

According to the historical access information, determine the high-heat data with the top M access times;

If there is third target data that is not stored locally in the high-hot data, determining a storage location of the third target data according to the data storage information;

The third target data is acquired according to the storage location of the third target data, and the third target data is stored locally.

According to one or more embodiments of the present disclosure, a data processing device is provided, which is applied to an edge storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the device includes:

A first determination module is configured to determine a data type of the first target data if a data storage request carrying first target data is received, the data type comprising a first type and a second type, and the first type of data has a higher heat than the second type of data;

A first storage module is used to store the first target data if the data type of the first target data is the first type, and to send the first target data to other storage nodes of the data storage network so that the other storage nodes store the first target data, and the other storage nodes include at least a central storage node.

According to one or more embodiments of the present disclosure, a data processing device is provided, wherein the first determining module includes:

A first determination submodule is used to determine a first heat value of the first target data, wherein the heat value is positively correlated with the heat of the data, and the heat value is calculated by at least one of the following: data title, data label, data attribute, data size;

A second determining submodule is configured to determine a data type of the first target data according to the first heat value.

According to one or more embodiments of the present disclosure, a data processing device is provided, wherein the first determination submodule is used to determine that the first target data belongs to the first type if the first target data carries tag information corresponding to the first type, otherwise determine a first heat value of the first target data, and determine the data type of the first target data based on the first heat value.

According to one or more embodiments of the present disclosure, a data processing device is provided, wherein the second determination submodule is used to determine a heat value ranking in descending order of heat values based on the first heat value and the second heat values corresponding to each data stored in the data storage network; if the first heat value is in the top N positions of the heat value ranking, it is determined that the first target data belongs to the first type; if the first heat value is not in the top N positions of the heat value ranking, it is determined that the first target data belongs to the second type.

According to one or more embodiments of the present disclosure, a data processing device is provided, the device further comprising:

a second storage module, configured to store the first target data if the data type of the first target data is the second type;

The information generation module is used to generate first storage information and send the first storage information to a central storage node, where the first storage information is used to indicate a storage location of the first target data.

According to one or more embodiments of the present disclosure, a data processing device is provided, the device further comprising:

A first sending module is used for, if a data access request for second target data is received from a data access party and the second target data is not stored locally, sending the data access request for the second target data to a central storage node to obtain a feedback result from the central storage node;

A second sending module is used to obtain the second target data from the edge storage node indicated by the second storage information if the feedback result of the central storage node indicates the second storage information of the storage location of the second target data, and send the second target data to the data access party;

The third sending module is used to send the second target data to the data access party if the second target data is stored locally or if the feedback result of the central storage node is the second target data.

According to one or more embodiments of the present disclosure, a data processing device is provided, which is applied to a central storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the device includes:

A second determination module, configured to determine whether the second target data is stored locally upon receiving a data access request for accessing second target data sent by a target edge storage node;

a fourth sending module, configured to send the second target data to the target edge storage node if the second target data is stored locally;

The fifth sending module is used to determine the second storage information for indicating the storage location of the second target data from the data storage information if the second target data is not stored locally, and send the second storage information to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network.

According to one or more embodiments of the present disclosure, a data processing device is provided, the device further comprising:

A first acquisition module, configured to acquire historical access information of each data stored in the data storage network, wherein the historical access information of the data is used to indicate the number of access times of data access requests received for the data;

A third determination module is used to determine the high popularity data with the top M access times according to the historical access information;

A fourth determination module is used to determine the storage location of the third target data according to the data storage information if there is third target data that is not stored locally in the high-heat data;

A third storage module is used to obtain the third target data according to the storage location of the third target data, and store the third target data locally.

The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims (11)

1. A data processing method, characterized in that it is applied to an edge storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the method comprises: If a data storage request carrying first target data is received, determining a data type of the first target data, the data type includes a first type and a second type, and the first type of data has a higher heat than the second type of data; If the data type of the first target data is the first type, storing the first target data, and sending the first target data to other storage nodes of the data storage network, so that the other storage nodes store the first target data, and the other storage nodes at least include a central storage node; If the data type of the first target data is the second type, the first target data is stored, and first storage information is generated and sent to a central storage node, where the first storage information is used to indicate a storage location of the first target data; The method further comprises: If a data access request for the second target data is received from the data access party, and the second target data is not stored locally, the data access request for the second target data is sent to the central storage node to obtain a feedback result from the central storage node; If the feedback result of the central storage node indicates second storage information of the storage location of the second target data, the second target data is obtained from the edge storage node indicated by the second storage information, and the second target data is sent to the data access party. 2. The method according to claim 1, wherein determining the data type of the first target data comprises: Determine a first heat value of the first target data, wherein the heat value is positively correlated with the heat of the data, and the heat value is calculated by at least one of the following: data title, data label, data attribute, data size; The data type of the first target data is determined according to the first heat value. 3. The method according to claim 2, characterized in that the method further comprises: If the first target data carries tag information corresponding to the first type, it is determined that the first target data belongs to the first type; otherwise, the steps of determining the first heat value of the first target data and determining the data type of the first target data based on the first heat value are performed. 4. The method according to claim 2, wherein determining the data type of the first target data according to the first heat value comprises: Determining a heat value ranking in descending order of heat values according to the first heat value and the second heat values corresponding to each data stored in the data storage network; If the first popularity value is in the top N positions of the popularity value ranking, it is determined that the first target data belongs to the first type; If the first popularity value is not in the top N positions of the popularity value ranking, it is determined that the first target data belongs to the second type. 5. The method according to claim 1, characterized in that the method further comprises: If the second target data is stored locally, or if the feedback result of the central storage node is the second target data, the second target data is sent to the data access party. 6. A data processing method, characterized in that it is applied to a central storage node of a data storage network, the data storage network being composed of a central storage node and an edge storage node, the method comprising: If a data access request for accessing second target data sent by the target edge storage node is received, determining whether the second target data is stored locally, wherein the data access request is sent by the target edge storage node to the central storage node when the target edge storage node does not store the second target data locally; If the second target data is stored locally, sending the second target data to the target edge storage node; If the second target data is not stored locally, second storage information for indicating the storage location of the second target data is determined from the data storage information, and the second storage information is sent to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network. 7. The method according to claim 6, characterized in that the method further comprises: Acquire historical access information of each data stored in the data storage network, wherein the historical access information of the data is used to indicate the number of access times of data access requests received for the data; According to the historical access information, determine the high-heat data with the top M access times; If there is third target data that is not stored locally in the high-hot data, determining a storage location of the third target data according to the data storage information; The third target data is acquired according to the storage location of the third target data, and the third target data is stored locally. 8. A data processing device, characterized in that it is applied to an edge storage node of a data storage network, wherein the data storage network is composed of a central storage node and an edge storage node, and the device comprises: A first determination module is configured to determine a data type of the first target data if a data storage request carrying first target data is received, the data type comprising a first type and a second type, and the first type of data has a higher heat than the second type of data; a first storage module, configured to store the first target data if the data type of the first target data is the first type, and send the first target data to other storage nodes of the data storage network so that the other storage nodes store the first target data, wherein the other storage nodes at least include a central storage node; a second storage module, configured to store the first target data if the data type of the first target data is the second type; An information generation module, used to generate first storage information and send the first storage information to a central storage node, wherein the first storage information is used to indicate a storage location of the first target data; A first sending module is used to send the data access request for the second target data to the central storage node if a data access request for the second target data is received from the data access party and the second target data is not stored locally, so as to obtain a feedback result of the central storage node; A second sending module is used to obtain the second target data from the edge storage node indicated by the second storage information if the feedback result of the central storage node indicates the second storage information of the storage location of the second target data, and send the second target data to the data access party. 9. A data processing device, characterized in that it is applied to a central storage node of a data storage network, the data storage network is composed of a central storage node and an edge storage node, and the device comprises: a second determination module, configured to determine whether the second target data is stored locally upon receiving a data access request for accessing the second target data sent by the target edge storage node, wherein the data access request is sent by the target edge storage node to the central storage node when the target edge storage node does not store the second target data locally; a fourth sending module, configured to send the second target data to the target edge storage node if the second target data is stored locally; The fifth sending module is used to determine the second storage information for indicating the storage location of the second target data from the data storage information if the second target data is not stored locally, and send the second storage information to the target edge storage node, wherein the data storage information is used to indicate the storage location of each data in the data storage network. 10. A computer-readable medium having a computer program stored thereon, characterized in that when the program is executed by a processing device, the program implements the steps of the method described in any one of claims 1 to 5, or when the program is executed by a processing device, the program implements the steps of the method described in claim 6 or 7. 11. An electronic device, comprising: a storage device having a computer program stored thereon; A processing device, used to execute the computer program in the storage device to implement the steps of the method described in any one of claims 1 to 5, or to implement the steps of the method described in claim 6 or 7.