CN112925809A - Data storage method, device and system - Google Patents
Data storage method, device and system Download PDFInfo
- Publication number
- CN112925809A CN112925809A CN202110206114.1A CN202110206114A CN112925809A CN 112925809 A CN112925809 A CN 112925809A CN 202110206114 A CN202110206114 A CN 202110206114A CN 112925809 A CN112925809 A CN 112925809A
- Authority
- CN
- China
- Prior art keywords
- data
- level cache
- level
- timestamps
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000013500 data storage Methods 0.000 title abstract description 15
- 230000001174 ascending effect Effects 0.000 claims abstract description 16
- 238000004590 computer program Methods 0.000 claims description 15
- 238000007726 management method Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides a data storage method, a device and a system, comprising the following steps: repeatedly executing the following steps until the remaining storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of the plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data sorted in the corresponding second-level caches; and storing the first data with the minimum time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large. By the method and the device, the problem of low data sorting efficiency is solved.
Description
Technical Field
The present invention relates to the field of communications, and in particular, to a method, an apparatus, and a system for storing data.
Background
With the increase of data needing to be processed intelligently and the increase of stored data, a single device cannot meet the performance requirement, and an intelligent cluster is generated. The intelligent cluster is mainly used for configuring cluster management service and displaying some data of the intelligent cluster; the intelligent management node is mainly used for managing each intelligent node; the intelligent nodes are mainly used for querying data, processing the queried data and the like, for example, detecting faces, recognizing faces and other intelligent service functions.
With the increase of the number of the devices, the capacity of the database of each device is inconsistent, and the difference of the data quantity inquired in unit time is large, if all the data of all the devices are inquired and then sequenced and displayed, the speed is quite slow, and a large amount of memory is consumed.
For the problem of low data sorting efficiency in the related art, no effective solution exists at present.
Disclosure of Invention
The embodiment of the invention provides a data storage method, a data storage device and a data storage system, which are used for at least solving the problem of low data sorting efficiency in the related technology.
According to an embodiment of the present invention, there is provided a data storage method including: repeatedly executing the following steps until the remaining storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches; and storing the first data with the minimum timestamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the timestamps from small to large.
Optionally, before the repeatedly performing the following steps until the remaining storage space of the third-level cache is less than or equal to the first threshold, the method further includes: and acquiring the data sent by the first-level cache through the second-level cache, wherein the second-level cache corresponds to the first-level cache one to one, and the first-level cache is arranged at a target node.
Optionally, obtaining, by the second-level cache, data sent by the first-level cache includes: and sending a data request to a corresponding target node under the condition that the residual space of the second-level cache is greater than or equal to a second threshold value so as to request the corresponding target node to send the data in the first-level cache to the second-level cache.
Optionally, the method further comprises: and displaying the data in the third-level cache according to the sequence of the timestamps from small to large.
Optionally, the target node is further configured to query data according to the obtained target request, timestamp the queried data, and store the queried data in the first-level cache according to a descending order of timestamps.
According to another embodiment of the present invention, there is provided a storage apparatus of data including: the execution module is used for repeatedly executing the comparison module and the storage module until the residual storage space of the third-level cache is less than or equal to the first threshold; the comparison module is used for comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches; and the storage module is used for storing the first data with the minimum time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large.
According to another embodiment of the present invention, there is provided a storage system of data including: the management node is used for repeatedly executing the following steps until the residual storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches; storing the first data with the minimum timestamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the timestamps from small to large, and the management node comprises the second-level cache and the third-level cache; and the target node is used for inquiring data according to the acquired target request, stamping timestamps on the inquired data, and storing the inquired data in the first-level cache according to the ascending sequence of the timestamps, wherein the target node comprises the first-level cache.
Optionally, the management node is further configured to obtain, through the second-level cache, data sent by the first-level cache, where the second-level cache corresponds to the first-level cache one to one.
According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
By the invention, the following steps are repeatedly executed until the residual storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of the plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data sorted in the corresponding second-level caches; and storing the first data with the minimum time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large. Therefore, the problem of low data sorting efficiency can be solved, and the effect of improving the data sorting efficiency is achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a block diagram of a hardware configuration of a mobile terminal according to a data storage method of an embodiment of the present invention;
FIG. 2 is a flow chart of a method of storing data according to an embodiment of the present invention;
FIG. 3 is an intelligent cluster node topology diagram in accordance with an alternative embodiment of the present invention;
FIG. 4 is a flow diagram of an intelligent cluster data processing routine in accordance with an alternative embodiment of the present invention;
FIG. 5 is a schematic diagram of an intelligent cluster level three cache architecture in accordance with an alternative embodiment of the present invention;
fig. 6 is a block diagram of a structure of a data storage device according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking an example of the method running on a mobile terminal, fig. 1 is a hardware structure block diagram of the mobile terminal of a data storage method according to an embodiment of the present invention. As shown in fig. 1, the mobile terminal 10 may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally may also include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the data storage method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, so as to implement the above-mentioned method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a method for storing data running on the mobile terminal is provided, and fig. 2 is a flowchart of a method for storing data according to an embodiment of the present invention, where as shown in fig. 2, the flowchart includes repeatedly performing the following steps until a remaining storage space of the third-level cache is less than or equal to a first threshold:
step S202, comparing timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data sorted first in the corresponding second-level caches;
step S204, storing the first data with the minimum time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large.
Through the steps, the following steps are repeatedly executed until the remaining storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of the plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data sorted in the corresponding second-level caches; and storing the first data with the minimum time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large. Therefore, the problem of low data sorting efficiency can be solved, and the effect of improving the data sorting efficiency is achieved.
Alternatively, the execution subject of the above steps may be a terminal or the like, but is not limited thereto.
As an optional implementation manner, as shown in fig. 3, the intelligent cluster node topology according to an optional embodiment of the present invention is shown, where the intelligent management node is mainly used to manage each intelligent node, and the intelligent node may query data, process the data, and other functions, for example, some intelligent service functions such as detecting faces and recognizing faces through processing image data. In this embodiment, the intelligent node may be a server node, e.g., an IVSS node.
As an optional implementation manner, the data may be picture data, and taking the intelligent cluster node including IVSS1, IVSS2 and IVSS3 nodes, and searching for pictures through the IVSS nodes as an example, as shown in fig. 4, a flow chart of processing timing of the intelligent cluster data according to an optional embodiment of the present invention is shown, where the flow chart includes the following steps:
s101: the WEB sends clue pictures to the intelligent management node;
s102: the intelligent management node sends the clue picture to IVSS1 for feature extraction;
s103: VSS1 returns the feature value of the thread picture;
s104: the intelligent management node sends the characteristic values of the thread pictures to IVSS1 for searching the pictures;
s105: the intelligent management node sends the characteristic values of the thread pictures to IVSS2 for searching the pictures;
s106: the intelligent management node sends the characteristic values of the thread pictures to IVSS3 for searching the pictures;
s107: the IVSS1 returns the searched target picture and information;
s108: the IVSS2 returns the searched target picture and information;
s109: the IVSS3 returns the searched target picture and information;
s110: and sorting the returned target pictures according to time and returning the target pictures to the WEB for displaying.
Optionally, before the repeatedly performing the following steps until the remaining storage space of the third-level cache is less than or equal to the first threshold, the method further includes: and acquiring the data sent by the first-level cache through the second-level cache, wherein the second-level cache corresponds to the first-level cache one to one, and the first-level cache is arranged at a target node.
As an alternative implementation, as shown in fig. 5, a schematic diagram of a three-level cache structure of an intelligent cluster according to an alternative embodiment of the present invention is shown, where an intelligent management node includes a second-level cache and a third-level cache, and an intelligent node IVSS includes a first-level cache. Suppose that in a scenario where the database capacity of each IVSS device is greatly different, IVSS1 has a 5000W historical picture database, IVSS2 has a 1000W historical picture database, and IVSS3 has a 10W historical picture database. The number of target pictures inquired in unit time and the inquiry speed of the target pictures are greatly different, all results inquired by the target pictures need to be sorted according to time, if the results are sorted after all devices finish inquiring, a large memory is needed for loading the inquired results, and a long time is needed for sorting. In the embodiment, the speed of searching the graph by the graph is improved by designing the three levels of buffer queues.
As an alternative, each IVSS device sets a first-level buffer queue, and the size of the first-level buffer queue may be 2 times that of the second-level buffer queue. Taking data as an example of image data, when the IVSS equipment receives an image searching message, image searching is started, and a result is placed in a first-level cache. The filling of the queue is suspended after the queue is full. The first level buffer queue sets a threshold value K1, when the number of queue idleness is lower than the threshold value, the graph searching service on the device is continuously started until the queue is full and stops.
The intelligent management node is provided with a second-level cache queue for storing the graph searching results sent by the corresponding first-level cache, and the pictures stored in the second-level cache queue are arranged according to the sequence of the timestamps from small to large. One IVSS device corresponds to one second-level buffer queue, and the size of the second-level buffer queue is 2 times that of the third-level buffer queue. And setting a threshold value K2 for the second-level cache queue, requesting data from the first-level cache when the data amount corresponding to the second-level cache is lower than the threshold value, and sending the image searching result to the second-level cache queue by the first-level cache queue.
The third-level cache queue is arranged on the intelligent management node, and an intelligent cluster can be only provided with one third-level cache queue. The size of the queue is fixed N, N is an integer, and the size can be determined according to actual conditions. And when the residual storage space of the third-level cache queue is larger than the threshold value, comparing the head element in each second-level cache queue, and taking the head element with earlier time to store in the third-level cache queue. Specifically, since the pictures are stored in the second-level cache in the order of the timestamps from small to large, the timestamp size of the first picture in each second-level cache can be compared respectively, the picture with the smallest timestamp is stored in the third-level cache queue, then the next round of timestamp comparison is performed, the picture with the smallest timestamp is selected each time to be stored, and thus the pictures in the third-level cache queue are sorted in the order of the timestamps from small to large until the remaining space of the third-level cache is smaller than the threshold value. In this embodiment, the threshold value may be determined according to actual situations, and is not limited herein. The third level cache queue size may be N; the second level cache queue size may be 2N; the size of the first-level cache queue may be 4N, N may be an integer, and the specific size may be determined according to actual situations.
Optionally, obtaining, by the second-level cache, data sent by the first-level cache includes: and sending a data request to a corresponding target node under the condition that the residual space of the second-level cache is greater than or equal to a second threshold value so as to request the corresponding target node to send the data in the first-level cache to the second-level cache.
As an optional implementation manner, when the number of elements in the second-level cache queue is lower than the preset threshold, or the remaining space of the second-level cache queue is greater than or equal to the preset threshold, the preset threshold may be determined according to an actual situation. The second level cache requests data from the first level cache queue.
Optionally, the method further comprises: and displaying the data in the third-level cache according to the sequence of the timestamps from small to large.
As an optional implementation manner, taking a data bit picture as an example, the third-level cache queue may sort the stored pictures according to time and return the sorted pictures to the WEB for presentation.
Optionally, the target node is further configured to query data according to the obtained target request, timestamp the queried data, and store the queried data in the first-level cache according to a descending order of timestamps.
As an optional implementation manner, the target node may be an IVSS device, and when the number of elements in the first-level cache queue is lower than a preset threshold, or the remaining storage space of the first-level cache queue is greater than or equal to the preset threshold, the graph searching function is started until the first-level cache queue is full, and then the graph searching function on the device is suspended. The IVSS equipment acquires a data query request sent by the intelligent management equipment, for example, after querying a target picture according to the characteristics of a clue picture, the IVSS equipment stamps timestamps on the target picture, stores the target picture in a first-level cache queue according to the sequence of the timestamps from small to large, and establishes the association relationship between the target picture and the timestamps. By the embodiment, the third-level cache can quickly supplement queue elements, so that the search speed is maximized; the mechanism enables the image searching and inquiring speed to be much faster than that of a single device, the inquired data volume is much larger, and the intelligent resource of the IVSS can be reasonably utilized.
According to the method and the device, a three-level cache mechanism is set in the IVSS device and the intelligent management node, the size of each level of cache has a dependency relationship, the transceiving of the first level cache queue element depends on the transceiving of the second level cache queue, and the transceiving of the second level cache queue element depends on the transceiving of the third level cache queue element. The method can achieve that the third-level cache queue can quickly acquire the graph searching result elements which are well sorted according to time. The advantages are that: the method can quickly search the images of a large number of images and can sort the images well.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
In this embodiment, a data storage device is further provided, and the data storage device is used to implement the foregoing embodiments and preferred embodiments, which have already been described and are not described again. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 6 is a block diagram of a structure of a data storage apparatus according to an embodiment of the present invention, as shown in fig. 6, the apparatus including: an execution module 62, configured to repeatedly execute the comparison module and the storage module until a remaining storage space of the third-level cache is smaller than or equal to the first threshold; a comparing module 64, configured to compare timestamps of a plurality of first data, where each of the plurality of first data is stored in a different second-level cache, the data stored in the second-level cache are sorted in an order from a smaller timestamp to a larger timestamp, and the first data is data sorted first in the corresponding second-level cache; the storage module 66 is configured to store the first data with the smallest timestamp in a third-level cache, where the data stored in the third-level cache are sorted according to the sequence of the timestamps from small to large.
Optionally, the above apparatus is further configured to, before the repeatedly performing the following steps until a remaining storage space of the third-level cache is smaller than or equal to a first threshold, obtain, by the second-level cache, data sent by the first-level cache, where the second-level cache corresponds to the first-level cache one to one, and the first-level cache is disposed in a target node.
Optionally, the apparatus is further configured to send a data request to a corresponding target node when the remaining space of the second-level cache is greater than or equal to a second threshold, so as to request the corresponding target node to send the data in the first-level cache to the second-level cache.
Optionally, the apparatus is further configured to display the data in the third-level cache in an order from small to large according to the timestamps.
Optionally, the target node is further configured to query data according to the obtained target request, timestamp the queried data, and store the queried data in the first-level cache according to a descending order of the timestamps.
In this embodiment, a data storage system is further provided, including: the management node is used for repeatedly executing the following steps until the residual storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches; storing the first data with the minimum timestamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the timestamps from small to large, and the management node comprises the second-level cache and the third-level cache; and the target node is used for inquiring data according to the acquired target request, stamping timestamps on the inquired data, and storing the inquired data in the first-level cache according to the ascending sequence of the timestamps, wherein the target node comprises the first-level cache.
Optionally, the management node is further configured to obtain, through the second-level cache, data sent by the first-level cache, where the second-level cache corresponds to the first-level cache one to one.
Optionally, the management node is further configured to send a data request to a corresponding target node to request the corresponding target node to send the data in the first-level cache to the second-level cache when the remaining space of the second-level cache is greater than or equal to a second threshold.
Optionally, the management node is further configured to display the data in the third-level cache in an order from small to large according to the timestamps.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Optionally, in this embodiment, the storage medium may be configured to store a computer program for repeatedly performing the following steps until the remaining storage space of the third level cache is less than or equal to the first threshold:
s1, comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data sorted first in the corresponding second-level caches;
s2, storing the first data with the smallest time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to repeatedly execute, by the computer program, the following steps until the remaining storage space of the third-level cache is less than or equal to the first threshold:
s1, comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data sorted first in the corresponding second-level caches;
s2, storing the first data with the smallest time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for storing data, comprising:
repeatedly executing the following steps until the remaining storage space of the third-level cache is less than or equal to the first threshold value:
comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches;
and storing the first data with the minimum timestamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the timestamps from small to large.
2. The method of claim 1, wherein before the repeatedly performing the following steps until the remaining storage space of the third level cache is less than or equal to the first threshold, the method further comprises:
and acquiring the data sent by the first-level cache through the second-level cache, wherein the second-level cache corresponds to the first-level cache one to one, and the first-level cache is arranged at a target node.
3. The method of claim 2, wherein retrieving, by the second level cache, data sent by the first level cache comprises:
and sending a data request to a corresponding target node under the condition that the residual space of the second-level cache is greater than or equal to a second threshold value so as to request the corresponding target node to send the data in the first-level cache to the second-level cache.
4. The method of claim 1, further comprising:
and displaying the data in the third-level cache according to the sequence of the timestamps from small to large.
5. The method according to any one of claims 1 to 4, wherein the target node is further configured to query data according to the obtained target request, timestamp the queried data, and store the queried data in the first-level cache in descending order of the timestamp.
6. An apparatus for storing data, comprising:
the execution module is used for repeatedly executing the comparison module and the storage module until the residual storage space of the third-level cache is less than or equal to the first threshold;
the comparison module is used for comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches;
and the storage module is used for storing the first data with the minimum time stamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the time stamps from small to large.
7. A system for storing data, comprising:
the management node is used for repeatedly executing the following steps until the residual storage space of the third-level cache is less than or equal to the first threshold value: comparing the timestamps of a plurality of first data, wherein each first data in the plurality of first data is respectively stored in different second-level caches, the data stored in the second-level caches are sorted according to the ascending sequence of the timestamps, and the first data is the data which is sorted first in the corresponding second-level caches; storing the first data with the minimum timestamp in a third-level cache, wherein the data stored in the third-level cache are sorted according to the sequence of the timestamps from small to large, and the management node comprises the second-level cache and the third-level cache;
and the target node is used for inquiring data according to the acquired target request, stamping timestamps on the inquired data, and storing the inquired data in the first-level cache according to the ascending sequence of the timestamps, wherein the target node comprises the first-level cache.
8. The system of claim 7, further comprising:
the management node is further configured to obtain, through the second-level cache, data sent by the first-level cache, where the second-level cache corresponds to the first-level cache one to one.
9. A storage medium, in which a computer program is stored, wherein the program is executable by a terminal device or a computer to perform the method of any one of claims 1 to 5.
10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110206114.1A CN112925809A (en) | 2021-02-24 | 2021-02-24 | Data storage method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110206114.1A CN112925809A (en) | 2021-02-24 | 2021-02-24 | Data storage method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112925809A true CN112925809A (en) | 2021-06-08 |
Family
ID=76171596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110206114.1A Pending CN112925809A (en) | 2021-02-24 | 2021-02-24 | Data storage method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112925809A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160007874A (en) * | 2014-07-07 | 2016-01-21 | 삼성전자주식회사 | Electronic device having cache memory and method for operating thereof |
| CN105740264A (en) * | 2014-12-10 | 2016-07-06 | 北大方正集团有限公司 | Distributed XML database sorting method and apparatus |
| CN106649385A (en) * | 2015-11-02 | 2017-05-10 | 北京国双科技有限公司 | Data ranking method and device based on HBase database |
| CN106909624A (en) * | 2017-01-19 | 2017-06-30 | 中国科学院信息工程研究所 | A kind of real-time sorting consistence method of mass data |
| CN107209768A (en) * | 2014-12-31 | 2017-09-26 | 华为技术有限公司 | Method and apparatus for the expansible sequence of data set |
| CN107729375A (en) * | 2017-09-13 | 2018-02-23 | 微梦创科网络科技（中国）有限公司 | A kind of method and device of daily record data sequence |
-
2021
- 2021-02-24 CN CN202110206114.1A patent/CN112925809A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160007874A (en) * | 2014-07-07 | 2016-01-21 | 삼성전자주식회사 | Electronic device having cache memory and method for operating thereof |
| CN105740264A (en) * | 2014-12-10 | 2016-07-06 | 北大方正集团有限公司 | Distributed XML database sorting method and apparatus |
| CN107209768A (en) * | 2014-12-31 | 2017-09-26 | 华为技术有限公司 | Method and apparatus for the expansible sequence of data set |
| CN106649385A (en) * | 2015-11-02 | 2017-05-10 | 北京国双科技有限公司 | Data ranking method and device based on HBase database |
| CN106909624A (en) * | 2017-01-19 | 2017-06-30 | 中国科学院信息工程研究所 | A kind of real-time sorting consistence method of mass data |
| CN107729375A (en) * | 2017-09-13 | 2018-02-23 | 微梦创科网络科技（中国）有限公司 | A kind of method and device of daily record data sequence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109117275B (en) | Account checking method and device based on data slicing, computer equipment and storage medium | |
| CN110413845B (en) | Resource storage method and device based on Internet of things operating system | |
| CN111538563A (en) | Event analysis method and device for Kubernetes | |
| CN104737157A (en) | A federated database system | |
| CN105468619B (en) | Resource allocation methods and device for database connection pool | |
| CN111324604A (en) | Database table processing method and device, electronic equipment and storage medium | |
| CN111355816B (en) | Server selection method, device, equipment and distributed service system | |
| CN113849499A (en) | Data query method and device, storage medium and electronic device | |
| CN111930770A (en) | Data query method and device and electronic equipment | |
| US20230037783A1 (en) | Resource scheduling method and related apparatus | |
| CN111159219B (en) | Data management method, device, server and storage medium | |
| CN110569129A (en) | Resource allocation method and device, storage medium and electronic device | |
| CN113111083B (en) | Method, apparatus, device, storage medium and program product for data query | |
| CN109101595A (en) | A kind of information query method, device, equipment and computer readable storage medium | |
| CN113434492A (en) | Data detection method and device, storage medium and electronic device | |
| US20200218708A1 (en) | Methods, Systems, Databases and Network Nodes of Data Communication Networks for Handling Data Posts | |
| CN112925809A (en) | Data storage method, device and system | |
| CN118656198A (en) | Data processing method, device, electronic device and storage medium | |
| CN114745275A (en) | Node updating method and device in cloud service environment and computer equipment | |
| CN117874033A (en) | Database sub-table method and device, electronic equipment and storage medium | |
| CN110716938A (en) | Data aggregation method and device, storage medium and electronic device | |
| CN110362706B (en) | Data searching method and device, storage medium and electronic device | |
| CN110213314B (en) | Method, device and server for determining storage node | |
| CN113268327A (en) | Transaction request processing method and device and electronic equipment | |
| CN113905252A (en) | Data storage method and device for live broadcast room, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210608 |
|
| RJ01 | Rejection of invention patent application after publication |