CN110737682A - cache operation method, device, storage medium and electronic equipment - Google Patents
cache operation method, device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN110737682A CN110737682A CN201910988274.9A CN201910988274A CN110737682A CN 110737682 A CN110737682 A CN 110737682A CN 201910988274 A CN201910988274 A CN 201910988274A CN 110737682 A CN110737682 A CN 110737682A
- Authority
- CN
- China
- Prior art keywords
- data
- cache space
- cache
- key value
- management information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000004140 cleaning Methods 0.000 claims description 57
- 230000006870 function Effects 0.000 claims description 17
- 238000007726 management method Methods 0.000 description 103
- 238000010586 diagram Methods 0.000 description 8
- 238000013523 data management Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000013507 mapping Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2315—Optimistic concurrency control
- G06F16/2329—Optimistic concurrency control using versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses cache operation methods, devices, storage media and electronic equipment, which particularly comprise the steps of obtaining a user service request, determining a cache operation type, data hierarchical management information and a data characteristic value according to the user service request, combining the data hierarchical management information and the data characteristic value to form a cache space key value, and operating a cache space according to the cache operation type and the cache space key value.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to cache operation methods and apparatuses, a storage medium, and an electronic device.
Background
With the development of internet technology, more and more businesses provide business services to users through networks. The relevant data of these business services is usually stored in a business database, and provides queries in the business services, such as MySQL, Oracle, DB2, and other relational databases. Because the data of the relational database are all stored in the disk, the disk needs to be read, if the network access volume is large and frequent, the load of the database is increased, the speed is low, and the service requirement is seriously influenced. Therefore, caching database technologies, such as Redis databases, are currently emerging. The cache database technology stores the common data configured by the business system in a cache, and avoids reading the relational database in the disk as much as possible, thereby improving the business service capability and reducing the load of the relational database.
However, in the existing caching technology, the cached data is lack of management, the cached data after caching operation is difficult to track, and steps can further affect the subsequent online business service.
Disclosure of Invention
Aiming at the prior art, the embodiment of the invention discloses cache operation methods, which can effectively manage cache data, facilitate the tracking of the data after cache operation and effectively ensure online service.
Specifically, the cache operation methods provided by the embodiments of the present application include:
acquiring a user service request;
determining a caching operation type, data grading management information and a data characteristic value according to the user service request, wherein the data grading management information comprises a data version number, the data version number is used for representing data timeliness, and the data characteristic value is used for representing data uniqueness ;
combining the data hierarchical management information and the data characteristic values to form a cache space key value;
and operating the cache space according to the cache operation type and the cache space key value.
In a step further , the method includes,
when the cache operation type is write operation, the step of operating the cache space according to the cache operation type and the cache space key value comprises:
matching the cache space key value with the existing cache space key value;
if the matching is successful, writing the data into the storage position corresponding to the cache space key value;
if the matching is unsuccessful, recording the cache space key value, and writing data into a storage position corresponding to the cache space key value; the data is corresponding to the data characteristic value.
In a step further , the method includes,
when the cache operation type is a read operation, the step of operating the cache space according to the cache operation type and the cache space key value comprises:
matching the cache space key value with the existing cache space key value;
if the matching is successful, reading data from a storage position corresponding to the cache space key value, wherein the data is corresponding to the data characteristic value;
and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the cache space key value, reading data from the relational database, and synchronizing the data read from the relational database into the cache space, wherein the data is business service data which is configured in advance and stored in the relational database.
In a step further , the method includes,
when the data in the cache space needs to be updated in the relational database, the method further includes:
pushing the updated data to the relational database;
updating the data version number corresponding to the updated data, determining the new data version number, taking the data grading management information containing the new data version number as the new data grading management information, and taking the data grading management information containing the original data version number as the original data grading management information.
In a step further , the method includes,
after the step of determining the new version number, the method further includes:
generating a cleaning affair message, wherein the cleaning affair message comprises the original data grading management information and the set delay consumption time;
pushing the generated cleaning transaction message to the established asynchronous message queue;
after delaying the delay consumption time, acquiring the cleaning transaction message from the asynchronous message queue;
and determining data needing to be cleaned in the cache space according to the original data grading management information in the cleaning transaction message, and deleting the data needing to be cleaned from the cache space according to the set cleaning amount and the set cleaning time interval.
The embodiment of the invention discloses cache operation devices, which can effectively manage cache data, facilitate the tracking of the data after cache operation and effectively ensure online service.
Specifically, the device includes:
the receiving and sending module is used for acquiring a user service request;
a service function module, configured to determine, according to the user service request, a cache operation type, data hierarchy management information and a data characteristic value, where the data hierarchy management information includes a data version number, the data version number is used to indicate data validity, and the data characteristic value is used to indicate uniqueness of data;
the cache management module is used for combining the data hierarchical management information and the data characteristic value to form a cache space key value and operating a cache space according to the cache operation type and the cache space key value;
and the cache space is used for storing the business service data.
Further , the cache management module includes:
the key value generation module is used for combining the data hierarchical management information and the data characteristic values to form a cache space key value;
the operation module is used for operating the cache space according to the cache operation type and the cache space key value;
wherein, the operation module includes:
the write operation module is used for matching the cache space key value with the existing cache space key value; if the matching is successful, writing the data into the storage position corresponding to the cache space key value; if the matching is unsuccessful, recording the cache space key value, and writing data into a storage position corresponding to the cache space key value; the data is corresponding to the data characteristic value;
the read operation module is used for matching the cache space key value with the existing cache space key value; if the matching is successful, reading data from a storage position corresponding to the cache space key value, wherein the data is corresponding to the data characteristic value; and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the key value of the cache space, reading data from the relational database, and synchronizing the data read from the relational database into the cache space, wherein the relational database is used for storing business service data configured in advance by a business system.
In a step further , the method includes,
the service function module comprises a service operation module and a data updating module;
the service operation module is used for determining a caching operation type, data grading management information and a data characteristic value according to the user service request, wherein the data grading management information comprises a data version number, the data version number is used for indicating data validity, and the data characteristic value is used for indicating uniqueness of data;
the data updating module is used for pushing updated data to the relational database;
the cache management module further comprises:
and the version number updating module is used for updating the data version number corresponding to the updated data, determining the new data version number, taking the data grading management information containing the new data version number as the new data grading management information, and taking the data grading management information containing the original data version number as the original data grading management information.
The cache management module further comprises:
the clearing module is used for pushing the generated clearing affair message to the established asynchronous message queue, wherein the clearing affair message comprises the original data grading management information and the set delay consumption time; after delaying the delay consumption time, acquiring the cleaning transaction message from the asynchronous message queue; and determining data needing to be cleaned in the cache space according to the original data grading management information in the cleaning transaction message, and deleting the data needing to be cleaned from the cache space according to the set cleaning amount and the set cleaning time interval.
The embodiment of the application also discloses computer readable storage media, and the instructions can realize the steps in the cache operation method when being executed by a processor.
The embodiment of the present application further discloses kinds of electronic devices, including:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instruction from the memory and executing the instruction to realize the cache operation method.
To sum up, the embodiment of the present application discloses cache operation methods, apparatuses, storage media, and electronic devices, which set data hierarchical management information for data, combine the data hierarchical management information and the data characteristic value to form a cache space key value, and perform an operation on a cache space by using the cache space key value.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings according to these drawings without any creative effort.
Fig. 1 is a schematic diagram of an application scenario 100 according to an embodiment of the present application.
Fig. 2 is a flow chart illustrating a method 200 of cache operation according to an embodiment of the present disclosure.
Fig. 3 is a flow chart illustrating a method 300 for cache operation according to an embodiment of the present disclosure.
Fig. 4 is a flowchart illustrating a method 400 for implementing a cache space synchronization according to an embodiment of the present application.
Fig. 5 is a flowchart illustrating a method 500 for cleaning failure data according to an embodiment of the present application.
Fig. 6 is a schematic structural diagram of an apparatus 600 for implementing a cache operation according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of an apparatus 700 for implementing a cache operation according to an embodiment of the present application.
Fig. 8 is a schematic structural diagram of an electronic device 800 implementing a caching operation according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application , rather than all embodiments.
It is to be understood that the terms "," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The technical solution of the present invention will be described in detail with specific examples. Several of the following embodiments may be combined with each other and some details of the same or similar concepts or processes may not be repeated in some embodiments.
In order to enhance management of data in a cache space, the embodiment of the invention sets data hierarchical management information, wherein the data hierarchical management information at least comprises data version numbers, the data hierarchical management information is kinds of information capable of representing data service characteristics, during cache operation, the data hierarchical management information and data characteristic values can be combined to form a cache space key value, and the cache space key value is used for operating the cache space.
Fig. 1 is a schematic diagram of an application scenario 100 according to an embodiment of the present application, as shown in fig. 1, the application scenario 100 includes a user terminal 101, a service server 102, a relational database 103, and a service configuration system 104, where the service configuration system 104 stores configured service data in the relational database 103, in order to avoid frequent access to the relational database 103, the system stores parts of common service data in a cache space of the service server 102, and thereafter, the user terminal 101 initiates a service request to the service server 102, and the service server 102 performs the steps shown in fig. 2 according to the service request, so as to implement a cache operation on the cache space.
Fig. 2 is a flowchart illustrating a method 200 for implementing a cache operation according to an embodiment of the present application. As shown in fig. 2, the cache operation method includes:
step 201: and acquiring a user service request.
The user service request here is initiated by the user terminal 101. During the business service process, the online user may access the business server 102, and the business server 102 obtains the user business request. The service request is associated with a specific service, such as a real estate transaction service request, a real estate origin query service request, or other task type service type.
And 202, determining a caching operation type, data grading management information and a data characteristic value according to the user service request, wherein the data grading management information comprises a data version number, the data version number is used for indicating data validity, and the data characteristic value is used for indicating the unique nature of data.
In practical applications, in executing a user service request, the service server 102 always operates the cache space according to service requirements, and the type of the cache operation may be determined according to a specific service. The type of cache operation may be a write operation or a read operation. Such as: when a user submits some user information, the service server 102 may determine that the cache operation type is write operation; when the user queries the house information, the service server 102 may determine that the caching operation type is a read operation. Thus, regardless of the service request, the service server 102 can determine its caching type when executing the user service request.
In , it can set three levels of service-scene-version, where "service" indicates what service the user service request belongs to, "scene" indicates what scene the service request belongs to in a certain service, and "version" indicates the timeliness of data.
The service server 102 may also determine the data characteristic value of the data in this step to represent -only of the data, the data may not be of the same size in size when accessed on-line services, and it is not convenient to directly transfer the data, if the data characteristic value is generated from the data, the data characteristic value of the size in length of the system is used instead of the real data.
Step 203: and combining the data hierarchical management information and the data characteristic values to form a cache space key value.
The cache space is accessed by adopting a mode of 'Key-Value', wherein 'Key' represents a cache space Key Value, and 'Value' represents cache data, which are corresponding to each other, according to the cache space Key Value, corresponding cache data can be obtained in the cache space, and the data can be stored in a corresponding storage area according to the cache space Key Value.
Step 204: and operating the cache space according to the cache operation type and the cache space key value.
As described above, this step may operate on the cache space using the cache space key value. If the cache operation type is write operation, storing the data in a storage space corresponding to the cache space key value; and if the cache operation type is read operation, reading data from the storage space corresponding to the cache space key value.
Therefore, in the embodiment 200 of the present application, data hierarchical management information is set for data, and the data hierarchical management information and the data feature value are combined into a cache space key value, so that the data and the data hierarchical management information form an association relationship, and the data in the cache space can be flexibly and accurately operated by using the data hierarchical management information, thereby enhancing management of the cache data.
In order to better describe the scheme of the application, the following detailed description is made by using other embodiments.
Fig. 3 is a flowchart illustrating a method for implementing a cache operation according to embodiment 300 of the present application. In the present embodiment, it is assumed that the service server 102 can implement a plurality of services, such as "real estate transaction service", "real estate subscription service", "real estate source service", and the like. Of course, the solution of the embodiment of the present application may also be not limited to real estate services, and may also implement other types of services. The data grading management information is set to be in a business-scene-version three-layer grade. As shown in fig. 3, the method includes:
step 301: and acquiring a user service request.
This step is the same as step 201, and is not described herein again.
Step 302: and determining a caching operation type, data grading management information and a data characteristic value according to the user service request, wherein the data grading management information comprises a service prefix, a scene classification number and a data version number.
The service Prefix corresponds to a service level and is used for isolating different services, the scene classification number corresponds to a scene level and is used for isolating different scene functions under the same service, and the data Version number corresponds to a data characteristic value and is used for representing data validity, Md5 and is used for representing the unique property of data, and the data Version number can be calculated through a hash function and other methods.
In practical applications, after receiving the service request, the service server 102 will start the backend framework and initialize the mapping relationship between the three levels of service-scene-version, and subsequently, the service server 102 can determine the corresponding hierarchical data management information directly from the mapping relationship in the process of executing the service request, table is an example of the hierarchical data management information mapping relationship, as shown in table :
TABLE
It is assumed that the service a corresponds to two scenes X and Y, and data version numbers corresponding to the scenes X and Y are initialized to 01. Accordingly, the service B corresponds to three scenes S, T and U, and the data version numbers corresponding to the scenes S, T and U are initialized to 01. Such as: the service prefix A corresponds to "real estate transaction service", the service prefix B corresponds to "real estate signing service", the scene classification number X corresponds to "buyer and seller user information", the scene classification number Y corresponds to "house information", the scene classification number S corresponds to "buyer and seller user information", the scene classification number T corresponds to "house information", and the scene classification number U corresponds to "fund information". Then it can be determined to belong to the "buyer and seller user information" scenario in the "real estate transaction service" according to the service request, and its data version number is determined to be 01 according to the mapping relationship, then the data rating management information should be "a-X-01".
Step 303: and combining the data hierarchical management information and the data characteristic values to form a cache space key value.
After determining the data-level management information and the data characteristic value, this step combines them to form a cache space Key value, i.e., "Key" ═ Prefix "+" Category "+" Version "+" Md5 ". Assuming that the data characteristic value Md5 is "abc", the cache space key value formed by combining the data hierarchy management information "a-X-01" and the data characteristic value "abc" in the above example should be "a-X-01-abc". Of course, the representation herein is merely a logical example for illustrating the principle method, and should be determined by actual circumstances, and not as a limitation to the scope of the present implementation.
Step 304: judging the cache operation type, and executing the step 305 when the cache operation type is write operation; step 308 is performed when the cache operation type is a read operation.
Step 305: and matching the cache space key value with the existing cache space key value.
Step 306: and if the matching is successful, writing the data into the storage position corresponding to the cache space key value, and finishing the cache operation.
Step 307: and if the matching is unsuccessful, recording the cache space key value, writing the data into a storage position corresponding to the cache space key value, and finishing the cache operation.
If the cache space has a certain cache space Key Value "Key", the corresponding storage location can be determined according to the cache space Key Value "Key", the data to be cached is written into the storage location, and the write operation is completed.
Step 308: and matching the cache space key value with the existing cache space key value.
Step 309: and if the matching is successful, reading data from the storage position corresponding to the cache space key value, and finishing the cache operation.
Step 310: and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the cache space key value, reading data from the relational database, synchronizing the data read from the relational database into the cache space, and finishing the caching operation.
The steps 308 to 310 are processes of reading the buffer space. Similarly, if a cache space has a certain cache space Key value "Key", the corresponding storage location can be determined according to the cache space Key value "Key", and data is read from the storage location, thereby completing the read operation. If the cache space does not have the cache space Key value "Key", it indicates that the cache space does not store the data, and it needs to read from the relational database 103. The relational database 103 in this embodiment may be a database of the type MySQL, Oracle, DB2, or the like. Taking the MySQL database as an example, the information for accessing the database may be used as parameters of the data characteristic values in advance, when reading the cache space fails, the access information is obtained from the data characteristic value of the cache space Key value "Key", and the access to the MySQL database is realized according to the access information.
The embodiment 300 exemplifies a specific scheme for performing fine management on cache data, sets three layers of data hierarchical management information, and combines the hierarchical management information with data characteristic values to form a cache space key value. Then, the business, scene, version number of data and data characteristic value to which the data belongs can be made clear through the cache space key value, so that the operation can be flexibly and accurately performed, and the management of the cache data is enhanced.
In practical application, besides performing basic read-write operation on the cache space, other operations may be performed, as described above, the service configuration system 104 stores the configured service data in the relational database 103, and stores the commonly used service data in the cache space of the service server 102, but in practical application, the service configuration system 104 may update the data and push the updated data to the relational database 103, at this time, the data stored in the cache space and the data in the relational database 103 are not .
Fig. 4 is a flowchart illustrating a method 400 for implementing a cache space synchronization according to an embodiment of the present application, as shown in fig. 4, the method includes:
step 401: and when the data in the cache space needs to be updated in the relational database, pushing the updated data to the relational database.
In practical applications, the business configuration system 104 may push updated data to the relational database 103.
Step 402: updating the data version number corresponding to the updated data, determining the new data version number, taking the data grading management information containing the new data version number as the new data grading management information, and taking the data grading management information containing the original data version number as the original data grading management information.
For example, "version number self-increment interfaces" may be set in practical application, and a cycle self-increment rule is defined, and the original data version number is sent to the "version number self-increment interface" to obtain a new data version number, for example, the data version number may be defined to cycle self-increment from "01" to "09" to avoid using too many bits to represent the version number, that is, the initial value of the data version number of a certain data is "01", the updated version number is "02", and so on.
Since the data version number is updated, when the data is read in the cache space in the subsequent service request execution process, the following steps are executed to realize data synchronization:
step 403: and determining the cache operation type as a read operation according to the user service request, and determining new data hierarchical management information and a data characteristic value, wherein the new data hierarchical management information comprises a new data version number.
It is assumed here that data whose data version number has been updated needs to be read from the buffer space.
Step 404: and combining the new data hierarchical management information and the data characteristic value to form a new cache space key value. Here, as the data version number is updated, the constituent cache space key values have also been updated.
Step 405: and matching the formed new cache space key value with the existing cache space key value, and determining that the matching is unsuccessful.
Here, the match should be unsuccessful because the constructed cache space key has been updated but not yet recorded into the cache space.
Step 406: and accessing a relational database according to the formed data characteristic value encapsulated by the key value of the new cache space, reading data from the relational database, and synchronizing the data read from the relational database into the cache space.
It can be seen from the above steps 403 to 406 that the process is the same as the process of reading data in the embodiment 300, because the data version number is updated, the formed cache space key values cannot be matched, and the data must be read from the relational database 103 and synchronized to the cache space, and the relational database 103 has been updated in step 401, so that the data in the relational database 103 and the data in the cache space are maintained again, and the synchronization of the data is realized.
In order to clean up the invalid data in the cache space in time, embodiments of a method 500 for cleaning up the invalid data are proposed in the present application.
Fig. 5 is a flowchart illustrating a method 500 for cleaning failure data according to an embodiment of the present application. As shown in fig. 5, the method includes:
step 501: generating a cleaning transaction message including the raw data rating management information and the set delay consumption time.
In practical applications, when the new data version number is generated in step 402 of the above embodiment, a cleaning transaction message may be generated, where the cleaning transaction message described herein belongs to kinds of asynchronous messages, and its function is to provide information related to cleaning of stale data in a cache space, so the cleaning transaction message is referred to herein as a cleaning transaction message, since data hierarchy management information is set for data in the embodiment of the present application, corresponding data may be determined in the cache space by cleaning the data hierarchy management information in the transaction message, for example, the service prefix a corresponds to "real estate transaction service", the scene classification number X corresponds to "buyer and seller user information", the data hierarchy management information "a-X-01" represents data of the service a scene X and the data version number is "01", and if the data of the service a scene X has updated its data version number to "02", the cleaning transaction message "a-X-01" indicates that all data in the cache of the data hierarchy management information "a-X-01" needs to be cleaned.
The delay consumption time is the time for delaying cleaning to ensure that the data in the buffer space is not cleaned in the same buffer operation cycles to avoid generating errors.
Step 502: and pushing the generated cleaning transaction message to the established asynchronous message queue.
Step 503: and after delaying the delay consumption time, acquiring the cleaning transaction message from the asynchronous message queue.
The asynchronous message queue described above in steps 502 and 503 may be divided into two different message queues. Such as: the message queue in step 502 is used as a delay message queue, and the message queue in step 503 is used as a consumable message queue. That is, the generated cleaning transaction message is pushed to the delay message queue, after the delay consumption time is delayed, the cleaning transaction message enters the consumable message queue, and the cleaning transaction message in the consumable message queue is acquired to perform consumption processing. The term "consumption" as used herein means that the process is started by acquiring a clear transaction message from a message queue.
Step 504: and determining data needing to be cleaned in the cache space according to the original data grading management information in the cleaning transaction message, and deleting the data needing to be cleaned from the cache space according to the set cleaning amount and the set cleaning time interval.
As described above, the data hierarchy management information in the cleaning transaction message is the data hierarchy management information before data update, that is, the original hierarchy management information. According to the original grading management information, corresponding data before updating, or data corresponding to the original data version number, namely the data needing to be cleaned, can be determined in the cache space. In addition, in practical application, the cleaning amount and the cleaning time interval can be flexibly set. For example, the size of the cleaning amount may be set as S pieces of data, the cleaning time interval is T, and the data corresponding to the version number of S pieces of original data in the cache space is deleted every time T until all the data are cleaned.
By applying the scheme of the embodiment of the application, because the data hierarchical management information is set, the valid data and the invalid data are distinguished through the data version number in the data hierarchical management information, only the data needing to be cleaned is deleted in the cache space, and other data in the cache space can still be provided for the service server 102 to execute the user service request, so that the online service is not influenced.
The embodiment of the present application further provides cache operation devices, which may be specific embodiments of the service server 102, and may be implemented as independent physical devices, or may be implemented by being distributed on multiple physical devices, fig. 6 is a schematic structural diagram of the cache operation device 600, as shown in fig. 6, the device includes a transceiver module 601, a service function module 602, a cache management module 603, and a cache space 604, where:
the transceiver module 601 is configured to obtain a user service request.
A service function module 602, configured to determine, according to the user service request, a caching operation type, data hierarchy management information, and a data characteristic value, where the data hierarchy management information includes a data version number, the data version number is used to indicate data validity, and the data characteristic value is used to indicate uniqueness of data.
The cache management module 603 is configured to combine the data hierarchical management information and the data characteristic value to form a cache space key, and operate the cache space 604 according to the cache operation type and the cache space key.
The buffer space 604 is used for storing business service data.
That is, after acquiring the user service request, the transceiver module 601 sends it to the service function module 602; the service function module 602 determines a cache operation type, data hierarchical management information and a data characteristic value according to a user service request, wherein the data hierarchical management information includes a data version number; the cache management module 602 combines the data hierarchical management information and the data characteristic value to form a cache space key value, and operates the cache space 604 according to the cache operation type and the cache space key value.
In the embodiment 600 of the present application, data hierarchical management information is set for data, and the data hierarchical management information and the data characteristic value are combined into a cache space key value, so that the data and the data hierarchical management information form an association relationship, and the data in the cache space can be flexibly and accurately operated by using the data hierarchical management information, thereby enhancing management of the cache data.
Fig. 7 is a schematic structural diagram of another embodiments 700 of the cache operating apparatus according to the present application, as shown in fig. 7, the apparatus still includes a transceiver module 601, a service function module 602, a cache management module 603, and a cache space 604, where:
the business function module 602 includes a business operation module 605 and a data update module 606.
A service operation module 605, configured to determine, according to the user service request, a caching operation type, data hierarchical management information, and a data characteristic value, where the data hierarchical management information includes a data version number, the data version number is used to indicate data validity, and the data characteristic value is used to indicate uniqueness of data.
And a data updating module 606 for pushing the updated data to the relational database.
The cache management module 603 includes a version number update module 607, a key value generation module 608, an operation module 609, and a cleaning module 610.
A version number updating module 607, configured to update the data version number corresponding to the updated data, determine a new data version number, use the hierarchical data management information including the new data version number as the hierarchical new data management information, and use the hierarchical data management information including the original data version number as the hierarchical original data management information.
A key value generating module 608, configured to combine the data hierarchical management information and the data feature value to form a cache space key value.
The operation module 609 is configured to operate the cache space according to the cache operation type and the cache space key value. The operation module 609 may further include a write operation module 611 and a read operation module 612.
A cleaning module 610, configured to push a generated cleaning transaction message to an established asynchronous message queue, where the cleaning transaction message includes the original data hierarchical management information and the set delay consumption time; after delaying the delay consumption time, acquiring the cleaning transaction message from the asynchronous message queue; according to the original data hierarchical management information in the cleaning transaction message, determining the data needing to be cleaned in the cache space 604, and deleting the data needing to be cleaned from the cache space 604 according to the set cleaning amount and the set cleaning time interval.
The write operation module 611 is configured to match the cache space key value with an existing cache space key value; if the matching is successful, writing the data into the storage position corresponding to the cache space key value; if the matching is unsuccessful, recording the cache space key value, and writing data into a storage position corresponding to the cache space key value; the data is corresponding to the data characteristic value.
A read operation module 612, configured to match the cache space key value with an existing cache space key value; if the matching is successful, reading data from a storage position corresponding to the cache space key value, wherein the data is corresponding to the data characteristic value; and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the key value of the cache space, reading data from the relational database, and synchronizing the data read from the relational database into the cache space, wherein the relational database is used for storing business service data configured in advance by a business system.
That is, when a user service request is executed, the transceiving module 601 obtains the user service request; the service operation module 605 determines a cache operation type, data hierarchical management information and a data characteristic value according to the user service request, wherein the data hierarchical management information includes a data version number; the key value generation module 608 combines the data hierarchy management information and the data feature value to form a cache space key value. If the cache operation type is write operation, the write operation module 611 matches the cache space key value with an existing cache space key value; if the matching is successful, writing the data into the storage position corresponding to the cache space key value; if the matching is unsuccessful, recording the cache space key value, and writing data into a storage position corresponding to the cache space key value; the data is corresponding to the data characteristic value. If the cache operation type is a read operation, the read operation module 612 matches the cache space key value with an existing cache space key value; if the matching is successful, reading data from a storage position corresponding to the cache space key value, wherein the data is corresponding to the data characteristic value; and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the key value of the cache space, reading data from the relational database, and synchronizing the data read from the relational database into the cache space, wherein the relational database is used for storing business service data configured in advance by a business system.
In this way, the cache operating apparatus 700 can complete basic read/write operations on the cache space 604. Of course, other operations can be performed by the cache operation device in practical application.
For example, the cache operation apparatus 700 may implement a synchronization operation. When the data in the relational database needs to be updated, the data updating module 606 pushes the updated data to the relational database; the version number updating module 607 updates the version number corresponding to the updated data, determines a new version number, and uses the data classification management information including the new version number as the new data classification management information and uses the data classification management information including the original version number as the original data classification management information. Therefore, since the version number is updated, the cache space key values cannot be matched when the data is subsequently read, and the process of reading the data from the relational database is triggered, so that the purpose of synchronizing the data in the relational database to the cache space is realized.
For another example, the cache operation device 700 may also implement an operation of cleaning aging data. When the version number updating module 607 updates the data version number, the cleaning module 610 may be triggered to clean the invalid data in the cache space 604, so as to enhance the management of the cache space.
In practice, the computer-readable storage medium carries or more programs which, when executed, implement the cache operation apparatus described with reference to fig. 6, according to the embodiments disclosed herein, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, but is not limited to, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing, but is not limited to the scope of the claims of the present application.
As shown in fig. 8, an embodiment of the present invention further provides electronic devices, where an apparatus implementing the method according to the embodiment of the present application may be integrated, and as shown in fig. 8, it shows a schematic structural diagram of an electronic device according to the embodiment of the present invention, specifically:
the electronic device may include a memory 802 of more than or processors 801, or computer-readable storage media and a computer program stored on the memory and executable on the processor, the cache operation method described above when executing the program of the memory 802.
Specifically, in practical applications, the electronic device may further include a power supply 803, an input unit 804, an output unit 805, and the like. Those skilled in the art will appreciate that the configuration of the electronic device shown in fig. 8 is not intended to be limiting of the electronic device and may include more or fewer components than shown, or some components in combination, or a different arrangement of components. Wherein:
the processor 801 is the control center of the electronic device and connects various parts of the whole electronic device using various interfaces and lines to perform overall monitoring of the electronic device by running or executing software programs and/or modules stored in the memory 802 and calling up data stored in the memory 802 to perform various functions of the server and process data, alternatively, the processor 801 may include or more processing cores, preferably, the processor 801 may integrate an application processor which mainly handles operating systems, user interfaces, application programs, etc. and a modem processor which mainly handles wireless communications, it being understood that the modem processor may not be integrated into the processor 801.
The memory 802 may be used to store software programs and modules, i.e., the computer-readable storage media described above, the processor 801 executes various functional applications and data processing by executing the software programs and modules stored in the memory 802, the memory 802 may mainly include a program storage area, which may store an operating system, application programs required for at least functions, and the like, and a data storage area, which may store data created according to the use of a server, and the like, and the memory 802 may further include a high-speed random access memory, a non-volatile memory, such as at least magnetic disk storage devices, flash memory devices, or other volatile solid-state storage devices, and accordingly, the memory 802 may further include a memory controller to provide the processor 801 with access to the memory 802.
The electronic device may further include a power source 803 for supplying power to the various components, which may be logically connected to the processor 801 via a power management system, such that the power management system may be used to manage charging, discharging, and power consumption management functions, the power source 803 may further include any of or or more dc or ac power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The electronic device may further include an input unit 804, and the input unit 804 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
The electronic device may further include an output unit 805, and the output unit 805 may be used to display information input by or provided to a user and various graphical user interfaces, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 708 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-emitting diode (OLED), or the like.
The principles and embodiments of the present invention are explained herein using specific examples, which are provided only to help understanding the method and the core idea of the present invention, and are not intended to limit the present application. It will be appreciated by those skilled in the art that changes may be made in this embodiment and its broader aspects and without departing from the principles, spirit and scope of the invention, and that all such modifications, equivalents, improvements and equivalents as may be included within the scope of the invention are intended to be protected by the claims.
Claims (10)
1, A cache operation method, comprising:
acquiring a user service request;
determining a caching operation type, data grading management information and a data characteristic value according to the user service request, wherein the data grading management information comprises a data version number, the data version number is used for representing data timeliness, and the data characteristic value is used for representing data uniqueness ;
combining the data hierarchical management information and the data characteristic values to form a cache space key value;
and operating the cache space according to the cache operation type and the cache space key value.
2. The method according to claim 1, wherein if the cache operation type is a write operation, the step of operating the cache space according to the cache operation type and the cache space key value comprises:
matching the cache space key value with the existing cache space key value;
if the matching is successful, writing the data into the storage position corresponding to the cache space key value;
if the matching is unsuccessful, recording the cache space key value, and writing data into a storage position corresponding to the cache space key value; the data is corresponding to the data characteristic value.
3. The method of claim 1, wherein if the cache operation type is a read operation, the step of operating the cache space according to the cache operation type and the cache space key value comprises:
matching the cache space key value with the existing cache space key value;
if the matching is successful, reading data from a storage position corresponding to the cache space key value, wherein the data is corresponding to the data characteristic value;
and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the cache space key value, reading data from the relational database, and synchronizing the data read from the relational database into the cache space, wherein the data is business service data which is configured in advance and stored in the relational database.
4. The method of claim 3, wherein when the data in the cache space needs to be updated in the relational database, the method further comprises:
pushing the updated data to the relational database;
updating the data version number corresponding to the updated data, determining the new data version number, taking the data grading management information containing the new data version number as the new data grading management information, and taking the data grading management information containing the original data version number as the original data grading management information.
5. The method of claim 4, wherein after the step of determining the new version number, the method further comprises:
generating a cleaning affair message, wherein the cleaning affair message comprises the original data grading management information and the set delay consumption time;
pushing the generated cleaning transaction message to the established asynchronous message queue;
after delaying the delay consumption time, acquiring the cleaning transaction message from the asynchronous message queue;
and determining data needing to be cleaned in the cache space according to the original data grading management information in the cleaning transaction message, and deleting the data needing to be cleaned from the cache space according to the set cleaning amount and the set cleaning time interval.
A cache operating apparatus of , comprising:
the receiving and sending module is used for acquiring a user service request;
a service function module, configured to determine, according to the user service request, a cache operation type, data hierarchy management information and a data characteristic value, where the data hierarchy management information includes a data version number, the data version number is used to indicate data validity, and the data characteristic value is used to indicate uniqueness of data;
the cache management module is used for combining the data hierarchical management information and the data characteristic value to form a cache space key value and operating a cache space according to the cache operation type and the cache space key value;
and the cache space is used for storing the business service data.
7. The apparatus of claim 6, wherein the cache management module comprises:
the key value generation module is used for combining the data hierarchical management information and the data characteristic values to form a cache space key value;
the operation module is used for operating the cache space according to the cache operation type and the cache space key value;
wherein, the operation module includes:
the write operation module is used for matching the cache space key value with the existing cache space key value; if the matching is successful, writing the data into the storage position corresponding to the cache space key value; if the matching is unsuccessful, recording the cache space key value, and writing data into a storage position corresponding to the cache space key value; the data is corresponding to the data characteristic value;
the read operation module is used for matching the cache space key value with the existing cache space key value; if the matching is successful, reading data from a storage position corresponding to the cache space key value, wherein the data is corresponding to the data characteristic value; and if the matching is unsuccessful, accessing a relational database according to the data characteristic value encapsulated by the key value of the cache space, reading data from the relational database, and synchronizing the data read from the relational database into the cache space, wherein the relational database is used for storing business service data configured in advance by a business system.
8. The apparatus of claim 7, wherein the service function module comprises a service operation module, and further comprises a data update module;
the service operation module is used for determining a caching operation type, data grading management information and a data characteristic value according to the user service request, wherein the data grading management information comprises a data version number, the data version number is used for indicating data validity, and the data characteristic value is used for indicating uniqueness of data;
the data updating module is used for pushing updated data to the relational database;
the cache management module further comprises:
a version number updating module for updating the data version number corresponding to the updated data, determining the new data version number, using the data classification management information containing the new data version number as the new data classification management information, and using the data classification management information containing the original data version number as the original data classification management information;
the clearing module is used for pushing the generated clearing affair message to the established asynchronous message queue, wherein the clearing affair message comprises the original data grading management information and the set delay consumption time; after delaying the delay consumption time, acquiring the cleaning transaction message from the asynchronous message queue; and determining data needing to be cleaned in the cache space according to the original data grading management information in the cleaning transaction message, and deleting the data needing to be cleaned from the cache space according to the set cleaning amount and the set cleaning time interval.
A computer readable storage medium having computer instructions stored thereon which, when executed by a processor, perform the steps of the method of caching as claimed in any one of claims 1 to 5 to .
10, electronic device, characterized in that, the electronic device comprises:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910988274.9A CN110737682A (en) | 2019-10-17 | 2019-10-17 | cache operation method, device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910988274.9A CN110737682A (en) | 2019-10-17 | 2019-10-17 | cache operation method, device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110737682A true CN110737682A (en) | 2020-01-31 |
Family
ID=69270090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910988274.9A Withdrawn CN110737682A (en) | 2019-10-17 | 2019-10-17 | cache operation method, device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110737682A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111352957A (en) * | 2020-02-25 | 2020-06-30 | 京东数字科技控股有限公司 | Remote dictionary service optimization method and related equipment |
| CN111708958A (en) * | 2020-06-12 | 2020-09-25 | 中国工商银行股份有限公司 | Multi-key value cache storage method, device and system |
| CN111737297A (en) * | 2020-06-15 | 2020-10-02 | 中国工商银行股份有限公司 | Method and device for processing link aggregation call information |
| CN111899088A (en) * | 2020-06-23 | 2020-11-06 | 四川新网银行股份有限公司 | Accurate asset limit calculation method under high-concurrency data flow field scene |
| CN111930781A (en) * | 2020-07-24 | 2020-11-13 | 北京金山云网络技术有限公司 | Method and device for processing data request of cache database |
| CN112035437A (en) * | 2020-08-31 | 2020-12-04 | 平安医疗健康管理股份有限公司 | Method and device for transmitting medical record data, computer equipment and storage medium |
| CN112187849A (en) * | 2020-08-06 | 2021-01-05 | 深圳市麦谷科技有限公司 | Cache data dynamic generation method and system |
| CN112860794A (en) * | 2021-02-03 | 2021-05-28 | 百果园技术(新加坡)有限公司 | Cache-based concurrency capability improving method, device, equipment and storage medium |
| CN113934749A (en) * | 2021-10-14 | 2022-01-14 | 中国软件与技术服务股份有限公司 | Database persistence method based on cache |
| CN114969003A (en) * | 2022-05-26 | 2022-08-30 | 平安付科技服务有限公司 | Method and device for cleaning tablespace, electronic equipment and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622361A (en) * | 2011-01-28 | 2012-08-01 | 天脉聚源(北京)传媒科技有限公司 | Method for querying database |
| CN104283956A (en) * | 2014-09-30 | 2015-01-14 | 腾讯科技（深圳）有限公司 | Strong consistence distributed data storage method, device and system |
-
2019
- 2019-10-17 CN CN201910988274.9A patent/CN110737682A/en not_active Withdrawn
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622361A (en) * | 2011-01-28 | 2012-08-01 | 天脉聚源(北京)传媒科技有限公司 | Method for querying database |
| CN104283956A (en) * | 2014-09-30 | 2015-01-14 | 腾讯科技（深圳）有限公司 | Strong consistence distributed data storage method, device and system |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111352957A (en) * | 2020-02-25 | 2020-06-30 | 京东数字科技控股有限公司 | Remote dictionary service optimization method and related equipment |
| CN111708958A (en) * | 2020-06-12 | 2020-09-25 | 中国工商银行股份有限公司 | Multi-key value cache storage method, device and system |
| CN111708958B (en) * | 2020-06-12 | 2024-04-12 | 中国工商银行股份有限公司 | Multi-key value cache storage method, device and system |
| CN111737297B (en) * | 2020-06-15 | 2023-08-25 | 中国工商银行股份有限公司 | Method and device for processing link aggregation call information |
| CN111737297A (en) * | 2020-06-15 | 2020-10-02 | 中国工商银行股份有限公司 | Method and device for processing link aggregation call information |
| CN111899088A (en) * | 2020-06-23 | 2020-11-06 | 四川新网银行股份有限公司 | Accurate asset limit calculation method under high-concurrency data flow field scene |
| CN111930781A (en) * | 2020-07-24 | 2020-11-13 | 北京金山云网络技术有限公司 | Method and device for processing data request of cache database |
| CN112187849A (en) * | 2020-08-06 | 2021-01-05 | 深圳市麦谷科技有限公司 | Cache data dynamic generation method and system |
| CN112035437B (en) * | 2020-08-31 | 2024-02-13 | 平安医疗健康管理股份有限公司 | Transmission method and device for medical records data, computer equipment and storage medium |
| CN112035437A (en) * | 2020-08-31 | 2020-12-04 | 平安医疗健康管理股份有限公司 | Method and device for transmitting medical record data, computer equipment and storage medium |
| CN112860794A (en) * | 2021-02-03 | 2021-05-28 | 百果园技术(新加坡)有限公司 | Cache-based concurrency capability improving method, device, equipment and storage medium |
| CN113934749A (en) * | 2021-10-14 | 2022-01-14 | 中国软件与技术服务股份有限公司 | Database persistence method based on cache |
| CN114969003A (en) * | 2022-05-26 | 2022-08-30 | 平安付科技服务有限公司 | Method and device for cleaning tablespace, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110737682A (en) | cache operation method, device, storage medium and electronic equipment | |
| CN109254733B (en) | Method, device and system for storing data | |
| CN107391653B (en) | Distributed NewSQL database system and picture data storage method | |
| US8548945B2 (en) | Database caching utilizing asynchronous log-based replication | |
| CN111258609A (en) | Upgrading method and device of Kubernetes cluster, electronic equipment and medium | |
| EP2863310B1 (en) | Data processing method and apparatus, and shared storage device | |
| CN107704202B (en) | Method and device for quickly reading and writing data | |
| JP2020140699A (en) | Methods, equipment, equipment, and media for storing and querying data | |
| CN104104717A (en) | Inputting channel data statistical method and device | |
| CN107016027A (en) | The method and apparatus for realizing business information fast search | |
| US9323798B2 (en) | Storing a key value to a deleted row based on key range density | |
| CN111625552A (en) | Data collection method, device, equipment and readable storage medium | |
| CN111046106A (en) | Cache data synchronization method, device, device and medium | |
| CN104423982A (en) | Request processing method and device | |
| CN104461929B (en) | Distributed data cache method based on blocker | |
| CN109992469B (en) | Method and device for merging logs | |
| CN109614411B (en) | Data storage method, device and storage medium | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN115114232A (en) | Method, device and medium for enumerating historical version objects | |
| CN114461950A (en) | A global caching method, apparatus, electronic device and storage medium | |
| WO2025011307A1 (en) | Data backup method and apparatus, data recovery method and apparatus, device, and storage medium | |
| CN111290714A (en) | Data reading method and device | |
| CN114860380B (en) | Data synchronization method, device, electronic device and computer-readable storage medium | |
| WO2024016789A1 (en) | Log data query method and apparatus, and device and medium | |
| US10067678B1 (en) | Probabilistic eviction of partial aggregation results from constrained results storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200131 |
|
| WW01 | Invention patent application withdrawn after publication |