CN106341402B - Processing method and device based on multi-architecture distributed cache - Google Patents

Abstract

The present invention provides a processing method and device based on multi-architecture distributed cache, wherein the method includes: when the application system starts, acquire and save the cache configuration information in the configuration center; The client instance corresponding to the application system; obtain the cached data according to the cache configuration information and the client instance corresponding to the current application system; add the current application system identifier to the key value of the obtained cached data so that each application system can share same cache cluster. The operation and maintenance cost of the cache is reduced; the application systems using the same cache product and the same version can partly or completely share the same set of cache resources, so as to realize the maximum utilization of hardware resources.

Description

Processing method and device based on multi-architecture distributed cache

technical field

The present invention relates to the technical field of the Internet, in particular to a processing method and device based on a multi-architecture distributed cache.

Background technique

With the development of Internet technology, when various websites and databases face large-scale data access, disk I/O often becomes a performance bottleneck, resulting in excessive response delays. Therefore, caching technology can be used to use high-speed memory as the storage medium of data objects, and the responses of the previous data requests of the terminal are distributed on the cache server, so that when the terminal sends the same data request, it does not need to send the data request to the server again. Instead of the data server, the cache server sends its cached copy response to the user, thereby reducing the bottleneck caused by disk I/O.

In the existing technology, an enterprise has dozens or even hundreds of different application systems, and the enterprise generally purchases several or even dozens of different IT service providers. The cache products and specific deployment solutions adopted by different suppliers will vary. There are differences. For example, commonly used cache clusters include Codis-Redis cache cluster, Twemproxy-Memcacheds cache cluster, Redis-Cluster cache cluster, and node cluster. In this way, the respective cache acquisitions of different application systems are performed.

However, in the existing technology, due to the lack of unified planning and deployment of the cache architecture and cache products adopted by different suppliers, each application system needs to build its own independent cache module, which in turn leads to repeated construction, thus increasing the operation of the cache. In addition, the cache products selected by various IT service providers are different, and the versions of the same product are also different, which makes it impossible for enterprises to carry out effective unified management and planning for cache and make full use of hardware resources.

Contents of the invention

The present invention provides a processing method and device based on a multi-architecture distributed cache, which is used to solve the problem that the operation and maintenance cost of the cache in the prior art is high, and the enterprise cannot effectively manage and plan the cache, and cannot fully manage the hardware resources. problem of utilization.

One aspect of the present invention is to provide a processing method based on a multi-architecture distributed cache, including:

When the application system starts, obtain and save the cache configuration information in the configuration center, the cache configuration information includes the current application system identifier;

Adapting a client instance corresponding to the current application system according to the cache configuration information;

Obtain cached data according to the cache configuration information and the client instance corresponding to the current application system;

Add the current application system identifier to the key value of the obtained cache data, so that each application system can share the same cache cluster.

Another aspect of the present invention is to provide a processing device based on a multi-architecture distributed cache, including:

The first obtaining module is used to obtain and save the cache configuration information in the configuration center when the application system is started, and the cache configuration information includes the current application system identifier;

A determining module, configured to adapt a client instance corresponding to the current application system according to the cache configuration information;

A cache acquisition module, configured to acquire cache data according to the cache configuration information and the client instance corresponding to the current application system;

The adding module is used to add the current application system identifier to the key value of the acquired cache data, so that each application system can share the same cache cluster.

The technical effect of the present invention is: when the application system is started, the cache configuration information in the configuration center is acquired and saved, the cache configuration information includes the current application system identifier; according to the cache configuration information, the client corresponding to the current application system is adapted Instance; obtain the cached data according to the cache configuration information and the client instance corresponding to the current application system; add the current application system identifier to the key value of the obtained cached data, so that each application system shares the same cache cluster. Therefore, the application system only needs to configure the cache product type, version, interface implementation type and other information to adapt to the appropriate cache client, and call the cache service through the client without caring about the background cache deployment method, thereby reducing the operation and maintenance of the cache Cost; at the same time, through simple client instance interface encapsulation, the application system can easily and conveniently change the cache product. It can manage heterogeneous cache systems in a unified way, provide a unified configuration center, and provide unified management for single-point, active-standby, distributed architectures, as well as Redis and Memcached; operation and maintenance personnel can manage cache device resources and applications through it The cache configuration of the system, the corresponding relationship between the application system and the device; you can also increase or decrease the cache nodes for the distributed system cache architecture in the configuration center, merge or split the single-point cache system; at the same time, make the use of the same cache product, the same version Application systems can partly or completely share the same set of cache resources, so as to realize the maximum utilization of hardware resources, and at the same time, the cache load of each application system can be manually adjusted; through the cache client adaptation module, the coupling between the cache system and the application system can be reduced sex.

Description of drawings

FIG. 1 is a flowchart of a processing method based on a multi-architecture distributed cache provided by Embodiment 1 of the present invention;

FIG. 2 is a flowchart of a processing method based on a multi-architecture distributed cache provided in Embodiment 2 of the present invention;

FIG. 3 is a schematic structural diagram of a processing device based on a multi-architecture distributed cache provided by Embodiment 3 of the present invention;

FIG. 4 is a schematic structural diagram of a processing device based on a multi-architecture distributed cache provided by Embodiment 4 of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Fig. 1 is a flowchart of a processing method based on a multi-architecture distributed cache provided by Embodiment 1 of the present invention. As shown in Fig. 1 , the method of this embodiment includes:

Step 101. When the application system is started, obtain and save cached configuration information in the configuration center, where the cached configuration information includes the current application system identifier.

In this embodiment, specifically, a configuration center, a cache client adaptation module deployed in each application system, and a cache resource pool are provided. Among them, the configuration center is used to save the cache configuration information of each application system and the existing cache cluster resource information, and provide high-availability configuration management services to the outside world. Each application system is configured with a cache client adaptation module. The cache resource pool includes server cache clusters of various heterogeneous cache systems. The cache resource pool includes Codis-Redis clusters, Twemproxy-Memcache clusters, Redis-Cluster clusters, and single-point, active and standby nodes of the two products.

The executor of this embodiment may be an apparatus and device provided with an application system and a caching client adaptation module. When the application system is started, the cache client adaptation module is invoked to read the cache configuration information in the configuration center, and at the same time, monitor the configuration fields of the configuration center. Then, the cache client adaptation module in the application system can save the read cache configuration information.

The cache configuration information includes the current application system identifier, cache information, and cache entry address, as well as the correspondence between the application system identifier, cache information, and cache entry address, and the number of cache nodes. Wherein, the cache information includes cache type, cache version number, client language and architecture type. Moreover, the cache entry address corresponds to the cache cluster.

For example, Table 1 is cache configuration information, as shown in Table 1 of cache configuration information.

Table 1 is the cache configuration information

Step 102, according to the cached configuration information, adapt the client instance corresponding to the current application system.

In this embodiment, specifically, the cache client adaptation module covers the enterprise's existing cache client collection in multiple programming languages, and can adapt a suitable client instance according to the cache configuration information read from the configuration center , so that the module can generate a client instance corresponding to a certain language and a certain architecture for use by the application system according to the cached configuration information in the configuration center. Furthermore, the client instance corresponding to the current application system can be adapted according to the cached configuration information.

Step 103: Obtain cached data according to the cache configuration information and the client instance corresponding to the current application system.

Wherein, the concrete realization mode of step 103 is:

Determine the cache entry address corresponding to the current application system identifier according to the cache configuration information;

Use the client instance corresponding to the current application system to obtain the cache data in the cache cluster corresponding to the cache entry address.

In this embodiment, specifically, the cache entry address corresponding to the current application system identifier is determined first according to the current application system identifier in the cache configuration information, the corresponding relationship between the application system identifier, cache information, and cache entry address. Then you can use the client instance corresponding to the current application system to obtain the cache data in the cache cluster corresponding to the cache entry address, and then adapt a suitable client instance according to the cache configuration information, so that the application system uses the generated client end instance to use cached data.

For example, the cache client adaptation module in application system A can adapt the client instance of the Redis client, the supported C language, architecture type Redis, cache version number 3.0, and the corresponding cache cluster is Redis-Cluster; The cache client adaptation module in application system B can adapt the client instance of Redis client, the supported JAVA language, architecture type Redis, cache version number 3.0, and the corresponding cache cluster is Redis-Cluster; in application system C The cache client adaptation module can adapt the client instance of the Redis client, the supported JAVA language and architecture type Redis, the cache version number is 2.9, and the corresponding cache cluster is Codis-Redis; the cache client in the application system D The adaptation module can adapt the client instance of the Memcached client, the supported JAVA language, architecture type Memcached, the cache version number 1.4.14, and the corresponding cache cluster is Twemproxy-Memcached; the cache client adaptation in the application system E The module can adapt the client instance of the Memcached client, the supported JAVA language, architecture type Memcached, the cache version number 1.4.14, and the corresponding cache cluster is Single.

Step 104: Add the current application system identifier to the key value of the obtained cache data, so that all application systems share the same cache cluster.

In this embodiment, specifically, the cache client adaptation module can add a prefix to the key value according to the application system name, thereby avoiding cache key value conflicts and performing logical isolation. Specifically, the cache client adaptation module will add the current application system identifier to the key value of the obtained cache data, and this current application system identifier is the unique identifier of the application system, thereby logically distinguishing the naming of each application system Space, so that each application system can share the same set of physical cache, that is, each application system shares the same cache cluster.

For example, application system A and application system B using the same version, the same product, and different client languages can share the same set of physical cache servers, that is, share the same cache cluster.

In this embodiment, when the application system is started, the cache configuration information in the configuration center is obtained and saved, and the cache configuration information includes the current application system identifier; according to the cache configuration information, a client instance corresponding to the current application system is adapted; The configuration information and the client instance corresponding to the current application system are used to obtain the cached data; the current application system identifier is added to the key value of the obtained cached data, so that each application system can share the same cache cluster. Therefore, the application system only needs to configure the cache product type, version, interface implementation type and other information to adapt to the appropriate cache client, and call the cache service through the client without caring about the background cache deployment method, thereby reducing the operation and maintenance of the cache Cost; at the same time, through simple client instance interface encapsulation, the application system can easily and conveniently change the cache product. It can manage heterogeneous cache systems in a unified way, provide a unified configuration center, and provide unified management for single-point, active-standby, distributed architectures, as well as Redis and Memcached; operation and maintenance personnel can manage cache device resources and applications through it The cache configuration of the system, the corresponding relationship between the application system and the device; you can also increase or decrease the cache nodes for the distributed system cache architecture in the configuration center, merge or split the single-point cache system; at the same time, make the use of the same cache product, the same version Application systems can partly or completely share the same set of cache resources, so as to realize the maximum utilization of hardware resources, and at the same time, the cache load of each application system can be manually adjusted; through the cache client adaptation module, the coupling between the cache system and the application system can be reduced sex.

FIG. 2 is a flow chart of the processing method based on multi-architecture distributed cache provided by Embodiment 2 of the present invention. On the basis of Embodiment 1, as shown in FIG. 2 , the method of this embodiment, before step 102, also includes :

Step 201. Obtain and save the modified cache configuration information in the configuration center, where the modified cache configuration information is obtained after the configuration center receives a modification instruction sent by the user and modifies the cache configuration information according to the modification instruction.

In this embodiment, specifically, the user can log in to the configuration center to configure the cache configuration information corresponding to the application system. Specifically, the user sends a modification instruction to the configuration center, and the configuration center modifies the cache configuration information according to the modification instruction. Get the modified cache configuration information. Furthermore, the cache client type, cache product type, cache version number, etc. can be modified; at the same time, the cache configuration information of each application system can be adjusted online according to the load of each cache node.

Since the cache client adaptation module is called to monitor the configuration field of the configuration center, the modified cache configuration information can be actively pushed to the cache client adaptation module of the application system, and then the cache client adaptation module of the application system The module can obtain and save the modified cache configuration information. Then, the application system can use the cache cluster according to the modified cache configuration information.

In this embodiment, when the application system is started, the cache configuration information in the configuration center is obtained and saved, and the cache configuration information includes the current application system identifier; and the modified cache configuration information in the configuration center can be obtained and saved; then according to the cache configuration Information, adapt the client instance corresponding to the current application system; obtain cached data according to the cache configuration information and the client instance corresponding to the current application system; add the current application system identifier to the key value of the obtained cached data , so that all application systems share the same cache cluster. Therefore, the application system only needs to configure the cache product type, version, interface implementation type and other information to adapt to the appropriate cache client, and call the cache service through the client without caring about the background cache deployment method, thereby reducing the operation and maintenance of the cache Cost; at the same time, through simple client instance interface encapsulation, the application system can easily and conveniently change the cache product. It can manage heterogeneous cache systems in a unified way, provide a unified configuration center, and provide unified management for single-point, active-standby, distributed architectures, as well as Redis and Memcached; operation and maintenance personnel can manage cache device resources and applications through it The cache configuration of the system, the corresponding relationship between the application system and the device; you can also increase or decrease the cache nodes for the distributed system cache architecture in the configuration center, merge or split the single-point cache system; after modifying the cache configuration information in the configuration center, you can The cache client adaptation module of all application systems pushes the updated cache configuration information; at the same time, application systems using the same cache product and version can share the same set of cache resources partially or completely, so as to realize the maximum utilization of hardware resources, and at the same time It is also possible to manually adjust the cache load of each application system; through the cache client adaptation module, the coupling between the cache system and the application system can be reduced.

FIG. 3 is a schematic structural diagram of a processing device based on a multi-architecture distributed cache provided in Embodiment 3 of the present invention. As shown in FIG. 3 , the device provided in this embodiment includes:

The first obtaining module 31 is used to obtain and save the cache configuration information in the configuration center when the application system is started, and the cache configuration information includes the current application system identifier;

The determining module 32 is configured to adapt the client instance corresponding to the current application system according to the cache configuration information;

A cache acquisition module 33, configured to acquire cache data according to the cache configuration information and the client instance corresponding to the current application system;

The adding module 34 is configured to add the current application system identifier to the key value of the acquired cache data, so that all application systems share the same cache cluster.

The processing device based on the multi-architecture distributed cache in this embodiment can execute the processing method based on the multi-architecture distributed cache provided in Embodiment 1 of the present invention, and its implementation principles are similar, so details will not be repeated here.

In this embodiment, when the application system is started, the cache configuration information in the configuration center is obtained and saved, and the cache configuration information includes the current application system identifier; according to the cache configuration information, a client instance corresponding to the current application system is adapted; The configuration information and the client instance corresponding to the current application system are used to obtain the cached data; the current application system identifier is added to the key value of the obtained cached data, so that each application system can share the same cache cluster. Therefore, the application system only needs to configure the cache product type, version, interface implementation type and other information to adapt to the appropriate cache client, and call the cache service through the client without caring about the background cache deployment method, thereby reducing the operation and maintenance of the cache Cost; at the same time, through simple client instance interface encapsulation, the application system can easily and conveniently change the cache product. It can manage heterogeneous cache systems in a unified way, provide a unified configuration center, and provide unified management for single-point, active-standby, distributed architectures, as well as Redis and Memcached; operation and maintenance personnel can manage cache device resources and applications through it The cache configuration of the system, the corresponding relationship between the application system and the device; you can also increase or decrease the cache nodes for the distributed system cache architecture in the configuration center, merge or split the single-point cache system; at the same time, make the use of the same cache product, the same version Application systems can partly or completely share the same set of cache resources, so as to realize the maximum utilization of hardware resources, and at the same time, the cache load of each application system can be manually adjusted; through the cache client adaptation module, the coupling between the cache system and the application system can be reduced sex.

FIG. 4 is a schematic structural diagram of a processing device based on a multi-architecture distributed cache provided in Embodiment 4 of the present invention. On the basis of Embodiment 3, as shown in FIG. 4 , the device provided in this embodiment, a cache acquisition module 33, includes :

The address determination sub-module 331 is configured to determine the cache entry address corresponding to the current application system identifier according to the cache configuration information;

The cache obtaining sub-module 332 is configured to use the client instance corresponding to the current application system to obtain cache data in the cache cluster corresponding to the cache entry address.

The device provided in this embodiment also includes:

The second obtaining module 41 is used to obtain and save the modified cache configuration information in the configuration center before the determination module 32 adapts the client instance corresponding to the current application system according to the cache configuration information, wherein the modified The cache configuration information is obtained after the configuration center receives the modification instruction sent by the user and modifies the cache configuration information according to the modification instruction.

The cache configuration information also includes at least one of the following: application system identification, cache information, and the correspondence between cache entry addresses, and the number of cache nodes;

Wherein, the cache information includes cache type, cache version number, client language and architecture type.

A cache client adaptation module is configured in the application system.

The processing device based on the multi-architecture distributed cache in this embodiment can execute the processing method based on the multi-architecture distributed cache provided in Embodiment 2 of the present invention, and its implementation principles are similar, so details are not repeated here.

In this embodiment, when the application system is started, the cache configuration information in the configuration center is obtained and saved, and the cache configuration information includes the current application system identifier; and the modified cache configuration information in the configuration center can be obtained and saved; then according to the cache configuration Information, adapt the client instance corresponding to the current application system; obtain cached data according to the cache configuration information and the client instance corresponding to the current application system; add the current application system identifier to the key value of the obtained cached data , so that all application systems share the same cache cluster. Therefore, the application system only needs to configure the cache product type, version, interface implementation type and other information to adapt to the appropriate cache client, and call the cache service through the client without caring about the background cache deployment method, thereby reducing the operation and maintenance of the cache Cost; at the same time, through simple client instance interface encapsulation, the application system can easily and conveniently change the cache product. It can manage heterogeneous cache systems in a unified way, provide a unified configuration center, and provide unified management for single-point, active-standby, distributed architectures, as well as Redis and Memcached; operation and maintenance personnel can manage cache device resources and applications through it The cache configuration of the system, the corresponding relationship between the application system and the device; you can also increase or decrease the cache nodes for the distributed system cache architecture in the configuration center, merge or split the single-point cache system; after modifying the cache configuration information in the configuration center, you can The cache client adaptation module of all application systems pushes the updated cache configuration information; at the same time, application systems using the same cache product and version can share the same set of cache resources partially or completely, so as to realize the maximum utilization of hardware resources, and at the same time It is also possible to manually adjust the cache load of each application system; through the cache client adaptation module, the coupling between the cache system and the application system can be reduced.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A processing method based on multi-architecture distributed cache, characterized in that, comprising: When the application system starts, obtain and save the cache configuration information in the configuration center, the cache configuration information includes the current application system identifier; Adapting a client instance corresponding to the current application system according to the cache configuration information; Obtain cached data according to the cache configuration information and the client instance corresponding to the current application system; The current application system identifier is added to the key value of the acquired cache data, so that each application system shares the same cache cluster; the current application system identifier is used to make each application system share the same set of physical cache. 2. The method according to claim 1, wherein the obtaining cached data according to the cache configuration information and the client instance corresponding to the current application system comprises: Determine a cache entry address corresponding to the current application system identifier according to the cache configuration information; The cached data in the cache cluster corresponding to the cache entry address is obtained by using the client instance corresponding to the current application system. 3. The method according to claim 1, further comprising: before adapting the client instance corresponding to the current application system according to the cache configuration information: Obtaining and saving the modified cache configuration information in the configuration center, wherein the modified cache configuration information is obtained after the configuration center receives a modification instruction sent by the user and modifies the cache configuration information according to the modification instruction. 4. The method according to any one of claims 1-3, wherein the cache configuration information further includes at least one of the following: Correspondence between application system identification, cache information, and cache entry address, and the number of cache nodes; Wherein, the cache information includes cache type, cache version number, client language and architecture type. 5. The method according to any one of claims 1-3, wherein a cache client adaptation module is configured in the application system. 6. A processing device based on a multi-architecture distributed cache, characterized in that it comprises: The first obtaining module is used to obtain and save the cache configuration information in the configuration center when the application system is started, and the cache configuration information includes the current application system identifier; A determining module, configured to adapt a client instance corresponding to the current application system according to the cache configuration information; A cache acquisition module, configured to acquire cache data according to the cache configuration information and the client instance corresponding to the current application system; The adding module is used to add the current application system identifier to the key value of the acquired cache data, so that each application system shares the same cache cluster; the current application system identifier is used to make each application system share the same set of physical cache. 7. The device according to claim 6, wherein the cache acquisition module comprises: The address determination submodule is configured to determine the cache entry address corresponding to the current application system identifier according to the cache configuration information; The cache obtaining sub-module is configured to use the client instance corresponding to the current application system to obtain cache data in the cache cluster corresponding to the cache entry address. 8. The device according to claim 6, further comprising: The second obtaining module is used to obtain and save the modified cache configuration information in the configuration center before the determining module adapts the client instance corresponding to the current application system according to the cache configuration information, wherein the modification The final cache configuration information is obtained after the configuration center receives the modification instruction sent by the user and modifies the cache configuration information according to the modification instruction. 9. The device according to any one of claims 6-8, wherein the cache configuration information further includes at least one of the following: Correspondence between application system identification, cache information, and cache entry address, and the number of cache nodes; Wherein, the cache information includes cache type, cache version number, client language and architecture type. 10. The device according to any one of claims 6-8, wherein a cache client adaptation module is configured in the application system.