CN109101635B

CN109101635B - Data processing method and device based on Redis Hash structure

Info

Publication number: CN109101635B
Application number: CN201810934756.1A
Authority: CN
Inventors: 刘宇; 周慧能
Original assignee: Guangzhou Xiaopeng Motors Technology Co Ltd
Current assignee: Guangzhou Xiaopeng Motors Technology Co Ltd
Priority date: 2018-08-16
Filing date: 2018-08-16
Publication date: 2020-09-11
Anticipated expiration: 2038-08-16
Also published as: CN109101635A

Abstract

The invention discloses a data processing method and a device based on a Redis Hash structure, wherein the method comprises the following steps: when a write-in instruction triggered by a user is detected, acquiring a key to be written and a plurality of fields to be written corresponding to the key to be written; calculating first splicing information corresponding to each field to be written, wherein the first splicing information corresponding to any two fields to be written is different from each other; splicing the first splicing information corresponding to each field to be written with the key to be written to obtain a first target key corresponding to the field to be written, wherein the first target keys corresponding to any two fields to be written are different from each other; and for each field to be written, correspondingly writing the field to be written and the first target key corresponding to the field to be written into the cache server corresponding to the first target key corresponding to the field to be written, wherein the cache servers corresponding to the first target keys corresponding to any two fields to be written are different from each other. By implementing the embodiment of the invention, the problem of uneven storage distribution of the service data can be solved.

Description

Data processing method and device based on Redis Hash structure

Technical Field

The invention relates to the technical field of internet, in particular to a data processing method and device based on a Redis Hash structure.

Background

At present, a large amount of service data (such as user state data) is stored in a non-relational database by using a Redis Hash structure, where the Redis Hash structure includes a plurality of keys (primary keys), each key corresponds to a plurality of fields (secondary keys), and each field stores service data corresponding to the field.

In practice, it is found that an expiration clearing mechanism of a non-relational database cannot clear expired business data in business data stored by using a Redis Hash structure, so that the expired business data occupies too much memory in the non-relational database. Therefore, the expiration clearing mode for the service data stored in the Redis Hash structure mainly comprises the following steps: and clearing the service data stored in the field corresponding to the key by managing the key in the RedisHash structure.

Although the above-mentioned cleaning method can utilize the characteristic that data converges on a key and implement quick cleaning of the service data stored in the field corresponding to the key by managing the key, in order to facilitate quick cleaning of the service data stored in the field corresponding to the key, the service data stored in a large number of fields of the same key can only be stored in the same cache server in a centralized manner, so that there is a case that the memory of one cache server is insufficient to store the service data which needs to be stored and the memory of another cache server is more than sufficient. Therefore, although the cleaning method can clean the service data efficiently, the problem of uneven storage distribution of the service data is caused.

Disclosure of Invention

The embodiment of the invention discloses a data processing method and device based on a Redis Hash structure, which can solve the problem of uneven storage distribution of service data.

The first aspect of the embodiments of the present invention discloses a data processing method based on a Redis Hash structure, including:

when a write instruction triggered by a user is detected, acquiring a key to be written and a plurality of fields to be written corresponding to the key to be written;

calculating first splicing information corresponding to each field to be written, wherein the first splicing information corresponding to any two fields to be written is different from each other;

splicing the first splicing information corresponding to each field to be written with the key to be written to obtain a first target key corresponding to the field to be written, wherein the first target keys corresponding to any two fields to be written are different from each other;

and correspondingly writing the fields to be written and the first target keys corresponding to the fields to be written into cache servers corresponding to the first target keys corresponding to the fields to be written, wherein the cache servers corresponding to the first target keys corresponding to any two fields to be written are different from each other.

As an optional implementation manner, in the first aspect of this embodiment of the present invention, the method further includes:

when a query instruction triggered by a user is detected, acquiring a key to be queried indicated by the query instruction and a field to be queried corresponding to the key to be queried;

calculating second splicing information corresponding to the field to be queried, and splicing the key to be queried and the second splicing information to obtain a second target key;

and inquiring the data information included in the field to be inquired in the cache server corresponding to the second target key.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, the obtaining a key to be queried of the query indicated by the query instruction and a field to be queried corresponding to the key to be queried includes:

and determining a field to be queried according to the first user ID information included in the query instruction, and determining a key to be queried corresponding to the field to be queried.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, before the obtaining the key to be written and the fields to be written corresponding to the key to be written, the method further includes:

when a login request triggered by a user is detected, judging whether user information included in the login request is matched with preset user information; the login request is used for requesting to login a database;

when the user information included in the login request is judged to be matched with the preset user information, judging whether a write-in instruction triggered by a user is detected;

when the write-in instruction triggered by the user is judged to be detected, the key to be written and a plurality of fields to be written corresponding to the key to be written are obtained;

when the write-in instruction triggered by the user is judged not to be detected, judging whether an inquiry instruction triggered by the user is detected or not;

and when the query instruction triggered by the user is detected, executing the key to be queried and the field to be queried corresponding to the key to be queried, which are indicated by the query instruction, to be obtained.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, the obtaining a key to be written and a plurality of fields to be written corresponding to the key to be written includes:

determining a plurality of fields to be written according to second user ID information included in the writing instruction, and determining keys to be written corresponding to the fields to be written.

The second aspect of the embodiments of the present invention discloses a data processing apparatus based on a Redis Hash structure, including:

the device comprises an acquisition unit, a storage unit and a control unit, wherein the acquisition unit is used for acquiring a key to be written and a plurality of fields to be written corresponding to the key to be written when a write instruction triggered by a user is detected;

the calculation unit is used for calculating first splicing information corresponding to each field to be written, wherein any two pieces of first splicing information corresponding to the fields to be written are different;

the splicing unit is used for splicing the first splicing information corresponding to each field to be written with the key to be written to obtain a first target key corresponding to the field to be written, wherein any two first target keys corresponding to the field to be written are different from each other;

and the writing unit is used for correspondingly writing the fields to be written and the first target keys corresponding to the fields to be written into cache servers corresponding to the first target keys corresponding to the fields to be written, wherein the cache servers corresponding to the first target keys corresponding to any two fields to be written are different from each other.

As an alternative implementation, in the second aspect of the embodiment of the present invention,

the acquiring unit is further configured to acquire a key to be queried indicated by the query instruction and a field to be queried corresponding to the key to be queried when the query instruction triggered by the user is detected;

the computing unit is further configured to compute second splicing information corresponding to the field to be queried, and splice the key to be queried and the second splicing information to obtain a second target key;

wherein, the data processing device based on Redis Hash structure further comprises:

and the query unit is used for querying the data information included in the field to be queried in the cache server corresponding to the second target key.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, a manner that the obtaining unit is configured to obtain a key to be queried indicated by the query instruction and a field to be queried corresponding to the key to be queried specifically is:

the obtaining unit is used for determining a field to be queried according to the first user ID information included in the query instruction and determining a key to be queried corresponding to the field to be queried.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the data processing apparatus based on a Redis Hash structure further includes:

the first judging unit is used for judging whether user information included in a login request is matched with preset user information or not when the login request triggered by a user is detected before the acquiring unit acquires the key to be written and a plurality of fields to be written corresponding to the key to be written; the login request is used for requesting to login a database;

a second judging unit, configured to judge whether a write instruction triggered by a user is detected when the first judging unit judges that the user information included in the login request matches the preset user information; when the writing instruction triggered by the user is judged to be detected, triggering the acquisition unit to execute the acquisition to-be-written key and a plurality of to-be-written fields corresponding to the to-be-written key;

a third judging unit, configured to judge whether a query instruction triggered by a user is detected when the second judging unit judges that the write instruction triggered by the user is not detected; and when the query instruction triggered by the user is detected, triggering the acquisition unit to execute the key to be queried for acquiring the query indicated by the query instruction and the field to be queried corresponding to the key to be queried.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, a manner that the obtaining unit is configured to obtain the key to be written and the plurality of fields to be written corresponding to the key to be written specifically is as follows:

the obtaining unit is used for determining a plurality of fields to be written according to the second user ID information included in the writing instruction, and determining keys to be written corresponding to the fields to be written.

A third aspect of the embodiments of the present invention discloses a computer-readable storage medium storing a computer program, where the computer program enables a computer to execute a data processing method based on a Redis Hash structure disclosed in the first aspect of the embodiments of the present invention.

A fourth aspect of the present embodiments discloses a computer program product, which, when run on a computer, causes the computer to perform some or all of the steps of any one of the methods of the first aspect.

A fifth aspect of an embodiment of the present invention discloses an application publishing platform, where the application publishing platform is configured to publish a computer program product, where when the computer program product runs on a computer, the computer is caused to perform part or all of the steps of any one of the methods in the first aspect.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, when data is written, first splicing information corresponding to a field to be written can be calculated, a key to be written is spliced with the first splicing information to obtain a first target key, and the first target key and the field to be written are correspondingly written into the cache server corresponding to the first target key. The process can store the service data included in the fields to be written into a cache server corresponding to a first target key, if the fields to be written into are multiple, a first target key corresponding to each field to be written into is determined, each field to be written into is stored to a cache server corresponding to the first target key corresponding to the field to be written into, the first splicing information corresponding to each field to be written into is different from the first target key corresponding to each first splicing information, so that the first target key corresponding to each field to be written into is different from the first target key corresponding to each field to be written into, and different first target keys correspond to different cache servers, so that different fields to be written into which the same field to be written into is stored to different cache servers, and when the service data stored in the fields to be written into needs to be cleaned, the first target key is obtained only according to the fields to be written into and the fields to be written into, so that the field corresponding to the first target key can be deleted from the cache server corresponding to the first target key, therefore, the method can not only efficiently clean the service data, but also solve the problem of uneven storage distribution of the service data.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flow chart of a data processing method based on a Redis Hash structure according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of another data processing method based on Redis Hash structure disclosed in the embodiment of the present invention;

FIG. 3 is a schematic flowchart of a conventional data processing method based on a Redis Hash structure according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of another data processing method based on Redis Hash structure disclosed in the embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a data processing apparatus based on Redis Hash structure according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of another Redis Hash structure-based data processing apparatus disclosed in the embodiment of the present invention;

FIG. 7 is a schematic storage diagram of a conventional data processing apparatus based on Redis Hash structure according to an embodiment of the present invention;

fig. 8 is a schematic storage diagram of another data processing apparatus based on the Redis Hash structure disclosed in the embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a data processing method and device based on a Redis Hash structure, which can solve the problem of uneven storage distribution of service data. The following are detailed below.

Example one

Referring to fig. 1, fig. 1 is a schematic flow chart of a data processing method based on a Redis Hash structure according to an embodiment of the present invention. As shown in fig. 1, the data processing method based on the Redis Hash structure may include the following steps:

101. when a write instruction triggered by a user is detected, the data processing device based on the Redis Hash structure acquires a key to be written and a plurality of fields to be written corresponding to the key to be written.

In the embodiment of the invention, the write-in instruction is used for indicating to write data to be written into the cache server. And storing the data to be written into the field to be written corresponding to each data to be written, and storing each field to be written into the key to be written corresponding to the field to be written.

102. The data processing device based on the Redis Hash structure calculates first splicing information corresponding to each field to be written, wherein the first splicing information corresponding to any two fields to be written are different from each other.

In the embodiment of the present invention, a consistent hash algorithm may be used to calculate the first splicing information corresponding to each field to be written, or other algorithms may be used to calculate the first splicing information corresponding to each field to be written, which is not limited in the embodiment of the present invention.

As an alternative implementation, when the consistent Hash algorithm is used to calculate the first splicing information corresponding to each field to be written, the calculating, by the data processing apparatus based on the Redis Hash structure, the first splicing information corresponding to each field to be written may include:

the data processing device based on the Redis Hash structure calculates a slot value corresponding to each field to be written by using a consistent Hash algorithm, and determines the slot value as first splicing information corresponding to the field to be written, wherein the slot value is a slot value corresponding to one Hash slot in a plurality of Hash slots in the Redis Hash structure, 16384 Hash slots are arranged in the Redis Hash structure, and the slot values corresponding to the 16384 Hash slots are 0-16383.

By implementing the optional implementation manner, a consistent hash algorithm can be used to calculate a slot value corresponding to each field to be written, and the slot value corresponding to each field to be written is determined as the first splicing information corresponding to the field to be written, so that the cache server corresponding to each field to be written can be determined according to the first splicing information and the key to be written. The process can obtain different slot values corresponding to the fields to be written according to different fields to be written, and different fields to be written under the same key to be written can be written into different cache servers due to different slot values.

103. And the data processing device based on the Redis Hash structure splices the first splicing information corresponding to each field to be written with the key to be written to obtain a first target key corresponding to the field to be written, wherein the first target keys corresponding to any two fields to be written are different from each other.

In the embodiment of the present invention, for example, when there are two to-be-written fields (field1 and field2) corresponding to the to-be-written key (key1) and the first splicing information corresponding to field1 is field1-hash and the first splicing information corresponding to field2 is field2-hash, the first target key obtained by splicing the field1-hash with the to-be-written key may be "key 1: field 1-hash" and the first target key obtained by splicing the field2-hash with the to-be-written key may be "key 1: field 2-hash". Since the field1-hash is different from the field2-hash, the first target key for field1 ("key 1: field 1-hash") is different from the first target key for field2 ("key 1: field 2-hash").

104. And for each field to be written, correspondingly writing the field to be written and a first target key corresponding to the field to be written into a cache server corresponding to the first target key corresponding to the field to be written by the data processing device based on the Redis Hash structure, wherein the cache servers corresponding to the first target keys corresponding to any two fields to be written are different from each other.

It can be seen that by implementing the data processing method based on the Redis Hash structure described in fig. 1, service data included in a field to be written may be stored in a cache server corresponding to a first target key, if there are multiple fields to be written, a first target key corresponding to each field to be written is determined, and each field to be written is stored in a cache server corresponding to the first target key corresponding to the field to be written. Thereby solving the problem of uneven storage distribution of the service data included in the field. In addition, different fields to be written corresponding to the same key to be written are respectively corresponding to different first target keys in the mode, the number of cluster nodes is expanded, and therefore the overall reading and writing performance of the cluster is improved.

Example two

Referring to fig. 2, fig. 2 is a schematic flow chart of another data processing method based on a Redis Hash structure according to an embodiment of the present invention. As shown in fig. 2, the data processing method based on the Redis Hash structure may include the following steps:

201. when a login request triggered by a user is detected, the data processing device based on the Redis Hash structure judges whether user information included in the login request is matched with preset user information, if so, step 202 is executed, and if not, the process is ended.

In the embodiment of the invention, the login request is used for requesting to login the database. When the user information included in the login request is matched with the preset user information, it indicates that the user information passes the verification, and at this time, step 202 may be executed; and when the user information included in the login request is not matched with the preset user information, indicating that the user information is not verified, and ending the process.

202. The data processing device based on the Redis Hash structure judges whether a writing instruction triggered by a user is detected, if so, the steps 203 to 206 are executed, and if not, the step 207 is executed.

In the embodiment of the present invention, the write command is used to instruct to write data into the database, when the write command triggered by the user is detected, step 203 to step 206 are executed, and when the write command triggered by the user is not detected, step 207 is executed.

203. The data processing device based on the Redis Hash structure obtains a key to be written and a plurality of fields to be written corresponding to the key to be written.

As an optional implementation manner, the obtaining, by the data processing apparatus based on the Redis Hash structure, a key to be written and a number of fields to be written corresponding to the key to be written may include:

and the data processing device based on the Redis Hash structure determines a plurality of fields to be written according to the second user ID information included in the writing instruction, and determines keys to be written corresponding to the fields to be written.

In this embodiment of the present invention, the second user ID information may be the same as the first user ID information, or may be different from the first user ID information.

By implementing the optional implementation mode, the field to be written can be determined according to the second user ID information, so that the user state data corresponding to the second user ID information is dispersedly stored in different cache servers, and the problem of uneven storage distribution of the service data is solved.

204. The data processing device based on the Redis Hash structure calculates first splicing information corresponding to each field to be written, wherein the first splicing information corresponding to any two fields to be written are different from each other.

205. And the data processing device based on the Redis Hash structure splices the first splicing information corresponding to each field to be written with the key to be written to obtain a first target key corresponding to the field to be written, wherein the first target keys corresponding to any two fields to be written are different from each other.

206. And for each field to be written, correspondingly writing the field to be written and a first target key corresponding to the field to be written into a cache server corresponding to the first target key corresponding to the field to be written by the data processing device based on the Redis Hash structure, wherein the cache servers corresponding to the first target keys corresponding to any two fields to be written are different from each other.

207. And the data processing device based on the Redis Hash structure judges whether an inquiry instruction triggered by a user is detected, if so, the step 208 to the step 210 are executed, and if not, the process is ended.

As an alternative implementation, when the data processing apparatus based on the Redis Hash structure determines that the query instruction triggered by the user is not detected, the following steps may be further performed:

the data processing device based on the Redis Hash structure detects whether a cleaning instruction triggered by a user is received, wherein the cleaning instruction at least comprises a key to be cleaned, and the cleaning instruction is used for indicating that data information included in a plurality of fields to be cleaned corresponding to the key to be cleaned is cleaned;

when a cleaning instruction triggered by a user is detected to be received, calculating a first target key corresponding to each field to be cleaned according to the keys to be cleaned and each field to be cleaned corresponding to the keys to be cleaned by the data processing device based on the Redis Hash structure;

and the data processing device based on the Redis Hash structure cleans each field to be cleaned in a cache server stored by a first target key corresponding to each field to be cleaned.

By implementing the optional implementation mode, the fields to be cleaned corresponding to the keys to be cleaned can be cleaned according to the cleaning instruction triggered by the user, and even if the fields to be cleaned are dispersedly stored in different cache servers, the fields to be cleaned can be cleaned by managing the keys to be cleaned, so that the aim of efficiently cleaning is fulfilled.

208. And the data processing device based on the Redis Hash structure acquires the key to be queried indicated by the query instruction and the field to be queried corresponding to the key to be queried.

209. And calculating second splicing information corresponding to the field to be queried by the data processing device based on the Redis Hash structure, and splicing the key to be queried and the second splicing information to obtain a second target key.

210. And the data processing device based on the Redis Hash structure inquires data information included in the field to be inquired in the cache server corresponding to the second target key.

As an optional implementation manner, querying, by the data processing apparatus based on the Redis Hash structure, data information included in the field to be queried in the cache server corresponding to the second target key may include:

and the data processing device based on the Redis Hash structure determines the field to be queried according to the first user ID information included in the query instruction, and determines the key to be queried corresponding to the field to be queried.

By implementing the optional implementation manner, the field to be queried can be determined according to the first user ID information, so that the query of the user state data corresponding to the first user ID information is realized.

Referring to fig. 3 and fig. 4 together, fig. 3 is a schematic flow chart of a conventional data processing method based on a Redis Hash structure according to an embodiment of the present invention, and fig. 4 is a schematic flow chart of another data processing method based on a Redis Hash structure according to an embodiment of the present invention. As shown in fig. 3, the data processing method based on the Redis Hash structure in the prior art processes data in the following manner:

301. and judging whether the user information included in the login request is matched with the preset user information, if so, executing the step 302, and if not, ending the process.

302. And judging whether a writing instruction triggered by a user is detected, if so, executing step 303, and if not, executing step 304.

303. And acquiring the key to be written and a plurality of fields to be written corresponding to the key to be written.

In the embodiment of the present invention, after the step 303 is executed, for each field to be written, the field to be written and the key to be written corresponding to the field to be written are correspondingly written into the cache server corresponding to the key to be written.

304. And judging whether an inquiry instruction triggered by a user is detected, if so, executing the step 305, and if not, ending the process.

305. And acquiring the key to be queried indicated by the query instruction and the field to be queried corresponding to the key to be queried.

In the embodiment of the present invention, after the step 305 is executed, the data information included in the field to be queried is queried in the cache server corresponding to the key to be queried, and the data information included in the field to be queried is returned to the browser for the user to view.

In the embodiment of the present invention, when the key to be written is key1 and the field to be written corresponding to the key to be written includes field1 and field2, the method shown in fig. 3 writes the field1 and field2 into the cache server corresponding to the key1 when writing data, and queries the cache server corresponding to the key1 for the data information included in the field1 and the data information included in the field 2. Therefore, the existing data processing method based on the Redis Hash structure can only store all fields under the same key to the same cache server, so that the problem of uneven distribution of cached data is caused.

As shown in fig. 4, fig. 4 is a schematic flow chart of another data processing method based on a Redis Hash structure, which is obtained based on the data processing method based on the Redis Hash structure shown in fig. 1 or fig. 2, and the method includes:

401. and judging whether the user information included in the login request is matched with the preset user information, if so, executing the step 402, and if not, ending the process.

402. And judging whether a writing instruction triggered by a user is detected, if so, executing the step 403 to the step 404, and if not, ending the process.

403. And acquiring the key to be written and a plurality of fields to be written corresponding to the key to be written.

404. And calculating first splicing information corresponding to each field to be written, and splicing the first splicing information corresponding to each field to be written and the key to be written to obtain a first target key corresponding to the field to be written.

In this embodiment of the present invention, after the step 404 is executed, the field to be written and the first target key corresponding to the field to be written are written into the cache server corresponding to the first target key corresponding to the field to be written.

404. And judging whether a query instruction triggered by a user is detected, if so, executing the step 406 to the step 407, and if not, ending the process.

406. And acquiring the key to be queried indicated by the query instruction and the field to be queried corresponding to the key to be queried.

407. And calculating second splicing information corresponding to the field to be queried, and splicing the key to be queried and the second splicing information to obtain a second target key.

After step 407 is executed, the cache server corresponding to the second target key queries the data information included in the field to be queried, and returns the data information included in the field to be queried.

With the method shown in fig. 4, the keys to be written (key1) and the fields to be written (field1 and field2) corresponding to the keys to be written are written in the following manner: calculating first stitching information (field1-hash) corresponding to field1 and first stitching information (field2-hash) corresponding to field 2; splicing the first splicing information corresponding to the field1 with a key1 to obtain a first target key (key1: field1-hash) corresponding to the field1, and splicing the first splicing information corresponding to the field2 with the key to obtain a first target key (key1: field2-hash) corresponding to the field 2; field1 is stored to a cache server corresponding to "key 1: field 1-hash" and field2 is stored to a cache server corresponding to "key 1: field 2-hash". The writing process can write different fields under the same key into different cache servers, so that the problem of uneven distribution of cache data is solved. By adopting the method shown in fig. 4, the way of querying a number of fields (field1 and field2) corresponding to the key to be queried (key1) is as follows: calculating a field1-hash using field1, calculating a field2-hash using field 2; "key 1: field 1-hash" from field1-hash and key1, "key 1: field 2-hash" from field2-hash and key 1; the data information included in field1 is queried in "key 1: field 1-hash", and the data information included in field2 is queried in "key 1: field 2-hash". This query process may still query the field included data information based on key and field, but the field under the same key is stored to a different cache server. Therefore, on the premise of convenient management and high-efficiency cleaning of the fields under the key, the method can dispersedly store all the fields under the same key to different cache servers.

It can be seen that the problem of uneven storage distribution of the service data included in the field is solved by implementing the data processing method based on the Redis Hash structure described in FIG. 2. In addition, different fields to be written corresponding to the same key to be written are respectively corresponding to different first target keys in the mode, the number of cluster nodes is expanded, and therefore the overall reading and writing performance of the cluster is improved.

EXAMPLE III

Referring to fig. 5, fig. 5 is a schematic structural diagram of a data processing apparatus based on a Redis Hash structure according to an embodiment of the present invention. As shown in fig. 5, the data processing apparatus 500 based on the Redis Hash structure may include an obtaining unit 501, a calculating unit 502, a splicing unit 503, and a writing unit 504, where:

the obtaining unit 501 is configured to obtain a key to be written and a plurality of fields to be written corresponding to the key to be written when a write instruction triggered by a user is detected.

The calculating unit 502 is configured to calculate first splicing information corresponding to each to-be-written field acquired by the acquiring unit 501, where any two pieces of first splicing information corresponding to the to-be-written fields are different.

As an optional implementation manner, when the consistent hash algorithm is used to calculate the first splicing information corresponding to each field to be written, the calculating unit 502 may calculate the first splicing information corresponding to each field to be written, including:

the calculating unit 502 calculates a slot value corresponding to each field to be written by using a consistent Hash algorithm, and determines the slot value as first splicing information corresponding to the field to be written, where the slot value is a slot value corresponding to a Hash slot in a plurality of Hash slots in a Redis Hash structure, where 16384 Hash slots are in the Redis Hash structure, and the slot values corresponding to the 16384 Hash slots are 0 to 16383.

The splicing unit 503 is configured to splice the first splicing information corresponding to each field to be written, which is obtained by the calculation unit 502, with the key to be written, which is obtained by the obtaining unit 501, to obtain a first target key corresponding to the field to be written, where any two first target keys corresponding to the field to be written are different from each other.

The writing unit 504 is configured to, for each field to be written obtained by the obtaining unit 501, correspondingly write the field to be written and the first target key corresponding to the field to be written obtained by the splicing unit 503 into the cache server corresponding to the first target key corresponding to the field to be written, where cache servers corresponding to the first target keys corresponding to any two fields to be written are different from each other.

It can be seen that the problem of uneven storage distribution of the service data included in the field is solved by implementing the data processing apparatus based on the Redis Hash structure described in fig. 5. In addition, different fields to be written corresponding to the same key to be written are respectively corresponding to different first target keys in the mode, the number of cluster nodes is expanded, and therefore the overall reading and writing performance of the cluster is improved.

Example four

Referring to fig. 6, fig. 6 is a diagram of another data processing apparatus based on a Redis Hash structure according to an embodiment of the present invention. Wherein the data processing apparatus 500 based on the Redis Hash structure shown in FIG. 6 is optimized by the data processing apparatus 500 based on the Redis Hash structure shown in FIG. 5, compared with the data processing apparatus 500 based on the Redis Hash structure shown in FIG. 5, in the data processing apparatus 500 based on the Redis Hash structure shown in FIG. 6,

the obtaining unit 501 is further configured to, when a query instruction triggered by a user is detected, obtain a key to be queried indicated by the query instruction and a field to be queried corresponding to the key to be queried.

The calculating unit 502 is further configured to calculate second splicing information corresponding to the field to be queried, which is obtained by the obtaining unit 501, and splice the key to be queried and the second splicing information to obtain a second target key.

The data processing apparatus based on the Redis Hash structure shown in fig. 6 may further include:

the querying unit 505 is configured to query the cache server corresponding to the second target key calculated by the calculating unit 502 for the data information included in the field to be queried.

Optionally, the manner that the obtaining unit 501 is configured to obtain the key to be queried and the field to be queried corresponding to the key to be queried, where the key to be queried is indicated by the query instruction, is specifically as follows:

the obtaining unit 501 is configured to determine a field to be queried according to the first user ID information included in the query instruction, and determine a key to be queried corresponding to the field to be queried.

Further optionally, the data processing apparatus based on the Redis Hash structure shown in fig. 6 may further include:

a first determining unit 506, configured to determine whether user information included in a login request matches preset user information when the login request triggered by a user is detected before the obtaining unit 501 obtains the key to be written and the plurality of fields to be written corresponding to the key to be written; the login request is used for requesting to login the database.

A second judging unit 507, configured to judge whether a write instruction triggered by a user is detected when the first judging unit 506 judges that the user information included in the login request matches preset user information; when it is determined that the write instruction triggered by the user is detected, the trigger obtaining unit 501 performs the above-mentioned obtaining of the key to be written and the plurality of fields to be written corresponding to the key to be written.

A third determining unit 508, configured to determine whether a query instruction triggered by a user is detected when the second determining unit 507 determines that a write instruction triggered by the user is not detected; when the query instruction triggered by the user is detected, the trigger obtaining unit 501 executes the key to be queried indicated by the obtained query instruction and the field to be queried corresponding to the key to be queried.

As an optional implementation manner, when the third determining unit 508 determines that the query instruction triggered by the user is not detected, the third determining unit 508 may be further configured to:

detecting whether a cleaning instruction triggered by a user is received, wherein the cleaning instruction at least comprises a key to be cleaned, and the cleaning instruction is used for indicating data information included in a plurality of fields to be cleaned corresponding to the key to be cleaned;

when a cleaning instruction triggered by a user is detected to be received, calculating a first target key corresponding to each field to be cleaned according to the key to be cleaned and each field to be cleaned corresponding to the key to be cleaned;

and cleaning each field to be cleaned in a cache server stored by a first target key corresponding to each field to be cleaned.

Further optionally, the manner of the obtaining unit 501 for obtaining the key to be written and the plurality of fields to be written corresponding to the key to be written specifically is as follows:

an obtaining unit 501, configured to determine, according to the second user ID information included in the write instruction, a number of fields to be written, and determine keys to be written corresponding to the number of fields to be written.

Referring to fig. 7 and 8 together, fig. 7 is a schematic storage diagram of a conventional data processing apparatus based on a Redis Hash structure according to an embodiment of the present invention, and fig. 8 is a schematic storage diagram of another data processing apparatus based on a Redis Hash structure according to an embodiment of the present invention. As shown in fig. 7, a user may operate a computer 701 to trigger a write command or a query command, where the write command or the query command triggered by the user reaches a user server 704 through a cloud 702 and a firewall 703, and the user server 704 performs a corresponding operation in a cache server 705 according to the write command or the query command. When a data processing apparatus with a conventional Redis Hash structure stores data information (value) included in fields, a plurality of fields under the same key are stored in the same cache server (the cache server 705 shown in fig. 7), and this storage manner causes a problem of uneven storage distribution, and there may be a phenomenon that the number of fields under a certain key is too large and the number of fields under another key is too small. The key with an excessive field number has the problem that the memory usage reaches the upper limit, and the key with a small field number has the problem of low memory usage rate. Accordingly, as shown in fig. 8, a user may operate the computer 801 to trigger a write command or a query command, the user-triggered write command or query command reaches the user server 804 through the cloud 802 and the firewall 803, and the user server 804 performs a corresponding operation in the cache server 805, the cache server 806, or the cache server 807 according to the write command or the query command. The data processing device based on the Redis Hash structure disclosed by the embodiment of the invention obtains a first target key corresponding to each field by calculating first splicing information (such as a Hash value corresponding to each field) of each field, splicing the key corresponding to each field with the first splicing information corresponding to each field, and storing each field to a cache server corresponding to the first target key corresponding to the field. As shown in FIG. 8, the first target key corresponding to field1 corresponding to key1 is "key 1: field 1-hash", the first target key corresponding to field2 corresponding to key1 is "key 1: field 2-hash", the first target key corresponding to field3 corresponding to key1 is "key 1: field 3-hash", and the cache server corresponding to "key 1: field 1-hash" is cache server 805 shown in FIG. 8, the cache server corresponding to "key 1: field 2-hash" is cache server 806 shown in FIG. 8, and the cache server corresponding to "key 1: field 3-hash" is cache server 807 shown in FIG. 8. Therefore, different fields corresponding to the same key are stored in different cache servers, and the problem that service data included in the fields are unevenly stored and distributed is solved.

It can be seen that the problem of uneven storage distribution of the service data included in the field is solved by implementing the data processing apparatus based on the Redis Hash structure described in FIG. 6. In addition, different fields to be written corresponding to the same key to be written are respectively corresponding to different first target keys in the mode, the number of cluster nodes is expanded, and therefore the overall reading and writing performance of the cluster is improved.

The embodiment of the invention discloses a computer-readable storage medium which stores a computer program, wherein the computer program enables a computer to execute any one data processing method based on a Redis Hash structure in figure 1 or figure 2.

Embodiments of the present invention also disclose a computer program product, wherein, when the computer program product is run on a computer, the computer is caused to execute part or all of the steps of the method as in the above method embodiments.

The embodiment of the present invention also discloses an application publishing platform, wherein the application publishing platform is used for publishing a computer program product, and when the computer program product runs on a computer, the computer is caused to execute part or all of the steps of the method in the above method embodiments.

In various embodiments of the present invention, it should be understood that the sequence numbers of the above-mentioned processes do not imply an inevitable order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

In the embodiments provided herein, it should be understood that "B corresponding to A" means that B is associated with A from which B can be determined. It should also be understood, however, that determining B from a does not mean determining B from a alone, but may also be determined from a and/or other information.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present invention, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of each embodiment of the present invention.

It will be understood by those skilled in the art that all or part of the steps in the methods of the embodiments described above may be implemented by instructions associated with a program, which may be stored in a computer-readable storage medium, where the storage medium includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), compact disc-Read-Only Memory (CD-ROM), or other Memory, magnetic disk, magnetic tape, or magnetic tape, Or any other medium which can be used to carry or store data and which can be read by a computer.

The data processing method and device based on the Redis Hash structure disclosed by the embodiment of the invention are introduced in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A data processing method based on Redis Hash structure is characterized by comprising the following steps:

calculating first splicing information corresponding to each field to be written, wherein the first splicing information corresponding to any two fields to be written is different from each other, and the first splicing information is calculated through a Hash algorithm;

2. The method of claim 1, further comprising:

3. The method of claim 2, wherein the obtaining a key to be queried of the query indicated by the query instruction and a field to be queried corresponding to the key to be queried comprises:

4. The method of claim 3, wherein prior to obtaining the key to be written and the number of fields to be written corresponding to the key to be written, the method further comprises:

5. The method according to any one of claims 1 to 4, wherein the obtaining the key to be written and the fields to be written corresponding to the key to be written comprises:

6. A data processing apparatus based on a Redis Hash structure, comprising:

the calculation unit is used for calculating first splicing information corresponding to each field to be written, wherein any two pieces of first splicing information corresponding to the fields to be written are different, and the first splicing information is obtained through calculation of a Hash algorithm;

7. The Redis Hash structure-based data processing apparatus according to claim 6, wherein:

8. The Redis Hash structure-based data processing apparatus according to claim 7, wherein the manner for the obtaining unit to obtain the key to be queried indicated by the query instruction and the field to be queried corresponding to the key to be queried specifically is:

9. The Redis Hash structure-based data processing apparatus according to claim 8, wherein the Redis Hash structure-based data processing apparatus further comprises:

10. The Redis Hash structure-based data processing device according to any one of claims 6 to 9, wherein the manner for the obtaining unit to obtain the key to be written and the plurality of fields to be written corresponding to the key to be written is specifically as follows: