CN111506582A - A data storage method and device - Google Patents

Abstract

The present application discloses a data storage method and device, wherein the method includes: when a data providing device needs to add a data record to a preset database table for the first time, obtain the currently available record identifier of the device from the database table, and use the The data record is added to the database table, the number of the record identifier is expanded by using the preset number of identifiers, the obtained identifier number range is stored in the memory, and the available record identifier of the device in the database table is updated to The maximum value of the identification number range is increased by 1; when the device needs to add a data record to the database table after the first time, the threads that control the current need to increase the data record access the memory one by one to obtain the record identifier, and use the record identifier to save the data record to In the database table, when all the record identifiers in the identification number range are used, it is triggered to update the identification number range in the memory. By adopting the present invention, the pressure on database access can be reduced.

Description

A data storage method and device

technical field

The present invention relates to computer application technology, in particular to a data storage method and device.

Background technique

In a system with a distributed architecture, data needs an identifier that can uniquely distinguish data records when it is stored and read. This identifier is usually called a serial number or ID. Ensuring the uniqueness of the ID is a problem that needs to be solved during system design, especially when the database uses sub-database and sub-table.

At present, in a system using a distributed architecture, each time a data providing device adds a new data record to the database, it needs to obtain an available data record identifier from the database first, and then associate the data record identifier with the new data record identifier. Additional data records are added to the database in association.

In the above method for adding data records, since the ID acquisition operation must be performed first each time a new data record is added, in the scenario of high data concurrency, the number of accesses to the database is too large, resulting in a great pressure on the access to the database.

SUMMARY OF THE INVENTION

In view of this, the main purpose of the present invention is to provide a data storage method and device, which can effectively reduce the access pressure of newly added data records to the database.

In order to achieve the above object, the technical scheme proposed by the present invention is:

A data storage method comprising:

When the data providing device needs to add a data record to a preset database table for the first time, it acquires the currently available record identifier of the data providing device from the database table, and uses the record identifier to add the data record to the database In the table, use the preset number of identification extensions to perform number expansion on the record identification, save the identification number range obtained by the number expansion in the memory of the data providing device, and store the data providing device in the database table. The available record identification is updated to the maximum value of the identification number range plus 1; the database table at least includes a record identification field;

When the data providing device needs to add data records to the database table after the first time, it adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use the obtained records. The identifier stores the corresponding data records that need to be added into the database table, and when the record identifiers in the identifier number range are all used, triggering the update of the identifier number range in the memory. ;

When the data providing device needs to add data records to the database table after the first time, it adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use the obtained records. The identifier stores the corresponding data records that need to be added into the database table, and when the record identifiers in the identifier number range are all used, triggering the update of the identifier number range in the memory.

Preferably, the identification number range is [a, b], where a is the currently acquired record identification; b=a+n-1, n is the identification extension number.

Preferably, the synchronous control method is used to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use the acquired record identifiers to store the corresponding data records that need to be added to the database table. , when the record identifiers in the identification number range are all used, triggering the update of the identification number range in the memory includes:

a1, the data providing equipment adopts the mode of synchronous control, selects a thread from the thread that currently needs to increase the data record in this data providing equipment, as the working thread that can currently increase the data record, and blocks other described threads;

a2. From the memory of the data providing device, take out the smallest identifier that has not been taken out in the identifier number range, and trigger the worker thread to use the smallest identifier to add the corresponding data records that need to be added to the database table; when When the minimum identifier is equal to the maximum value of the identifier number range, the data providing device re-acquires the available record identifiers corresponding to the data providing device from the database table, and uses the identifier extension number to re-acquire The record identification carries out number expansion, replaces the identification number range in the memory with the identification number range obtained by the current number expansion, and updates the available record identification of the data providing equipment in the database table to the identification obtained by the current number expansion Add 1 to the maximum value of the number range;

a3. If there is currently a blocked thread, the data providing device adopts a synchronous control method to select a thread from the currently blocked threads and wake up, and use the thread as the current working thread, Go to step a2.

Preferably, the data providing device is a server.

A data storage device, set in a data providing device, includes:

The first unit, when the data providing device needs to add a data record to a preset database table for the first time, obtains the currently available record identifier of the data providing device from the database table, and uses the record identifier to add the data record. In the database table, use the preset number of identification extensions to perform number expansion on the record identification, save the identification number range obtained by the number expansion in the memory of the data providing device, and store the data in the database table. The record identification available to the data providing device is updated to the maximum value of the identification number range plus 1; the database table at least includes a record identification field;

The second unit, when the data providing device needs to add data records to the database table after the first time, adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use all The acquired record identifiers store corresponding data records that need to be added in the database table, and when all record identifiers in the identifier number range are used, triggering the update of the identifier number range in the memory.

Preferably, the identification number range is [a, b], where a is the currently acquired record identification; b=a+n-1, n is the identification extension number.

Preferably, the second unit adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use the acquired record identifiers to store the corresponding data records that need to be added to the memory. In the database table, when the record identifiers in the identification number range are all used, triggering the update of the identification number range in the memory includes:

a1, the data providing equipment adopts the mode of synchronous control, selects a thread from the thread that currently needs to increase the data record in this data providing equipment, as the working thread that can currently increase the data record, and blocks other described threads;

a2. From the memory of the data providing device, take out the smallest identifier that has not been taken out in the identifier number range, and trigger the worker thread to use the smallest identifier to add the corresponding data records that need to be added to the database table; when When the minimum identifier is equal to the maximum value of the identifier number range, the data providing device re-acquires the available record identifiers corresponding to the data providing device from the database table, and uses the identifier extension number to re-acquire The record identification carries out number expansion, replaces the identification number range in the memory with the identification number range obtained by the current number expansion, and updates the available record identification of the data providing equipment in the database table to the identification obtained by the current number expansion Add 1 to the maximum value of the number range;

a3. If there is currently a blocked thread, the data providing device adopts a synchronous control method to select a thread from the currently blocked threads and wake up, and use the thread as the current working thread, Go to step a2.

Preferably, the data providing device is a server.

A non-volatile computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the steps of the above-described data storage method.

An electronic device includes a non-volatile computer-readable storage medium as described above, and the processor accessible to the non-volatile computer-readable storage medium.

To sum up, the data storage scheme proposed by the present invention performs number expansion on the basis of the record identification obtained from the database, generates a segment of identification number, and saves it in the memory. In this way, the data providing device can access the database once , you can obtain multiple available data record identification resources instead of one identification, and save them in the memory. In this way, each time the data providing device needs to add data records to the database, it only needs to obtain the available data records from its own memory. Only after all the identification numbers in the identification number segment in the memory are used, the new available data record identification is requested from the database, and then the newly obtained identification is used to generate an identification number segment and save it in the memory. . Therefore, the number of accesses to the database can be reduced, so that the access pressure on the database can be reduced in a scenario of high data concurrency, and it is easy to implement.

Description of drawings

1 is a schematic flowchart of a method according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a schematic flowchart of a method according to an embodiment of the present invention. As shown in FIG. 1 , the data storage method implemented by this embodiment mainly includes:

Step 101: When the data providing device needs to add a data record to a preset database table for the first time, obtain the currently available record identifier of the data providing device from the database table, and use the record identifier to add the data record to the preset database table. In the database table, use the preset number of identification extensions to perform number expansion on the record identification, save the identification number range obtained by the number expansion in the memory of the data providing device, and store the number in the database table. The record identification available to the data providing device is updated to the maximum value of the identification number range plus one.

The database table includes at least a record identification field, but is not limited to this, and other fields can also be set according to actual application needs. For example, in a microservice application scenario, in order to facilitate the generation of a global identification, it These fields can be combined to generate a global identifier to facilitate the global retrieval of the database. In addition, in order to improve retrieval efficiency, the above fields can be represented by coding.

In this step, when the data providing device needs to add data records to the preset database table for the first time, after obtaining the currently available record identifier of the data providing device, it not only uses the record identifier to save the data record to be added in the database table, It is also necessary to carry out number expansion based on the record identification to obtain an identification number range (that is, a section of identification number), and the identification number contained in the identification number range will be stored in the memory as the record identification pre-allocated to the data providing device. Update the available record identification of the data providing device in the database table to the next identification number in the identification number range, that is, add one to the maximum value of the identification number range, so that the identification number in the identification number range will not be assigned to Other data provides device usage. In this way, equivalent to one database access, a segment of data record can be obtained to identify resource usage, thereby greatly reducing the number of accesses to the database by the data providing device, thereby reducing the access pressure on the database.

Preferably, the identification number range can be expressed as [a,b].

Wherein, a is the currently acquired record identifier, that is, the minimum value of the identifier number range.

b is the maximum value of the identification number range, b=a+n-1.

n is the preset number of identification extensions, and the value can be set by those skilled in the art according to actual needs.

Specifically, the data providing device may be a server, which is not limited thereto, and may also be other devices that can store data in a database.

The specific generation method of the identification number range in this embodiment will be described in detail below in conjunction with a specific application scenario:

Application Scenario: Microservice application scenario.

The data providing device is a microservice server, and the database table to be accessed has at least two fields, a record identification field and a microservice identification (address) field.

Among them, the record identification field belongs to the primary key, auto-increment setting, bigint unsigned type.

The address field is the registered microservice name identifier, string type, and unique index in the table.

Specifically, based on the MySQL language, the table structure corresponding to the database table is as follows:

CREATE TABLE uid_microservice(

`id` bigint(20) unsigned NOT NULL auto_increment,

`address`varchar(20) NOT NULL default",

PRIMARY KEY(`id`),

UNIQUE KEY `address`(`address`)

)ENGINE=MyISAM;

When the microservice server adds data to the database for the first time, it needs to query the record ID of the corresponding microservice object through the SELECT query statement, such as: SELECT id FROM uid_microservice WHERE address="pt_ums_service".

Then, use the ID taken out to add data records to the database table, and perform number expansion on the ID taken out. For example, if the ID value taken out is 1000, if the number of ID extensions set is 1000, that is, you want to generate 1000 IDs each time ID, the corresponding identification number range is 1000-1999, save this identification number range in the memory; at the same time, use the REPLACEINTO statement to update the ID of address="pt_ums_service" in the database table to 2000.

Step 102, when the data providing device needs to add data records to the database table after the first time, the method of synchronous control is adopted to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use all the records. The acquired record identifiers store corresponding data records that need to be added in the database table, and when all record identifiers in the identifier number range are used, triggering the update of the identifier number range in the memory.

In this step, considering that when there are multiple threads in the data providing device concurrently acquiring the record ID from the memory, there will be multiple threads acquiring the same ID from the device memory, which will lead to errors in data operations, so synchronization control is adopted. In this way, the threads that currently need to add data records are controlled to access the memory one by one to obtain the corresponding available record identifiers, so as to ensure that the IDs used by each thread are unique, so that errors in data addition operations can be effectively avoided.

In practical applications, the atomic feature of AtomicLong in the JAVA language can be used to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers. The specific implementation method is mastered by those skilled in the art.

Preferably, in this step, for the method of synchronous control, the threads that control the current need to increase data records access the memory one by one to obtain the corresponding available record identifiers, and use the acquired record identifiers to store the corresponding data records that need to be added to all the records. In the described database table, when the record identification in the described identification number range is all used, trigger to the described identification number range in the memory to be updated, specifically can adopt the following method to realize:

Step 1021, the data providing device adopts a synchronous control method, selects a thread from the threads that currently need to add data records in the data providing device, as the working thread that can currently add data records, and blocks other described threads .

Step 1022, from the memory of the data providing device, take out the minimum mark that has not been taken out in the identification number range, and trigger the worker thread to use the minimum mark to add the corresponding data records that need to be added to the database table; When the minimum identifier is equal to the maximum value of the identifier number range, the data providing device re-acquires the available record identifiers corresponding to the data providing device from the database table, and uses the identifier to expand the number of new records. The obtained record identification carries out number expansion, and the identification number range in the memory is replaced by the identification number scope obtained by the current number expansion, and the available record identification of the data providing equipment in the database table is updated to the current number expansion obtained. The maximum value of the identification number range is incremented by 1.

Step 1023: If there is currently a blocked thread, the data providing device adopts a synchronous control method to select a thread from the currently blocked threads and wake up, and use the thread as the current working thread. , and step 1022 is executed.

Using the above method embodiments can reduce the number of accesses to the database when adding data to the database, thereby reducing the access pressure to the database in a scenario of high data concurrency.

Fig. 2 is a kind of data storage device corresponding to the above-mentioned method, and this device is provided in the data providing equipment, including:

The first unit, when the data providing device needs to add a data record to a preset database table for the first time, obtains the currently available record identifier of the data providing device from the database table, and uses the record identifier to add the data record. In the database table, use the preset number of identification extensions to perform number expansion on the record identification, save the identification number range obtained by the number expansion in the memory of the data providing device, and store the data in the database table. The record identifier available to the data providing device is updated to the maximum value of the identifier number range plus 1; the database table at least includes a record identifier field.

The second unit, when the data providing device needs to add data records to the database table after the first time, adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use The acquired record identifier saves the corresponding data record that needs to be added in the database table, and when all record identifiers in the identifier number range are used, it is triggered to update the identifier number range in the memory.

Preferably, the range of the identification number is [a,b].

Wherein, a is the currently acquired record identifier; b=a+n-1, and n is the identifier extension number.

Preferably, the second unit adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use the acquired record identifiers to store the corresponding data records that need to be added to the memory. In the database table, when the record identifiers in the identification number range are all used, triggering the update of the identification number range in the memory includes:

Step a1, the data providing equipment adopts the mode of synchronous control, selects a thread from the thread that currently needs to increase the data record in the data providing equipment, as the working thread that can currently increase the data record, and blocks other described threads .

Step a2, from the memory of the data providing device, take out the minimum mark that has not been taken out in the range of the identification number, trigger the working thread to use the minimum mark to add the corresponding data record that needs to be added to the database table; When the minimum identifier is equal to the maximum value of the identifier number range, the data providing device re-acquires the available record identifiers corresponding to the data providing device from the database table, and uses the identifier to expand the number of new records. The obtained record identification carries out number expansion, and the identification number range in the memory is replaced by the identification number scope obtained by the current number expansion, and the available record identification of the data providing equipment in the database table is updated to the current number expansion obtained. The maximum value of the identification number range is incremented by 1.

Step a3: If there is currently a blocked thread, the data providing device adopts a synchronous control method, selects a thread from the currently blocked thread and wakes up, and uses the thread as the current working thread. , go to step a2.

Preferably, the data providing device is a server.

Embodiments of the present invention further provide a non-volatile computer-readable storage medium, where the non-volatile computer-readable storage medium stores instructions, which, when executed by a processor, cause the processor to execute the above-mentioned data Store the steps of a method embodiment.

Correspondingly, an embodiment of the present invention further provides an electronic device, including the non-volatile computer-readable storage medium as described above, and the processor that can access the non-volatile computer-readable storage medium.

To sum up, the data storage scheme proposed by the present invention performs number expansion on the basis of the record identification obtained from the database, generates an identification number segment, and saves it in the memory. In this way, the data providing equipment needs to increase each time. When the data is recorded to the database, it is only necessary to obtain the available data record identifiers from its own memory. Only after the identifiers in the identification number segment in the memory are all used, the new available data record identifiers are requested from the database. , and then use the newly obtained identification to generate an identification number segment and save it to the memory. In this way, the number of accesses to the database can be reduced, thereby reducing the access pressure to the database in the scenario of high data concurrency.

To sum up, the above are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a data storage method, is characterized in that, comprises: When the data providing device needs to add a data record to a preset database table for the first time, it acquires the currently available record identifier of the data providing device from the database table, and uses the record identifier to add the data record to the database In the table, use the preset number of identification extensions to perform number expansion on the record identification, save the identification number range obtained by the number expansion in the memory of the data providing device, and store the data providing device in the database table. The available record identification is updated to the maximum value of the identification number range plus 1; the database table at least includes a record identification field; When the data providing device needs to add data records to the database table after the first time, it adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use the obtained records. The identifier stores the corresponding data records that need to be added into the database table, and when the record identifiers in the identifier number range are all used, triggering the update of the identifier number range in the memory. 2. The method according to claim 1, wherein the identification number range is [a, b], wherein a is the currently acquired record identification; b=a+n-1, n is the Describe the number of ID extensions. 3. method according to claim 1, is characterized in that, described adopting the mode of synchronous control, control the thread that currently needs to increase data record to access memory one by one to obtain corresponding available record identification, and utilize the acquired record identification to correspond to The data records that need to be added are stored in the database table, and when the record identifiers in the identification number range are all used, triggering the update of the identification number range in the memory includes: a1, the data providing equipment adopts the mode of synchronous control, selects a thread from the thread that currently needs to increase the data record in this data providing equipment, as the working thread that can currently increase the data record, and blocks other described threads; a2. From the memory of the data providing device, take out the smallest identifier that has not been taken out in the identifier number range, and trigger the worker thread to use the smallest identifier to add the corresponding data records that need to be added to the database table; when When the minimum identifier is equal to the maximum value of the identifier number range, the data providing device re-acquires the available record identifiers corresponding to the data providing device from the database table, and uses the identifier extension number to re-acquire The record identification carries out number expansion, replaces the identification number range in the memory with the identification number range obtained by the current number expansion, and updates the available record identification of the data providing equipment in the database table to the identification obtained by the current number expansion Add 1 to the maximum value of the number range; a3. If there is currently a blocked thread, the data providing device adopts a synchronous control method to select a thread from the currently blocked threads and wake up, and use the thread as the current working thread, Go to step a2. 4. The method according to claim 1, wherein the data providing device is a server. 5. A data storage device, characterized in that it is located in a data providing device, comprising: The first unit, when the data providing device needs to add a data record to a preset database table for the first time, obtains the currently available record identifier of the data providing device from the database table, and uses the record identifier to add the data record. In the database table, use the preset number of identification extensions to perform number expansion on the record identification, save the identification number range obtained by the number expansion in the memory of the data providing device, and store the data in the database table. The record identification available to the data providing device is updated to the maximum value of the identification number range plus 1; the database table at least includes a record identification field; The second unit, when the data providing device needs to add data records to the database table after the first time, adopts a synchronous control method to control the threads that currently need to add data records to access the memory one by one to obtain the corresponding available record identifiers, and use all The acquired record identifiers store corresponding data records that need to be added in the database table, and when all record identifiers in the identifier number range are used, triggering the update of the identifier number range in the memory. 6. The device according to claim 5, wherein the identification number range is [a, b], wherein a is the currently acquired record identification; b=a+n-1, n is the Describes the number of ID extensions. 7. device according to claim 5, is characterized in that, described second unit, adopts the mode of synchronous control, controls the thread that currently needs to increase data record to access memory one by one to obtain corresponding available record identification, and utilize the acquired The record identifier saves the corresponding data records that need to be added to the database table, and when the record identifiers in the identifier number range are all used, triggering to update the identifier number range in the memory includes: a1, the data providing equipment adopts the mode of synchronous control, selects a thread from the thread that currently needs to increase the data record in this data providing equipment, as the working thread that can currently increase the data record, and blocks other described threads; a2. From the memory of the data providing device, take out the smallest identifier that has not been taken out in the identifier number range, and trigger the worker thread to use the smallest identifier to add the corresponding data records that need to be added to the database table; when When the minimum identifier is equal to the maximum value of the identifier number range, the data providing device re-acquires the available record identifiers corresponding to the data providing device from the database table, and uses the identifier extension number to re-acquire The record identification carries out number expansion, replaces the identification number range in the memory with the identification number range obtained by the current number expansion, and updates the available record identification of the data providing equipment in the database table to the identification obtained by the current number expansion Add 1 to the maximum value of the number range; a3. If there is currently a blocked thread, the data providing device adopts a synchronous control method to select a thread from the currently blocked thread and wake it up, and use the thread as the current working thread, Go to step a2. 8. The apparatus according to claim 5, wherein the data providing device is a server. 9. A non-volatile computer-readable storage medium storing instructions, wherein the instructions, when executed by a processor, cause the processor to execute the method as claimed in the claims Steps of the data storage method described in any one of 1 to 4. 10. An electronic device, comprising the non-volatile computer-readable storage medium of claim 9, and the processor that can access the non-volatile computer-readable storage medium.

Images