CN112765215B - Data query method and device - Google Patents
Data query method and device Download PDFInfo
- Publication number
- CN112765215B CN112765215B CN202110067904.6A CN202110067904A CN112765215B CN 112765215 B CN112765215 B CN 112765215B CN 202110067904 A CN202110067904 A CN 202110067904A CN 112765215 B CN112765215 B CN 112765215B
- Authority
- CN
- China
- Prior art keywords
- cache
- database
- data
- query
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000013515 script Methods 0.000 claims abstract description 71
- 238000012545 processing Methods 0.000 claims abstract description 18
- 230000004044 response Effects 0.000 claims abstract description 18
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 17
- 238000005516 engineering process Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000010295 mobile communication Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
- G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data query method and a data query device. The method comprises the following steps: analyzing a query statement generated based on a data query request triggered by a client to acquire parameter information from the query statement; acquiring version information of a data table corresponding to the data table name; processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name; searching whether a cache file matched with the cache key name exists in a preset first database; if the cache file matched with the cache key name does not exist, inquiring a preset second database according to the inquiry statement to acquire a database inquiry result; generating script type file cache from the database query result; and pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request. The application can improve the data query efficiency.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data query method and apparatus.
Background
The relational database stores a class of lazy data, i.e., data that is queried more frequently than modified. In order to improve the query efficiency and reduce the database pressure, the query result of such inert data is usually converted into a key value file cache.
The conventional key value file generation caching method is divided into a text type file caching method and a script type file caching method. The text file cache needs serialization and deserialization during reading and writing, so that the efficiency of cache reading and analyzing is reduced. The script file cache does not need serialization and can be precompiled into byte codes by the Opcache technology of PHP and stored in a memory.
However, the inventors have found that although the reading efficiency can be further improved by storing as a script-type file cache, when the cached content is modified, the update of the bytecode can only be triggered by restarting the PHP service or waiting for Opcache to refresh. The update of the byte code by restarting the PHP service can cause interruption of user request, affect data query efficiency, and the update of the byte code can not be triggered in real time by waiting for Opcache to refresh, so that the byte code can not be updated in time, and also affect data query efficiency.
Disclosure of Invention
In view of this, a data query method, apparatus, computer device and computer readable storage medium are provided to solve the problem that in the prior art, updated byte codes need to be obtained by restarting the PHP service or waiting for Opcache to refresh, which affects the data query efficiency.
The application provides a data query method, which comprises the following steps:
analyzing a query statement generated based on a data query request to obtain parameter information from the query statement, wherein the parameter information at least comprises a data table name;
acquiring version information of a data table corresponding to the data table name;
processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name;
searching whether a cache file matched with the cache key name exists in a preset first database;
If the first database does not have the cache file matched with the cache key name, inquiring a preset second database according to the inquiry statement to acquire a database inquiry result;
generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value;
And pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request.
Optionally, the method further comprises:
And if the first database contains the cache file matched with the cache key name, acquiring a second byte code generated by precompiled the cache file from a memory, and taking the second byte code as response data corresponding to the data query request.
Optionally, if the first database does not have the cache file matched with the cache key name, querying a preset second database according to the query statement to obtain a database query result includes:
If the first database does not have the cache file matched with the cache key name, executing a preset command to acquire a distributed lock;
And if the distributed lock is successfully obtained, executing the step of inquiring the preset second database according to the inquiry statement to obtain the database inquiry result.
Optionally, if the first database does not have the cache file matched with the cache key name, querying a preset second database according to the query statement to obtain a database query result further includes:
And if the distributed lock is not successfully acquired, re-executing the preset command at intervals of preset time to acquire the distributed lock until the distributed lock is successfully acquired or until the number of times of re-executing the preset command reaches the preset number of times.
Optionally, if the distributed lock is not successfully acquired, re-executing the preset command at intervals of preset time to acquire the distributed lock, and after the step of re-executing the preset command for a preset number of times, further including:
Executing the command for releasing the distributed lock, and returning to the step of executing the preset command to acquire the distributed lock after the release of the distributed lock is completed.
Optionally, after the step of generating the script-type file cache from the database query result, the method further includes:
executing a command to release the distributed lock.
Optionally, the method further comprises:
when a data update request is received, updating version information of a data table associated with update data corresponding to the data update request;
updating script type file caches associated with the data table according to version information of the updated data table, and generating updated script type file caches;
and after generating an updated script type file cache, pre-compiling the updated script type file cache to generate a third byte code.
The application also provides a data query device, which comprises:
the acquisition module is used for acquiring a function to be tested selected by a user based on a target interface in the protocol simulation test tool;
The analysis module is used for analyzing a query statement generated based on the data query request to acquire parameter information from the query statement, wherein the parameter information at least comprises a data table name;
The acquisition module is used for acquiring version information of the data table corresponding to the data table name;
the processing module is used for processing the parameter information and the version information by adopting a preset algorithm so as to generate a cache key name;
The searching module is used for searching whether a cache file matched with the cache key name exists in a preset first database;
The query module is used for querying a preset second database according to the query statement if the first database does not have the cache file matched with the cache key name, so as to obtain a database query result;
the generation module is used for generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value;
and the return module is used for pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request.
The application also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the computer program.
The application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method.
The beneficial effects of the technical scheme are that:
In the embodiment of the application, a query statement generated based on a data query request triggered by a client is analyzed to obtain parameter information from the query statement, wherein the parameter information at least comprises a data table name; acquiring version information of a data table corresponding to the data table name; processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name; searching whether a cache file matched with the cache key name exists in a preset first database; if the first database does not have the cache file matched with the cache key name, inquiring a preset second database according to the inquiry statement to acquire a database inquiry result; generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value; and pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request. In the embodiment of the application, the cache key name formed by the parameter information and the version number associated with the query statement of the generated script type file cache is used as the key of the script type file cache, so that Opcache can identify the key as a new script and correspondingly generate a new byte code, and further updating the byte code can be realized without restarting PHP service or waiting for Opcache refreshing, and the data query efficiency is improved.
Drawings
FIG. 1 is an environmental schematic diagram of a data query method according to an embodiment of the present application;
FIG. 2 is a flow chart of an embodiment of a data query method according to the present application;
FIG. 3 is a detailed flowchart of a step of searching a second database according to the search statement to obtain a database search result if a cache file matching the cache key name does not exist in the first database in an embodiment of the present application;
FIG. 4 is a flowchart of another embodiment of a data query method according to the present application;
FIG. 5 is a block diagram illustrating a data query device according to an embodiment of the present application;
fig. 6 is a schematic hardware structure of a computer device for executing a data query method according to an embodiment of the present application.
Detailed Description
Advantages of the application are further illustrated in the following description, taken in conjunction with the accompanying drawings and detailed description.
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" depending on the context.
In the description of the present application, it should be understood that the numerical references before the steps do not identify the order in which the steps are performed, but are merely used to facilitate description of the present application and to distinguish between each step, and thus should not be construed as limiting the present application.
Fig. 1 schematically shows a frame diagram of a data query method according to an embodiment of the application. In an exemplary embodiment, the system of the application environment may include a computer device 10, a server 20, and a client 30. Wherein the computer device 10 forms a wireless or wired connection with the server 20, and the client 30 is wired or wireless connected with the computer device 10. The computer device 10 may be a cell phone, iPAD, tablet computer, server, etc. The server 20 may be a rack server, a blade server, a tower server, or a rack server (including a stand-alone server, or a server cluster composed of multiple servers), etc. The user may trigger a data query request through the client 30, and when receiving the data query request sent by the client 30, the computer device 10 interacts with the server 20 to respond to the data query request, and returns a response result to the client 30.
Fig. 2 is a flow chart of a data query method according to an embodiment of the application. It will be appreciated that the flow charts in the method embodiments are not intended to limit the order in which the steps are performed. As can be seen from the following description with the computer device as the execution body, the data query method provided in this embodiment includes:
step S20, analyzing a query statement generated based on the data query request to acquire parameter information from the query statement, wherein the parameter information at least comprises a data table name.
Specifically, when a user wants to query service data, the user can trigger a data query request through a client to query service data corresponding to the data query request according to the data query request.
It will be appreciated that the data query request includes data query conditions indicating what data the user needs to query.
In this embodiment, when a data query request is received, in order to obtain data corresponding to the data query request from the database, the data query request is generated into a corresponding query statement, so that the database can be queried by the query statement.
As an example, a query statement may be analyzed by a query statement analysis module to extract parameter information from the query statement. Specifically, the query statement analysis module may obtain parameter information from the query statement according to a preset rule, where the preset rule is set according to an actual service.
It should be noted that, the method name refers to the name of the function called by the query statement for implementing the data query function; the parameter column refers to the name of an input parameter used for realizing a function called by the data query function in the query statement; the data table name is the name of the data table storing the data to be queried.
By way of example, one query statement for querying a game card list is as follows:
function get_card_list($job_id，$rarity){
$sq1＝"SELECT a.*,b.*FROM'card_mst'a
INNER J0IN‘item_mst'b ON b.'card_id'＝a.'card_id'
WHERE'job_id'＝？AND'rarity'＝？"；
$this->_query($sq1，$job_id，$rarity)；
In the query statement, get_card_list is the method name, card_id (card ID) and rarity (card star) are the parameter columns, and card_mst is the data table name.
Preferably, in order to avoid a conflict between the subsequently generated cache key names, in this embodiment, the method name, the parameter sequence and the data table name may be obtained from the query statement at the same time.
And S21, obtaining version information of the data table corresponding to the data table name.
In particular, the version information may include a version number of the data table. In another embodiment, the version information may include information such as creation time, modification time, etc. of the data table, in addition to the version number of the data table.
In this embodiment, when a data table is newly created, version information associated with the created data table is generated, and when data included in the data table is updated, the version information associated with the data table is updated.
Step S22, processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name.
Specifically, the preset algorithm is preset, and the preset algorithm may be a hash algorithm (or referred to as a hash algorithm), such as MD5 algorithm, SHA-1 algorithm, or the like.
Because the parameter information and the version information are a string of uncontrollable strings (i.e. the parameter information and the version information in different scenes are different), in order to make the length of the generated cache key name controllable, in this embodiment, the parameter information and the version information may be processed by the preset algorithm to generate a string with a fixed length, and the generated string with a fixed length is used as the cache key name (key name).
It can be understood that when the cache key name is generated, the parameter information and the version information need to be spliced, and then the spliced character string is processed to generate the cache key name.
Step S23, searching whether a cache file matched with the cache key name exists in a preset first database.
Specifically, the first database is a database for storing the cache file, and the database is a non-relational database for storing data in a key-value (key-value pair) form.
As an example, when searching the cache file, the cache key name may be sequentially compared with key names (key names) corresponding to respective cache files stored in the first database, and when it is found that there is a key name identical to the cache key name, a value corresponding to the key name is used as the cache file.
Step S24, if the first database does not have the cache file matched with the cache key name, a preset second database is queried according to the query statement to obtain a database query result.
Specifically, the second database is a database for storing service data, and the database is a relational database, wherein the relational database is a database for organizing data by adopting a relational model, and the database stores data in a form of rows and columns.
In this embodiment, when the cache file matching the cache key name is not found in the first database, it indicates that the cache file does not exist in the first database. At this time, the second database needs to be queried through the data query statement to obtain the database query result from the second database.
In an exemplary embodiment, if the first database includes a cache file that matches the cache key name, a second bytecode generated by precompiled the cache file is obtained from a memory, and the second bytecode is used as response data corresponding to the data query request.
Specifically, when a cache file matched with the cache key exists in the first database, in order to improve the data query efficiency, a second byte code generated by precompiled the cache file through Opcache technology is stored in a local memory, so that in the subsequent receiving of a query request of the client for the cache file, the second byte code can be directly obtained from the memory, and the second byte code is used as response data corresponding to the data query request to be returned to the client.
The Opcache technology is an extension method for improving performance of PHP by storing byte codes generated by pre-compiling PHP scripts into a shared memory.
In an exemplary embodiment, referring to fig. 3, in order to prevent multiple processes from simultaneously querying the second database to obtain a database query result, the step S24 may include: steps S30-S31, wherein:
step S30, if the first database does not have the cache file matching the cache key name, executing a preset command to obtain the distributed lock.
Specifically, the preset command is a command for locking the query statement, which may be a Setnx command, by way of example, through which the distributed lock function may be implemented, where Setnx command is a command in the Redis database, and is a shorthand for SET if not exists (SET if not present). In executing the command, the value of the key is set to value if and only if the key does not exist. If a given key already exists, setnx does nothing.
Wherein the Redis database is a high-performance key-value database.
Step S31, if the distributed lock is successfully obtained, inquiring a preset second database according to the inquiry statement to obtain a database inquiry result.
Specifically, when the distributed lock is successfully obtained, a data query operation may be performed to obtain a database query result. And when the distributed lock is not successfully acquired, indicating that other processes acquire the distributed lock, re-executing the preset command to acquire the distributed lock at preset time intervals until the distributed lock is successfully acquired or until the times of re-executing the preset command reach preset times.
As an example, assuming that the preset time is 5 seconds and the preset number of times is 8, when the distributed lock is not acquired, the Setnx command may be re-executed every 5 seconds to continue to acquire the distributed lock until the distributed lock is successfully acquired; or the Setnx commands may be re-executed every 5 seconds when no distributed lock is acquired, and the Setnx commands may be stopped when the distributed lock is still not acquired after 8 times of the Setnx commands are executed.
It should be noted that the foregoing preset time and the preset number of times are exemplary, and specific values thereof may be set and modified according to actual situations, which are not limited in this embodiment.
In an exemplary embodiment, in order to avoid a situation that other processes have deadlock after acquiring the distributed lock, in this embodiment, if the distributed lock is not successfully acquired, the step of re-executing the preset command at intervals of preset time to acquire the distributed lock until the number of times of re-executing the preset command reaches a preset number of times further includes:
Executing the command for releasing the distributed lock, and returning to the step of executing the preset command to acquire the distributed lock after the release of the distributed lock is completed.
As an example, release of the distributed lock may be achieved by executing a DEL lock info command, and after release of the distributed lock is completed, the Setnx command may be re-executed to acquire the distributed lock.
Step S25, generating a script type file cache from the database query result, wherein the script type file cache uses the cache key name as a key name, and the database query result is used as a value.
Specifically, after the database query result is obtained, the database query result in the form of rows and columns may be converted into a script file cache that is stored in the form of key-value pairs (key-value).
The script type file cache refers to a cache file stored in a key value pair storage mode, and each key corresponds to a unique value when the cache file is stored in the key value pair storage mode. The script type file cache is a cache file which can be precompiled into byte codes to be stored in a memory by the Opcache technology of PHP without serialization.
Note that PHP is a hypertext preprocessor, which is a general purpose open source scripting language.
The Opcache technology is an extension method for improving the performance of PHP by storing byte codes generated by pre-compiling PHP scripts into a shared memory.
In an exemplary embodiment, after the current process completes the generation of the script-type file cache, in order to enable other processes to read the script-type cache file, in this embodiment, after the step of generating the script-type file cache from the database query result, the method further includes the steps of:
executing a command to release the distributed lock.
As an example, release of the distributed lock may be achieved by executing a DEL lock.
It will be appreciated that, since the distributed lock may be released by other processes, in this embodiment, in order to avoid repeatedly executing the command for releasing the distributed lock, which results in time expense, in this embodiment, before executing the step of releasing the command for releasing the distributed lock, it may be further determined whether the distributed lock is still present at the current time, if not, it is not necessary to execute the command for releasing the distributed lock, and only when the distributed lock is still present at the current time, it is necessary to execute the command for releasing the distributed lock to release the distributed lock.
And S26, pre-compiling the script file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request.
Specifically, after the script type file cache is generated, the script type file cache can be immediately precompiled through Opcache technology to generate a first byte code, and the first byte code is used as response data corresponding to the data query request to be returned to the client, so that query data corresponding to the query request can be returned to the client in time after the data query request of the client is received again, and the data query efficiency is improved.
It may be appreciated that, in order to pre-compile the script type file cache to generate the first bytecode by using the Opcache technology, it is required to first determine whether the Opcache extended function is started, and when the Opcache extended function is not started, the user may be reminded to start the Opcache extended function, so that the script type file cache may be pre-compiled to generate the first bytecode by using the Opcache technology.
In the embodiment of the application, a query statement generated based on a data query request triggered by a client is analyzed to obtain parameter information from the query statement, wherein the parameter information at least comprises a data table name; acquiring version information of a data table corresponding to the data table name; processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name; searching whether a cache file matched with the cache key name exists in a preset first database; if the first database does not have the cache file matched with the cache key name, inquiring a preset second database according to the inquiry statement to acquire a database inquiry result; generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value; and pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request. In the embodiment of the application, the cache key name formed by the parameter information and the version number associated with the query statement of the generated script type file cache is used as the key of the script type file cache, so that the computer equipment can recognize the key as a new script through a preset technology (such as Opcache technology) and correspondingly generate a new byte code, and updating of the byte code can be realized without restarting PHP service or waiting for Opcache refreshing, and the data query efficiency is improved. Meanwhile, the embodiment of the application can realize the purpose of efficiently distributing the locks by utilizing the characteristic of Setnx commands of Redis, and can avoid the deadlock condition by adding the expiration time to the distributed locks, thereby improving the overall user experience.
Fig. 4 schematically shows a flow chart of a data query method according to another embodiment of the application. As shown in fig. 4, the method further comprises steps S40-S42, wherein:
Step S40, when a data update request is received, version information of a data table associated with update data corresponding to the data update request is updated.
Specifically, when the service data is changed, a data update request may be triggered by the client to update the previously stored service data.
In this embodiment, after the data update is completed, the version information of the data table associated with the updated data needs to be updated synchronously, for example, before the data update, the version information of the data table associated with the updated data is 1.02.42, and after the data update is completed, the version information of the data table associated with the updated data may be updated to 1.02.43.
And step S41, updating the script type file cache associated with the data table according to the version information of the updated data table, and generating an updated script type file cache.
Specifically, after the version information of the data table changes, the key name of the script-type file cache associated with the data table changes due to the change of the version information. After the key name is changed, in order to keep the script type file cache corresponding to the updated key name, the script type file cache needs to be updated correspondingly, that is, a new script type file cache needs to be regenerated, and the generating manner is the same as that of the script type cache file in the above embodiment, which is not described in detail in this embodiment.
Step S42, after generating the updated script type file cache, pre-compiling the updated script type file cache to generate a third byte code.
Specifically, after the script type file cache is changed, the updated script type file cache may be precompiled by using Opcache technology to generate a third byte code for use by a subsequent process.
According to the embodiment, after the data is updated, the updated third byte codes are timely generated, so that a subsequent process can timely inquire the updated data, and the data inquiry efficiency is improved.
Referring to FIG. 5, a block diagram of a data query device 50 according to an embodiment of the present application is shown.
In this embodiment, the data query device 50 includes a series of computer program instructions stored on a memory, which when executed by a processor, implement the data query functions of the various embodiments of the present application. In some embodiments, the data querying device 50 may be divided into one or more modules based on the particular operations implemented by portions of the computer program instructions. For example, in fig. 5, the data query device 50 may be partitioned into an analysis module 51, an acquisition module 52, a processing module 53, a lookup module 54, a query module 55, a generation module 56, and a return module 57. Wherein:
An analysis module 51, configured to analyze a query statement generated based on a data query request, so as to obtain parameter information from the query statement, where the parameter information includes at least a data table name;
an obtaining module 52, configured to obtain version information of a data table corresponding to the data table name;
The processing module 53 is configured to process the parameter information and the version information by using a preset algorithm to generate a cache key name;
the searching module 54 is configured to search whether a cache file matching the cache key name exists in a preset first database;
A query module 55, configured to query a preset second database according to the query statement if the first database does not have a cache file matching the cache key name, so as to obtain a database query result;
The generating module 56 is configured to generate a script file cache from the database query result, where the script file cache uses the cache key name as a key name, and the database query result is used as a value;
And the return module 57 is configured to pre-compile the script file cache to generate a first byte code, and use the first byte code as response data corresponding to the data query request.
In an exemplary embodiment, the return module 57 is further configured to, if a cache file matching the cache key name exists in the first database, obtain, from a memory, a second bytecode generated by precompiled the cache file, and use the second bytecode as response data corresponding to the data query request.
In an exemplary embodiment, the query module 55 is further configured to execute a preset command to obtain a distributed lock if the first database does not have a cache file matching the cache key name; and if the distributed lock is successfully obtained, inquiring a preset second database according to the inquiry statement to obtain a database inquiry result.
In an exemplary embodiment, the query module 55 is further configured to re-execute the preset command to acquire the distributed lock at preset time intervals if the distributed lock is not successfully acquired, until the distributed lock is successfully acquired or until the number of times of re-executing the preset command reaches a preset number of times.
In an exemplary embodiment, the query module 55 is further configured to execute the command for releasing the distributed lock, and execute the preset command to obtain the distributed lock after the release of the distributed lock is completed.
In an exemplary embodiment, the data query device 50 further includes a release module.
And the release module is used for executing a command for releasing the distributed lock so as to release the distributed lock.
In an exemplary embodiment, the data query device 50 further includes an update module.
The updating module is used for updating version information of a data table associated with update data corresponding to the data updating request when the data updating request is received;
the generating module 56 is further configured to update a script-type file cache associated with the data table according to version information of the updated data table, and generate an updated script-type file cache;
The return module 57 is further configured to pre-compile the updated script-type file cache to generate a third bytecode after generating the updated script-type file cache.
In the embodiment of the application, a query statement generated based on a data query request triggered by a client is analyzed to obtain parameter information from the query statement, wherein the parameter information at least comprises a data table name; acquiring version information of a data table corresponding to the data table name; processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name; searching whether a cache file matched with the cache key name exists in a preset first database; if the first database does not have the cache file matched with the cache key name, inquiring a preset second database according to the inquiry statement to acquire a database inquiry result; generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value; and pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request. In the embodiment of the application, the cache key name formed by the parameter information and the version number associated with the query statement of the generated script type file cache is used as the key of the script type file cache, so that the computer equipment can recognize the key as a new script through a preset technology (such as Opcache technology) and correspondingly generate a new byte code, and updating of the byte code can be realized without restarting PHP service or waiting for Opcache refreshing, and the data query efficiency is improved. Meanwhile, the embodiment of the application can realize the purpose of efficiently distributing the locks by utilizing the characteristic of Setnx commands of Redis, and can avoid the deadlock condition by adding the expiration time to the distributed locks, thereby improving the overall user experience.
Fig. 6 schematically shows a hardware architecture diagram of a computer device 10 adapted to implement a data query method according to an embodiment of the application. In the present embodiment, the computer device 10 is a device capable of automatically performing numerical calculation and/or information processing in accordance with instructions set or stored in advance. For example, the server may be a tablet computer, a notebook computer, a desktop computer, a rack-mounted server, a blade server, a tower server, or a rack server (including a stand-alone server or a server cluster formed by a plurality of servers), etc. As shown in fig. 6, the computer device 10 includes at least, but is not limited to: memory 120, processor 121, and network interface 123 may be communicatively linked to each other by a system bus. Wherein:
Memory 120 includes at least one type of computer-readable storage medium that may be volatile or nonvolatile, and specifically, readable storage media include flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 120 may be an internal storage module of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 120 may also be an external storage device of the computer device 10, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, abbreviated as SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the computer device 10. Of course, the memory 120 may also include both internal memory modules of the computer device 10 and external memory devices. In this embodiment, the memory 120 is typically used to store an operating system and various types of application software installed on the computer device 10, such as program codes of a data query method, and the like. In addition, the memory 120 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 121 may be a central processing unit (Central Processing Unit, simply CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 121 is typically used to control the overall operation of the computer device 10, such as performing control and processing related to data interaction or communication with the computer device 10, and the like. In this embodiment, the processor 121 is configured to execute program codes or process data stored in the memory 120.
The network interface 123 may include a wireless network interface or a wired network interface, the network interface 123 typically being used to establish a communication link between the computer device 10 and other computer devices. For example, the network interface 123 is used to connect the computer device 10 to an external terminal through a network, establish a data transmission channel and a communication link between the computer device 10 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, abbreviated as GSM), wideband code division multiple access (Wideband Code Division Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc.
It should be noted that fig. 6 only shows a computer device having components 120-122, but it should be understood that not all of the illustrated components are required to be implemented, and that more or fewer components may be implemented instead.
In this embodiment, the data query method stored in the memory 120 may be divided into one or more program modules and executed by one or more processors (the processor 121 in this embodiment) to complete the present application.
The embodiment of the application provides a computer readable storage medium, and the computer readable storage medium stores a computer program thereon, and the computer program when executed by a processor realizes the steps of the data query method in the embodiment.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, abbreviated as SMC), a Secure Digital (abbreviated as SD) card, a flash memory card (FLASH CARD), or the like, which are provided on the computer device. Of course, the computer-readable storage medium may also include both internal storage units of a computer device and external storage devices. In this embodiment, the computer readable storage medium is typically used to store an operating system and various types of application software installed on a computer device, such as program codes of the data query method in the embodiment, and the like. Furthermore, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
The apparatus embodiments described above are merely illustrative, wherein elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over at least two network elements. Some or all modules in the system can be screened out according to actual needs to realize the purpose of the scheme of the embodiment of the application. Those of ordinary skill in the art will understand and implement the present application without undue burden.
From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Those skilled in the art will appreciate that all or part of the processes implementing the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and where the program may include processes implementing the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-only memory (ROM), a random access memory (RandomAccessMemory, RAM), or the like.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.
Claims (10)
1. A method of querying data, comprising:
analyzing a query statement generated based on a data query request, and acquiring parameter information from the query statement, wherein the parameter information at least comprises a data table name;
acquiring version information of a data table corresponding to the data table name;
processing the parameter information and the version information by adopting a preset algorithm to generate a cache key name;
searching whether a cache file matched with the cache key name exists in a preset first database;
If the first database does not have the cache file matched with the cache key name, inquiring a preset second database according to the inquiry statement to acquire a database inquiry result;
generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value;
And precompiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request, wherein the first byte code is generated by precompiling the script type file cache through Opcache technology immediately after the script type file cache is generated.
2. The data query method of claim 1, wherein the method further comprises:
And if the first database contains the cache file matched with the cache key name, acquiring a second byte code generated by precompiled the cache file from a memory, and taking the second byte code as response data corresponding to the data query request.
3. The method of claim 1, wherein if the first database does not have a cache file matching the cache key name, the querying a preset second database according to the query statement to obtain a database query result includes:
If the first database does not have the cache file matched with the cache key name, executing a preset command to acquire a distributed lock;
And if the distributed lock is successfully obtained, executing the step of inquiring the preset second database according to the inquiry statement to obtain the database inquiry result.
4. The data query method according to claim 3, wherein if the first database does not have the cache file matching the cache key name, querying a preset second database according to the query statement to obtain a database query result further comprises:
And if the distributed lock is not successfully acquired, re-executing the preset command at intervals of preset time to acquire the distributed lock until the distributed lock is successfully acquired or until the number of times of re-executing the preset command reaches the preset number of times.
5. The data query method as claimed in claim 4, wherein, if the distributed lock is not successfully acquired, re-executing the preset command at intervals of preset time to acquire the distributed lock, until the number of re-executing the preset command reaches a preset number of times, further comprising:
Executing the command for releasing the distributed lock, and returning to the step of executing the preset command to acquire the distributed lock after the release of the distributed lock is completed.
6. The data query method of claim 3, wherein after the step of generating a script-type file cache from the database query results, further comprising:
executing a command to release the distributed lock.
7. The data query method of any one of claims 1 to 6, further comprising:
when a data update request is received, updating version information of a data table associated with update data corresponding to the data update request;
updating script type file caches associated with the data table according to version information of the updated data table, and generating updated script type file caches;
and after generating an updated script type file cache, pre-compiling the updated script type file cache to generate a third byte code.
8. A data query device, comprising:
The analysis module is used for analyzing a query statement generated based on the data query request and acquiring parameter information from the query statement, wherein the parameter information at least comprises a data table name;
The acquisition module is used for acquiring version information of the data table corresponding to the data table name;
the processing module is used for processing the parameter information and the version information by adopting a preset algorithm so as to generate a cache key name;
The searching module is used for searching whether a cache file matched with the cache key name exists in a preset first database;
The query module is used for querying a preset second database according to the query statement if the first database does not have the cache file matched with the cache key name, so as to obtain a database query result;
the generation module is used for generating a script type file cache from the database query result, wherein the script type file cache takes the cache key name as a key name, and the database query result is taken as a value;
And the return module is used for pre-compiling the script type file cache to generate a first byte code, and taking the first byte code as response data corresponding to the data query request, wherein after the script type file cache is generated, the script type file cache is immediately pre-compiled by Opcache technology to generate the first byte code.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the data querying method according to any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the data query method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110067904.6A CN112765215B (en) | 2021-01-19 | 2021-01-19 | Data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110067904.6A CN112765215B (en) | 2021-01-19 | 2021-01-19 | Data query method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112765215A CN112765215A (en) | 2021-05-07 |
| CN112765215B true CN112765215B (en) | 2024-07-12 |
Family
ID=75703065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110067904.6A Active CN112765215B (en) | 2021-01-19 | 2021-01-19 | Data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112765215B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113779091A (en) * | 2021-09-15 | 2021-12-10 | 贵阳货车帮科技有限公司 | Information query method, device, electronic device and readable storage medium |
| CN114036191A (en) * | 2021-11-05 | 2022-02-11 | 上海派拉软件股份有限公司 | Efficient development query method and system based on automatic caching of database data |
| CN114237806B (en) * | 2021-12-29 | 2024-03-12 | 瀚云科技有限公司 | Page information display method and device, electronic equipment and storage medium |
| CN114579617A (en) * | 2022-03-14 | 2022-06-03 | 中国工商银行股份有限公司 | Data query method and device, computer equipment and storage medium |
| CN115544038A (en) * | 2022-09-30 | 2022-12-30 | 山石网科通信技术股份有限公司 | Log data processing method, device, storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110134705A (en) * | 2018-02-09 | 2019-08-16 | 中国移动通信集团有限公司 | A data query method, cache server and terminal |
| CN110909025A (en) * | 2018-09-17 | 2020-03-24 | 深圳市优必选科技有限公司 | Database query method, query device and terminal |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7930301B2 (en) * | 2003-03-31 | 2011-04-19 | Microsoft Corporation | System and method for searching computer files and returning identified files and associated files |
| CN106980506A (en) * | 2017-03-31 | 2017-07-25 | 山东超越数控电子有限公司 | A kind of PHP protection of source codes method |
| CN107577506B (en) * | 2017-08-07 | 2021-03-19 | 台州市吉吉知识产权运营有限公司 | Data preloading method and system |
| CN111046116A (en) * | 2019-07-10 | 2020-04-21 | 广东星舆科技有限公司 | Rapid query and transmission method for geographic space data |
| CN111190928A (en) * | 2019-12-24 | 2020-05-22 | 平安普惠企业管理有限公司 | Cache processing method, apparatus, computer equipment, and storage medium |
| CN111339171B (en) * | 2020-02-07 | 2021-05-04 | 拉扎斯网络科技（上海）有限公司 | Data query method, device and device |
| CN111737295B (en) * | 2020-06-11 | 2023-02-03 | 上海达梦数据库有限公司 | Database cursor query method, device, equipment and storage medium |
-
2021
- 2021-01-19 CN CN202110067904.6A patent/CN112765215B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110134705A (en) * | 2018-02-09 | 2019-08-16 | 中国移动通信集团有限公司 | A data query method, cache server and terminal |
| CN110909025A (en) * | 2018-09-17 | 2020-03-24 | 深圳市优必选科技有限公司 | Database query method, query device and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112765215A (en) | 2021-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112765215B (en) | Data query method and device | |
| US8782604B2 (en) | Sandbox support for metadata in running applications | |
| US9778991B2 (en) | Exporting and importing database tables in a multi-user database environment | |
| US8583613B2 (en) | On demand data conversion | |
| US20180046675A1 (en) | Automatic adjustment of an execution plan for a query | |
| US20160171039A1 (en) | Generating hash values | |
| US20140164391A1 (en) | Data block saving system and method | |
| US20100153929A1 (en) | Converting javascript into a device-independent representation | |
| KR20130054325A (en) | Encoding a schema version in table names | |
| US9384236B2 (en) | Method and system for operating on database queries | |
| CN109586948A (en) | Update method, apparatus, computer equipment and the storage medium of system configuration data | |
| CN111737227B (en) | Data modification method and system | |
| CN111414379A (en) | Serial number generation method, apparatus, device, and computer-readable storage medium | |
| CN109284125B (en) | Method, device, equipment and medium for configuring dependency package in big data platform | |
| US11360975B2 (en) | Data providing apparatus and data providing method | |
| JP2021002830A (en) | Advanced database decompression | |
| CN104881454A (en) | Updating method and system of parameter | |
| CN111723077A (en) | Data dictionary maintenance method and device and computer equipment | |
| US9633035B2 (en) | Storage system and methods for time continuum data retrieval | |
| US9922080B2 (en) | Query plan determination | |
| US11423003B2 (en) | Optimistic concurrency control for database transactions | |
| CN114625751B (en) | Data traceability query method and device based on blockchain | |
| CN113590144B (en) | Dependency processing method and device | |
| CN111367869A (en) | Mirror image file processing method and device, storage medium and electronic equipment | |
| CN111400243B (en) | Development management system based on pipeline service and file storage method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |