CN114077619A - Data query method and device, electronic equipment and storage medium - Google Patents
Data query method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114077619A CN114077619A CN202010843817.0A CN202010843817A CN114077619A CN 114077619 A CN114077619 A CN 114077619A CN 202010843817 A CN202010843817 A CN 202010843817A CN 114077619 A CN114077619 A CN 114077619A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- architecture
- target
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000013500 data storage Methods 0.000 claims abstract description 172
- 230000006870 function Effects 0.000 claims description 56
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012423 maintenance Methods 0.000 abstract description 19
- 230000002159 abnormal effect Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 230000015556 catabolic process Effects 0.000 description 7
- 238000006731 degradation reaction Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 239000002131 composite material Substances 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 238000003780 insertion Methods 0.000 description 3
- 230000037431 insertion Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 238000010792 warming Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24532—Query optimisation of parallel queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the disclosure provides a query method, a query device, electronic equipment and a storage medium. The method comprises the following steps: when a data query request is received, acquiring scene information of the data query request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file; loading a target data storage architecture according to the target architecture configuration information; and querying the data to be queried of the data query request based on the target data storage architecture. By adopting the technical scheme, the configuration information of the data storage architecture used by the server when the server inquires data in different application scenes is recorded in the configuration file of the server instead of the program code of the inquiry service provided by the server, so that the program code of the inquiry service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced.
Description
Technical Field
The disclosed embodiments relate to data query technologies, and in particular, to a data query method and apparatus, an electronic device, and a storage medium.
Background
When the user has a data query requirement, the server needs to provide a data query service for the user.
When providing a service with a high Query Per Second (QPS) for a user, such as a service with a QPS greater than one hundred thousand, the bottom layer needs to support the service through various data storage services (such as MySQL, Redis, MongoDB, local cache, etc.). Because the performance of different storage services is different, and the positions and roles of the same data storage service in the data storage architectures of different data query scenarios may also be different, the data storage architecture of each data query scenario needs to be preset.
In the prior art, a data storage architecture of a few common scenes is specified in a program code of a data query service mainly in a program writing stage. However, when the data storage architecture is more hierarchical and complex, this arrangement may make the program code cumbersome and difficult to maintain.
Disclosure of Invention
The embodiment of the disclosure provides a data query method, a data query device, electronic equipment and a storage medium, so as to simplify program codes of a server for providing data query service and reduce maintenance difficulty of the program codes.
In a first aspect, an embodiment of the present disclosure provides a data query method, including:
when a data query request is received, acquiring scene information of the data query request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file;
loading a target data storage architecture according to the target architecture configuration information;
and querying the data to be queried of the data query request based on the target data storage architecture.
In a second aspect, an embodiment of the present disclosure further provides a data query apparatus, including:
the information acquisition module is used for acquiring scene information of a data query request and target architecture configuration information corresponding to the scene information when the data query request is received, and the target architecture configuration information is recorded in a configuration file;
the architecture loading module is used for loading a target data storage architecture according to the target architecture configuration information;
and the data query module is used for querying the data to be queried of the data query request based on the target data storage architecture.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the data query method according to the embodiment of the disclosure.
In a fourth aspect, the disclosed embodiment also provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data query method according to the disclosed embodiment.
According to the data query method, the data query device, the electronic device and the storage medium provided by the embodiment of the disclosure, when a data query request is received, scene information of the data query request and target architecture configuration information corresponding to the scene information and recorded in a configuration file are obtained, and a target data storage architecture is loaded according to the target architecture configuration information, so that data to be queried of the data query request is queried based on the target data storage architecture. By adopting the technical scheme, the configuration information of the data storage architecture used by the server when the server inquires data in different application scenes is recorded in the configuration file of the server instead of the program code of the inquiry service provided by the server, so that the program code of the inquiry service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced. And the storage architecture is configured in a scene in the configuration file, so that different data storage architectures are set for different application scenes, and the expandability of the program code can be enhanced.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
Fig. 1 is a schematic flow chart of a data query method provided in an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of another data query method provided in the embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a multi-way tree corresponding to a data storage architecture according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a multi-way tree corresponding to an updated data storage architecture according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of a multi-way tree corresponding to another updated data storage architecture according to an embodiment of the present disclosure;
fig. 6 is a block diagram of a data query device according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units. [ ordinal numbers ]
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Fig. 1 is a schematic flowchart of a data query method according to an embodiment of the present disclosure. The method may be performed by a data query apparatus, wherein the apparatus may be implemented by software and/or hardware, may be configured in an electronic device, and may typically be configured in a server. Optionally, the data query method provided in the embodiment of the present disclosure is suitable for a case of performing data query based on a received data query request, and is particularly suitable for a case of performing data query based on a data storage architecture in which a storage node performs data storage in a key value pair storage manner after receiving the data query request. As shown in fig. 1, a data query method provided by an embodiment of the present disclosure may include:
s101, when a data query request is received, scene information of the data query request and target architecture configuration information corresponding to the scene information are obtained, and the target architecture configuration information is recorded in a configuration file.
The data query request is a data reading request, the data query request carries a data ID of data to be queried, and the data query request can be generated by the terminal device based on a trigger operation of a user when the user has a data query requirement, and is sent to the server. The context information of the data query request may be understood as information describing an application context of the data query request, such as a context ID of the application context of the data query request, or context parameters such as an upstream service party, a service cluster and/or a sending time of the data query request, and the like. The target architecture configuration information is configuration information of a data storage architecture suitable for an application scenario to which the data query request belongs, and in different application scenarios, data can be queried based on different data storage architectures. The configuration file may be understood as a file for recording architecture configuration information of each data storage architecture configured by the server for implementing data query, and the architecture configuration information recorded in the configuration file may be preset by a developer or an operation and maintenance person of the server before the server operates and/or updated during the server operates. Here, the data ID is identity information of data, the scene ID is identity information of an application scene, and the identity information is unique in the server, that is, the data can be uniquely determined by the data ID of a certain data, and the application scene can be uniquely determined by the scene ID of a certain application scene.
Specifically, after receiving a data query request sent by a terminal device, a server analyzes the data query request to obtain scene parameters of an application scene of the data query request, and further, when the scene parameters of each application scene and the architecture configuration information of the corresponding application scene are correspondingly recorded in a configuration file, the server can query and obtain the architecture configuration information corresponding to the scene parameters according to the corresponding relationship between each scene parameter and each architecture configuration information recorded in the configuration file, and the architecture configuration information is used as target architecture configuration information; when the configuration file correspondingly records the scene ID of each application scene and the architecture configuration information of the corresponding application scene, the scene ID of the application scene corresponding to the scene parameter can be obtained by querying according to a preset corresponding relation table between the scene parameter and the scene ID, and the architecture configuration information of the application scene to which the data query request belongs can be obtained by querying according to the architecture configuration information corresponding to each scene ID recorded in the configuration file, and the obtained architecture configuration information serves as the target architecture configuration information.
And S102, loading a target data storage framework according to the target framework configuration information.
The target data storage architecture can be understood as a storage architecture adopted when data to be queried of a data query request is queried, and the architecture configuration information is target architecture configuration information. The target data storage architecture may include a plurality of storage nodes capable of providing data storage services, and node types of the storage nodes may be the same or different, for example, the target data storage architecture may be provided with a plurality of storage nodes of the same type, or only with storage nodes of different types. The target data storage architecture can be abstracted into a multi-branch tree, that is, the target data storage architecture can be presented in a memory in the form of a multi-branch tree, correspondingly, each storage node in the target data storage architecture can be abstracted into a node in the multi-branch tree, the flow of data can be abstracted into the edge of the multi-branch tree, and the data storage architectures of different scenes can form a forest. The node type of the storage node can be local cache, Redis, MySQL or MongoDB, and correspondingly, the target data storage node can be provided with the local cache, one or more Redis databases, one or more MySQL databases and/or one or more MongoDB databases, and the like.
Specifically, the server reads the target architecture configuration information in the configuration file, and loads the target architecture configuration information into the memory according to the read target architecture configuration information to obtain a data storage architecture corresponding to the target architecture configuration information, that is, a target data storage architecture.
In this embodiment, the server may execute the operation of loading the target data storage architecture each time a data query request is received; the operation of loading the target data storage architecture may be executed only when the data storage architecture corresponding to the application scenario to which the received data query request belongs is not loaded in the memory or the architecture configuration information of the data storage architecture corresponding to the application scenario to which the received data query request belongs loaded in the memory is different from the acquired target architecture configuration information, which is not limited in this embodiment.
S103, inquiring the data to be inquired of the data inquiry request based on the target data storage architecture.
The data to be queried is data required to be queried by the data query request.
Specifically, the server analyzes the received data query request in advance to obtain the data ID of the data to be queried of the data query request. Therefore, after the target data storage architecture is loaded, the server may obtain the data to be queried from one or more storage nodes in the target data storage architecture according to the data ID of the data to be queried, for example, obtain the data to be queried from one or more storage nodes in the target data storage architecture randomly or in a certain order (for example, in an order from a large response speed to a small response speed of each storage node or in an order from a high level to a low level of each storage node in the target data storage architecture).
Considering that a developer or an operation and maintenance person usually sets a hierarchical relationship of each storage node in a target data storage architecture as an optimal data query mode when configuring the target data storage architecture, that is, a sequence from high to low of the hierarchy of each storage node in the target data storage architecture is usually an optimal sequence when querying data by using the target data storage architecture, in this embodiment, it is preferable that data query is performed according to a sequence from high to low of the hierarchy of each storage node in the target data storage architecture, and at this time, querying data to be queried of the data query request based on the target data storage architecture includes: and acquiring data from at least one storage node according to the sequence of each storage node in the target data storage architecture to obtain the data to be queried of the data query request.
Illustratively, a server acquires data to be queried from a first storage node (i.e. a root node) of a target data storage architecture, and determines whether missing data to be queried exists in the acquired data to be queried, if missing data exists, the data to be queried is not acquired completely, at this time, the missing data to be queried can be continuously acquired from a next storage node, and the operation of determining whether missing data to be queried exists in the acquired data to be queried is executed in a return manner; if not, the data query request means that all the data to be queried requested to be queried by the data query request are obtained, at this time, a data query result carrying each data to be queried can be generated, and the data query result is sent to the terminal equipment which sends the data query request to the local terminal, so that the data to be queried is displayed to the user through the terminal equipment.
In the data query method provided in this embodiment, when a data query request is received, scene information of the data query request and target architecture configuration information corresponding to the scene information and recorded in a configuration file are obtained, and a target data storage architecture is loaded according to the target architecture configuration information, so that data to be queried of the data query request is queried based on the target data storage architecture. By adopting the technical scheme, the configuration information of the data storage architecture used by the server when the server inquires data in different application scenes is recorded in the configuration file of the server instead of the program code of the server providing the inquiry service, so that the program code of the server providing the data inquiry service can be simplified, and the maintenance difficulty of the program code is reduced. And the storage architecture is configured in a scene in the configuration file, so that different data storage architectures are set for different application scenes, and the expandability of the program code can be enhanced.
Fig. 2 is a schematic flow chart of a data query method provided in an embodiment of the present disclosure, and the scheme in this embodiment may be combined with one or more of the alternatives in the above embodiments. In this embodiment, optionally, the obtaining data from at least one storage node according to the sequence of each storage node in the target data storage architecture to obtain the data to be queried of the data query request includes: determining a root storage node of the target data storage architecture as a current node, and determining data to be queried corresponding to the data request to be queried as current data; determining a data acquisition function of the current node according to the target architecture configuration information; calling the data acquisition function to acquire data and receiving return information of the data acquisition function; if the returned information contains current feedback data, judging whether the current feedback data contains missing data, if so, determining the missing data as the current data, determining a fallback node of the current node as the current node, and returning to execute the operation of determining the data acquisition function of the current node according to the target architecture configuration information; if not, determining the acquired current feedback data as the data to be inquired.
Optionally, the output query method further includes: if the returned information comprises data acquisition abnormity prompting information, judging whether the current node is not in a state of stopping acquiring data and a bottom node exists, if so, determining the bottom node as the current node, and returning to execute the operation of determining the data acquisition function of the current node according to the target architecture configuration information; if not, determining the acquired current feedback data as the data to be inquired.
Optionally, before the loading the target data storage architecture according to the target architecture configuration information, the method further includes: determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.
Optionally, the data query method further includes: and when a configuration updating instruction is received, updating the target architecture configuration information according to the configuration updating instruction.
Correspondingly, as shown in fig. 2, the data query method provided in this embodiment may include:
s201, when a configuration updating instruction is received, updating the target architecture configuration information according to the configuration updating instruction.
Specifically, when monitoring that a developer or an operation and maintenance person triggers an operation of modifying architecture configuration information (such as target architecture configuration information) in a configuration file, the server determines that a configuration update instruction is received, and updates the architecture configuration information in the configuration file based on an input operation of the developer or the operation and maintenance person, such as adding, deleting or modifying the architecture configuration information recorded in the configuration file, thereby realizing the addition, deletion or modification of a data storage architecture corresponding to the server.
In this embodiment, when a server developer and an operation and maintenance worker encounter an emergency (e.g., an access volume surge, a Redis avalanche, MySQL blocking, or the like), the server developer and the operation and maintenance worker may change the configuration information of the architecture recorded in the configuration file, so as to change the corresponding data storage architecture of the server, thereby implementing a thermal update of the data storage architecture of the server, satisfying requirements for fast emergency degradation, current limiting, offloading, and preheating of storage nodes in the data storage architecture, and converging management of the architecture information of the data storage architecture, disaster tolerance, degradation, current limiting, offloading, preheating, source return, and the like of the data storage architecture to one place.
For example, it is assumed that a data storage structure of the server is abstracted into a multi-branch tree as shown in fig. 3, the data storage architecture includes four storage nodes, i.e., a local cache, a Redis, MySQL and a MongoDB, and MySQL and MongoDB are located at the same level, and it is assumed that the traffic proportion of MySQL is 60% and the traffic proportion of MongoDB is 40% in the level. When a server inquires data, the server firstly acquires the data from a local cache; if the data is not acquired or the acquired data is lost, the data is continuously acquired from the Redis; if the data acquired from Redis still has loss, then according to the flow ratio of the MySQL and the MongoDB, the MySQL or the MongoDB is randomly selected, and the data is continuously acquired from the selected MySQL or MongoDB.
If at a certain moment, MySQL breaks down to cause a large amount of slow SQL, and if the stability of the server and the data storage architecture is reduced if MySQL is continuously accessed, at this moment, the operation and maintenance personnel can update the data storage architecture by changing the architecture configuration information of the data storage architecture, delete MySQL in the data storage architecture, and process subsequently received data query requests based on the updated data storage architecture, and a multi-branch tree abstracted from the updated data storage architecture is shown in fig. 4.
If the MySQL is recovered after a period of time, but the MySQL cannot bear 60% of flow, the operation and maintenance personnel can add the MySQL to the hierarchy to which the MongoDB belongs in the data storage architecture again, set the flow ratio of the MySQL to a proper value and correspondingly modify the flow ratio of the MongoDB; MySQL can also be used as the bottom pocket node, for example, MySQL can be added to the next level of MongoDB through configuration, and the data storage architecture after MySQL is added is abstracted into a multi-branch tree as shown in FIG. 5.
S202, when a data query request is received, scene information of the data query request and target architecture configuration information corresponding to the scene information are obtained, and the target architecture configuration information is recorded in a configuration file.
S203, determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, where the original data storage architecture corresponds to the scene information.
Correspondingly, if the original configuration information of the original data storage architecture loaded in the memory is the same as the configuration information of the target architecture, the original data storage architecture can be determined as the target data storage architecture, and the data to be queried in the data query request can be queried based on the target data storage architecture.
The memory can be understood as a memory space of the server, that is, an internal memory of the computer device bearing the server; the original architecture configuration information may be understood as a data storage architecture corresponding to an application scenario of the data query request loaded before the current time.
In this embodiment, the server reloads the target data storage architecture only when the original data storage architecture corresponding to the application scenario of the data query request is not loaded in the memory (i.e., the original data storage architecture corresponding to the scenario information of the data query request is not loaded) or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information recorded in the configuration file; when the original data storage architecture is loaded in the memory and the original architecture configuration information of the original data storage architecture is the same as the target architecture configuration information recorded in the configuration file, the operation of loading the target data storage architecture is not executed any more, but data is directly inquired based on the original data storage architecture, so that the operation required to be executed during data inquiry is further simplified, and the response speed of the server is improved.
Specifically, after acquiring scene information and target architecture configuration information of a data query request, a server judges whether an original data storage architecture corresponding to the scene information exists in a memory, and if not, loads a target data storage architecture according to the target architecture configuration information; if the configuration information of the original data storage architecture is the same as the configuration information of the target architecture, directly determining the original data storage architecture as the target data storage architecture; if not, loading a target data storage framework according to the target framework configuration information, and deleting the original data storage framework from the memory after all original data query requests which are received before the data query requests and are subjected to data query based on the original data storage framework are processed, so as to avoid occupation of useless data storage frameworks on the memory.
It is understood that, the embodiment may also determine whether the original architecture configuration information of the original storage architecture is the same as the target architecture configuration information directly according to the loading time of the original storage architecture and the updating time of the target architecture configuration information without acquiring the original architecture configuration information of the original data storage architecture, for example, the server may record the update time of the target architecture configuration information when updating the target architecture configuration information, and recording the loading time of the original data storage architecture when the original data storage architecture is loaded, thereby, after determining that the original data storage structure corresponding to the scene information of the data query request is loaded in the memory, it may be determined whether the update time of the target architecture configuration information is later than the load time of the original data storage architecture, and if so, determining that the original architecture configuration information of the original data storage architecture is different from the target architecture configuration information; if not, determining that the original architecture configuration information of the original data storage architecture is the same as the target architecture configuration information.
And S204, loading a target data storage framework according to the target framework configuration information.
S205, determining the root storage node of the target data storage architecture as a current node, and determining the data to be queried corresponding to the data request to be queried as current data.
The current node can be understood as a storage node through which data needs to be acquired currently; the current data may be understood as data that needs to be acquired in the current node.
S206, determining a data acquisition function of the current node according to the target architecture configuration information.
In this embodiment, the target architecture configuration information includes configuration information of each storage node in the target storage architecture, that is, node configuration information. The node configuration information of a certain storage node may include: one or more of an object name of a data access object of the storage node, a fallback node of the storage node, a bottom pocket node of the storage node, and a traffic proportion of the storage node in the hierarchy. The data access object is an interface object for accessing a corresponding storage node (such as a database), and object information of the data access object can be set in advance by a developer and stored in a server or the corresponding storage node, and may include an object name and a data acquisition function of the data object, and may further include a pointer type and a data insertion function of the data access object.
The object name of the data access object is the unique identification of the data access object when the server carries out data query. The fallback node of the storage node can be understood as a storage node which further acquires the missing data when the data acquired by the server in the storage node is incomplete (i.e. there is a miss), i.e. when the data acquired in the storage node stores a miss, the server further acquires the missing data from the fallback node of the storage node. The bottom-of-pocket node of a storage node may be understood as a storage node for further acquiring data when the server acquires data in the storage node and an exception (e.g., timeout, block, etc.) occurs, that is, when the data acquired in the storage node is an exception, the server further acquires data from the bottom-of-pocket node of the storage node, and the bottom-of-pocket node and the fall-back node of the storage node may be the same node or different nodes. The flow ratio of a storage node in the hierarchy to which the storage node belongs can be understood as the ratio of data query requests distributed by the server to acquire data from the storage node among a plurality of data query requests required to acquire data between the hierarchies to which the storage node belongs. The data acquisition function can be used for acquiring data in batches from the corresponding storage nodes; the pointer type can be used for representing the specific type of the data acquired by the data access object; the data insertion function may be used to insert data into the corresponding storage node.
Specifically, after determining the current node, the server may determine the data access object of the current node according to the object name of the data access object of the current node recorded in the node configuration information of the current node, and further determine the data acquisition function of the data access object recorded in the object information of the data access object of the current node, as the data acquisition function of the current node when processing the data query request.
In one embodiment, in order to facilitate control of each storage node, each storage node may be set as a normal node, a compound node, or a virtual node according to the number of data access objects of each storage node and whether each storage node has a data reading function. The common node is a storage node which has only one data access object (for example, when the storage node is only deployed on one computer device and does not have a standby node) and has a data reading function; the composite node is a storage node which has a plurality of data access objects (such as when the storage node is deployed on a plurality of computer devices or has a standby node) and has a data reading function; the virtual node is a storage node without data access function. Therefore, the node configuration information of the storage node may further record object group information of a data access object group of the storage node, and when the storage node is a composite node, the object group information records an object name and a traffic ratio of each data access object of the storage node. When a certain storage node cannot normally provide a data reading function, a developer or an operation and maintenance person of the server may set the storage node as a virtual node, that is, change the type information of the storage node recorded in the node configuration information of the storage node into a virtual node, and at this time, before determining the data obtaining function of the current node according to the target architecture configuration information, the method further includes: determining that the current node is a non-virtual node; correspondingly, the method further comprises the following steps: and if the current node is a virtual node, determining the fallback node of the current node as the current node, and judging whether the current node is a virtual node or not until the current node is a non-virtual node.
For example, after determining the current node, the server first determines whether the node is a virtual node according to the node configuration information of the current node. If the current node is a virtual node, further judging whether the current node has a fallback node according to the node configuration information of the current node, if so, determining the fallback node of the current node as the current node, and returning to execute the operation of judging whether the node is a virtual node according to the node configuration information of the current node; and if no fallback node exists, determining the obtained current feedback data as the data to be inquired. If the current node is not a virtual node, further judging whether the current node is a composite node according to the node configuration information of the current node, if the current node is the composite node, randomly selecting a data object according to the traffic proportion of each data access object of the current node recorded in the object group information of the current node, and calling a data acquisition function of the data object to acquire data; and if the current node is not the composite node, directly calling a data acquisition function of the data object of the current node to acquire data.
In one embodiment, the node configuration information of the storage node may further record bottom information of the storage node, and the bottom information indicates whether the server directly skips the storage node to obtain data from the bottom node of the storage node, that is, when the bottom information of the storage node is not empty, the server directly skips the storage node to obtain data from the bottom node of the storage node, thereby implementing thermal degradation of the data storage architecture in an emergency. At this time, for example, after determining the current node or after determining that the current node is not a virtual node, the server may determine whether the bottom information in the node configuration information of the current node is empty, and if so, perform subsequent operations; if not, determining the bottom pocket node of the current node as the current node, and returning to execute the operation of judging whether the bottom pocket information in the node configuration information of the current node is empty.
S207, calling the data acquisition function to acquire data, and receiving the return information of the data acquisition function.
Specifically, the server sends the data ID of the current data to the current node; correspondingly, the current node executes a data acquisition function of the current node according to the data ID of the current data to acquire the data, generates return information containing the acquired current feedback data when the data acquisition process is not abnormal, and sends the return information to the server; and when the data acquisition is abnormal, generating return information containing abnormal prompt information for prompting the server that the data acquisition process is abnormal, and returning the return information to the server.
In an embodiment, the data query method provided in this embodiment may further include: and if the current node is provided with a preheating identifier, determining a target storage node corresponding to the preheating identifier, and controlling the target storage node to preheat.
In the foregoing embodiment, if the current node is provided with the warming identifier, when the server controls the current node to obtain data, the server may also concurrently control a node (i.e., a target storage node) that needs to be warmed in the target data storage architecture to warm up, for example, execute a data obtaining function of a data access object of the target storage node, and store data obtained by the data obtaining function in a cache. Whether the current node is provided with the preheating identifier and the node which needs to be preheated can be determined according to the node configuration information of the current node.
At this time, the node configuration information of the storage node in the target data storage architecture may further include the warming-up information of the storage node, so as to achieve warming-up of the target storage node. The preheating information can be set by a server developer or an operation and maintenance person according to needs. When the node name of the target storage node is recorded in the preheating information in the node configuration information of a certain storage node, it indicates that the storage node is provided with the preheating identifier, and the target storage node needs to be controlled to preheat while the storage node is controlled to acquire data.
S208, judging whether the return information contains current feedback data, if so, executing S209; if not, S211 is executed.
The current feedback data may be understood as data actually obtained by the current node according to the data ID of the current data sent by the server.
Specifically, after receiving the return information of the data acquisition function of the current node, the server may determine whether a data field in the return information, which is used for recording the acquired current feedback data, is not empty, and if so, determine that the return information includes the current feedback data; if not, determining that the returned information does not contain the current feedback data.
In this embodiment, when the return information includes the current feedback data and the abnormal prompt information at the same time, it means that an error occurs in the process of acquiring the current feedback data by the current node, and the acquired current feedback data may be inaccurate at this time. Therefore, after determining that the current feedback data is included in the return information, it may be further determined whether the return information includes the abnormal prompt information, and when the return information includes the abnormal prompt information, S211 may be executed, and when the return information does not include the abnormal prompt information, S209 may be executed; after receiving the return information of the data acquisition function of the current node, it may be directly determined whether the return information includes the abnormal prompt information, if so, S211 may be executed, and if not, S209 may be executed.
S209, judging whether missing data exists in the current feedback data, if so, executing S210; if not, S213 is executed.
Specifically, the server may compare the data ID of each current feedback data recorded in the return information with the data ID of the current data that the server needs to acquire in the current node, and determine whether the data ID of the current feedback data corresponds to the data ID of the current data one to one, and if yes, determine that the current feedback data does not have missing data; if not, determining that the current feedback data has missing data.
In one embodiment, after missing data of a certain storage node is acquired from other storage nodes, if the storage node sets a data back source, a data insertion function of the storage node may be called to backfill the acquired missing data into the storage node. For example, when the current node is not abnormal in the data acquisition process, or when the current node is not abnormal in the data acquisition process and the current feedback data of the current node does not have missing data, it may be determined whether a previous node using the current node as a fallback node has set a fallback, and if the previous node has set a data fallback identifier, the missing data is sent to the previous node for storage; if not, the previous node is not executed with the data back-to-source operation. At this time, preferably, the data query method provided in this embodiment may further include: and if the current node is a fallback node of the previous node and the previous node is provided with a data source returning identifier, sending the current feedback data to the previous node for storage, wherein the previous node is a storage node which is determined to be the current node in the target data storage architecture.
The node configuration information of the storage node in the target data storage architecture may further include source returning information of the storage node, where the source returning information may be set by a server developer or an operation and maintenance person as needed, and when source returning information in node configuration information of a certain storage node is "yes," it means that the storage node is provided with a data source returning identifier, and at this time, when missing data of current feedback data of the storage node is acquired, the acquired missing data needs to be stored in the storage node subsequently.
It should be noted that, when returning to the source of a certain storage node, it may be directly sent the current feedback data of the fallback node to the storage node for storage, without considering whether the missing data is still stored in the current feedback data of the fallback node of the storage node. Whether missing data is still stored in the current feedback data of the fallback node of the storage node or not can be considered, when missing data still exists in the current feedback data of the fallback node, the fallback node is not executed to perform the back source operation temporarily, data are continuously obtained from the fallback node of the fallback node, and the like until complete missing data are obtained from other nodes, and after the complete missing data are obtained, the current feedback data and the complete missing data are stored in the storage node; when the current feedback data of the falling node does not have missing data, the current feedback data can be directly stored in the storage node.
S210, determining the missing data as current data, determining a fallback node of the current node as the current node, and returning to execute S206.
Correspondingly, if the current node does not have a fallback node, the obtained current feedback data is determined as the data to be queried.
Specifically, when determining that missing data exists in current feedback data of a current node, the server may determine whether a fallback node exists in the current node according to node configuration information of the current node, and if yes, determine the fallback node as the current node, and determine the missing data as current data that needs to be queried by the current node; if not, directly determining the current feedback data returned by each storage node before the current time as the data to be inquired.
S211, judging whether the current node is not in a state of stopping acquiring data and has a bottom pocket node, if so, executing S212, and if not, executing S213.
Specifically, when determining that an abnormality occurs in the process of acquiring data by the current node, the server determines whether the current node is in a state of stopping acquiring data, and if yes, directly executes S213; if not, further determining whether the current node has a bottom node, and if so, determining whether the node configuration information of the current node has a bottom node of the current node recorded therein, if so, executing S212, otherwise, executing S213.
The node configuration information of the storage node in the target data storage architecture may further include fast failure information of the storage node, and the fast failure information may be set by a server developer or an operation and maintenance worker as needed when an emergency occurs, so as to implement thermal degradation in the emergency. When the fast failure information in the node configuration information of a certain storage node is "yes", it means that the storage node is in a state of stopping acquiring data, and at this time, when the storage node is abnormal when acquiring data, even if the storage node has a bottom node, the storage node is not accessed, but the operation of acquiring data to be queried is stopped.
In one embodiment, the node configuration information of the storage node in the target data storage architecture may further include fallback, i.e., bottom-of-pocket information of the storage node, where the fallback, i.e., bottom-of-pocket information may be set by a server developer or an operation and maintenance worker as needed when an emergency occurs, so as to implement thermal degradation in the emergency. When the fallback, i.e., the bottom entry information in the node configuration information of a certain storage node is "yes," this indicates that the fallback node of the storage node can be used as the bottom entry node when the storage node does not have a bottom entry node. At this time, correspondingly, when it is determined that the current node does not have a bottom node, it may be further determined whether the current node has a fallback node, if so, the fallback node is determined as the current node, and the step returns to execute S206; if not, S213 is executed.
And S212, determining the bottom pocket node as the current node, and returning to execute the S206.
And S213, determining the acquired current feedback data as the data to be inquired.
Specifically, the current feedback data included in the acquired return information of each storage node is used as the data to be queried, a query result carrying the data to be queried is generated, and the query result is sent to the terminal device sending the data query request to the home terminal. It can be understood that, when the return information of each storage node does not include the current feedback data, the server may generate an inquiry result not carrying the data to be inquired, and send the inquiry result to the terminal device that sends the data inquiry request to the home terminal, so as to notify the user at the terminal device side that the data to be inquired is not obtained.
Therefore, by setting the fallback nodes and the bottom pocket nodes for the nodes in the data storage structure, the configuration information of the data storage structure is recorded through the configuration file, and the configuration information comprises the node names of the fallback nodes and the node names of the bottom pocket nodes of all storage nodes in the data storage structure and the data state information of stopping obtaining; when an updating instruction is received, updating the one or more items of configuration information recorded in the configuration file; data are inquired based on the updated data storage architecture, so that the program codes of the inquiry service provided by the server can be simplified, and the maintenance difficulty of the program codes is reduced; the data storage system can also realize the functions of hot updating of the data storage architecture, disaster tolerance and degradation of the data storage architecture and the like, can converge the schemes of the data storage architecture, the disaster tolerance and the degradation of the data storage architecture and the like at one place for management, and reduces the management difficulty of the data storage architecture.
Fig. 6 is a block diagram of a data query device according to the disclosed embodiment. The apparatus may be implemented by software and/or hardware, may be configured in an electronic device, and typically may be configured in a server, and may query data by performing a data query method. As shown in fig. 6, the data query apparatus provided by the embodiment of the present disclosure may include an information acquisition module 301, a framework loading module 302, and a data query module 303, wherein,
an information obtaining module 301, configured to, when a data query request is received, obtain scene information of the data query request and target architecture configuration information corresponding to the scene information, where the target architecture configuration information is recorded in a configuration file;
a framework loading module 302, configured to load a target data storage framework according to the target framework configuration information;
a data query module 303, configured to query the data to be queried of the data query request based on the target data storage architecture.
In the data query apparatus provided in this embodiment, when the information obtaining module receives the data query request, the scene information of the data query request and the target architecture configuration information corresponding to the scene information, which is recorded in the configuration file, are obtained, the target data storage architecture is loaded by the architecture loading module according to the target architecture configuration information, and then the data query module queries the data to be queried of the data query request based on the target data storage architecture. By adopting the technical scheme, the configuration information of the data storage architecture used by the server when the server inquires data in different application scenes is recorded in the configuration file of the server instead of the program code of the inquiry service provided by the server, so that the program code of the inquiry service provided by the server can be simplified, and the maintenance difficulty of the program code is reduced. And the storage architecture is configured in a scene in the configuration file, so that different data storage architectures are set for different application scenes, and the expandability of the program code can be enhanced.
In the foregoing solution, the data query module 303 may be specifically configured to: and acquiring data from at least one storage node according to the sequence of each storage node in the target data storage architecture to obtain the data to be queried of the data query request.
In the above solution, the data query module 303 may include: a node determining unit, configured to determine a root storage node of the target data storage architecture as a current node, and determine data to be queried corresponding to the data request to be queried as current data; a function determining unit, configured to determine a data obtaining function of the current node according to the target architecture configuration information; the information receiving unit is used for calling the data acquisition function to acquire data and receiving the return information of the data acquisition function; the missing judging unit is used for judging whether the missing data exists in the current feedback data when the return information contains the current feedback data, if so, determining the missing data as the current data, determining a fallback node of the current node as the current node, and returning to the function determining unit; if not, determining the acquired current feedback data as the data to be inquired.
In the foregoing solution, the data query module 303 may further include: a bottom node determining unit, configured to determine whether the current node is not in a state of stopping data acquisition and a bottom node exists if the return information includes data acquisition abnormality prompt information, and if so, determine the bottom node as the current node and return to the function determining unit; if not, determining the acquired current feedback data as the data to be inquired.
In the foregoing solution, the data query module 303 may further include: and the data backfilling unit is used for sending the current feedback data to the previous node for storage if the current node is a fallback node of the previous node and the previous node is provided with a data fallback identifier, wherein the previous node is a storage node which is determined to be the current node in the target data storage architecture.
In the foregoing solution, the data query module 303 may further include: and the node preheating unit is used for determining a target storage node corresponding to the preheating identifier and controlling the target storage node to preheat if the current node is provided with the preheating identifier.
In the foregoing solution, the data query module 303 may further include: a virtual node judging unit, configured to determine that the current node is a non-virtual node before determining the data obtaining function of the current node according to the target architecture configuration information; correspondingly, the virtual node determining unit is further configured to determine, when the current node is a virtual node, a fallback node of the current node as the current node, and determine whether the current node is a virtual node until the current node is a non-virtual node.
In the foregoing solution, the data query module 303 may further include: and the data determining unit is used for determining the acquired current feedback data as the data to be inquired when the fallback node does not exist in the current node.
Optionally, the data query apparatus provided in this embodiment may further include: and the architecture determination module is used for determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information before the target data storage architecture is loaded according to the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.
Optionally, the data query apparatus provided in this embodiment may further include: and the information updating module is used for updating the target architecture configuration information according to the configuration updating instruction when the configuration updating instruction is received.
The data query device provided by the embodiment of the disclosure can execute the data query method provided by any embodiment of the disclosure, and has the corresponding functional module and beneficial effect of executing the data query method. For technical details that are not described in detail in this embodiment, reference may be made to a data query method provided in any embodiment of the present disclosure.
Referring now to FIG. 7, a block diagram of an electronic device (e.g., server) 700 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 7, the electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage means 706 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Generally, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, and the like; storage 706 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 706, or may be installed from the ROM 702. The computer program, when executed by the processing device 701, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: when a data query request is received, acquiring scene information of the data query request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file; loading a target data storage architecture according to the target architecture configuration information; and querying the data to be queried of the data query request based on the target data storage architecture.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the names of the modules do not in some cases constitute a limitation of the unit itself.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Example 1 provides, in accordance with one or more embodiments of the present disclosure, a data query method including:
when a data query request is received, acquiring scene information of the data query request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file;
loading a target data storage architecture according to the target architecture configuration information;
and querying the data to be queried of the data query request based on the target data storage architecture.
Example 2 the method of example 1, the querying the data to be queried of the data query request based on the target data storage architecture, comprising, in accordance with one or more embodiments of the present disclosure:
and acquiring data from at least one storage node according to the sequence of each storage node in the target data storage architecture to obtain the data to be queried of the data query request.
Example 3 the method according to example 2, wherein the obtaining data from at least one storage node according to an order of the storage nodes in the target data storage architecture to obtain the data to be queried of the data query request includes:
determining a root storage node of the target data storage architecture as a current node, and determining data to be queried corresponding to the data request to be queried as current data;
determining a data acquisition function of the current node according to the target architecture configuration information;
calling the data acquisition function to acquire data and receiving return information of the data acquisition function;
if the returned information contains current feedback data, judging whether the current feedback data contains missing data, if so, determining the missing data as the current data, determining a fallback node of the current node as the current node, and returning to execute the operation of determining the data acquisition function of the current node according to the target architecture configuration information; if not, determining the acquired current feedback data as the data to be inquired.
Example 4 the method of example 3, in accordance with one or more embodiments of the present disclosure, further comprising:
if the return information contains data acquisition abnormity prompting information, judging whether the current node is not in a state of stopping acquiring data and a bottom node exists, if so, determining the bottom node as the current node, and returning to execute the operation of determining the data acquisition function of the current node according to the target architecture configuration information; if not, determining the acquired current feedback data as the data to be inquired.
Example 5 the method of example 3, in accordance with one or more embodiments of the present disclosure, further comprising:
and if the current node is a fallback node of the previous node and the previous node is provided with a data source returning identifier, sending the current feedback data to the previous node for storage, wherein the previous node is a storage node which is determined to be the current node in the target data storage architecture.
Example 6 the method of example 3, in accordance with one or more embodiments of the present disclosure, further comprising:
and if the current node is provided with a preheating identifier, determining a target storage node corresponding to the preheating identifier, and controlling the target storage node to preheat.
Example 7 the method of example 3, prior to the determining the data acquisition function for the current node from the target architecture configuration information, according to one or more embodiments of the present disclosure, further comprising:
determining that the current node is a non-virtual node;
correspondingly, the method further comprises the following steps:
and if the current node is a virtual node, determining the fallback node of the current node as the current node, and judging whether the current node is a virtual node or not until the current node is a non-virtual node.
Example 8 the method of any of examples 3-7, in accordance with one or more embodiments of the present disclosure, further comprising:
and if the current node does not have a fallback node, determining the obtained current feedback data as the data to be inquired.
Example 9 the method of any of examples 1-7, further comprising, prior to the loading the target data storage architecture according to the target architecture configuration information, in accordance with one or more embodiments of the present disclosure:
determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.
Example 10 the method of example 9, in accordance with one or more embodiments of the present disclosure, further comprising:
and when a configuration updating instruction is received, updating the target architecture configuration information according to the configuration updating instruction.
Example 11 provides, in accordance with one or more embodiments of the present disclosure, a data query apparatus, including:
the information acquisition module is used for acquiring scene information of a data query request and target architecture configuration information corresponding to the scene information when the data query request is received, and the target architecture configuration information is recorded in a configuration file;
the architecture loading module is used for loading a target data storage architecture according to the target architecture configuration information;
and the data query module is used for querying the data to be queried of the data query request based on the target data storage architecture.
Example 12 provides, in accordance with one or more embodiments of the present disclosure, an electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the data query method of any of examples 1-10.
Example 13 provides a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the data query method of any of examples 1-10, in accordance with one or more embodiments of the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (13)
1. A method for querying data, comprising:
when a data query request is received, acquiring scene information of the data query request and target architecture configuration information corresponding to the scene information, wherein the target architecture configuration information is recorded in a configuration file;
loading a target data storage architecture according to the target architecture configuration information;
and querying the data to be queried of the data query request based on the target data storage architecture.
2. The method of claim 1, wherein the querying the data to be queried of the data query request based on the target data storage architecture comprises:
and acquiring data from at least one storage node according to the sequence of each storage node in the target data storage architecture to obtain the data to be queried of the data query request.
3. The method according to claim 2, wherein the obtaining data from at least one of the storage nodes according to the order of the storage nodes in the target data storage architecture to obtain the data to be queried of the data query request comprises:
determining a root storage node of the target data storage architecture as a current node, and determining data to be queried corresponding to the data request to be queried as current data;
determining a data acquisition function of the current node according to the target architecture configuration information;
calling the data acquisition function to acquire data and receiving return information of the data acquisition function;
if the returned information contains current feedback data, judging whether the current feedback data contains missing data, if so, determining the missing data as the current data, determining a fallback node of the current node as the current node, and returning to execute the operation of determining the data acquisition function of the current node according to the target architecture configuration information; if not, determining the acquired current feedback data as the data to be inquired.
4. The method of claim 3, further comprising:
if the return information contains data acquisition abnormity prompting information, judging whether the current node is not in a state of stopping acquiring data and a bottom node exists, if so, determining the bottom node as the current node, and returning to execute the operation of determining the data acquisition function of the current node according to the target architecture configuration information; if not, determining the acquired current feedback data as the data to be inquired.
5. The method of claim 3, further comprising:
and if the current node is a fallback node of the previous node and the previous node is provided with a data source returning identifier, sending the current feedback data to the previous node for storage, wherein the previous node is a storage node which is determined to be the current node in the target data storage architecture.
6. The method of claim 3, further comprising:
and if the current node is provided with a preheating identifier, determining a target storage node corresponding to the preheating identifier, and controlling the target storage node to preheat.
7. The method of claim 3, further comprising, prior to said determining a data acquisition function for the current node based on the target architecture configuration information:
determining that the current node is a non-virtual node;
correspondingly, the method further comprises the following steps:
and if the current node is a virtual node, determining the fallback node of the current node as the current node, and judging whether the current node is a virtual node or not until the current node is a non-virtual node.
8. The method of any of claims 3-7, further comprising:
and if the current node does not have a fallback node, determining the obtained current feedback data as the data to be inquired.
9. The method of any of claims 1-7, further comprising, prior to said loading a target data storage architecture according to said target architecture configuration information:
determining that the original data storage architecture is not loaded in the memory or the original architecture configuration information of the original data storage architecture loaded in the memory is different from the target architecture configuration information, wherein the original data storage architecture corresponds to the scene information.
10. The method of claim 9, further comprising:
and when a configuration updating instruction is received, updating the target architecture configuration information according to the configuration updating instruction.
11. A data query apparatus, comprising:
the information acquisition module is used for acquiring scene information of a data query request and target architecture configuration information corresponding to the scene information when the data query request is received, and the target architecture configuration information is recorded in a configuration file;
the architecture loading module is used for loading a target data storage architecture according to the target architecture configuration information;
and the data query module is used for querying the data to be queried of the data query request based on the target data storage architecture.
12. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a data query method as claimed in any one of claims 1-10.
13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a data query method as claimed in any one of claims 1 to 10.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010843817.0A CN114077619B (en) | 2020-08-20 | 2020-08-20 | Data query method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010843817.0A CN114077619B (en) | 2020-08-20 | 2020-08-20 | Data query method, device, electronic equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114077619A true CN114077619A (en) | 2022-02-22 |
CN114077619B CN114077619B (en) | 2024-08-02 |
Family
ID=80281863
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010843817.0A Active CN114077619B (en) | 2020-08-20 | 2020-08-20 | Data query method, device, electronic equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114077619B (en) |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6058394A (en) * | 1997-08-29 | 2000-05-02 | International Business Machines Corporation | Manager server selects an agent server to execute query based on availability of the server connections to data source and target |
US6161125A (en) * | 1998-05-14 | 2000-12-12 | Sun Microsystems, Inc. | Generic schema for storing configuration information on a client computer |
US20150095341A1 (en) * | 2013-09-27 | 2015-04-02 | International Business Machines Corporation | System and a method for hierarchical data column storage and efficient query processing |
US20150254126A1 (en) * | 2014-03-07 | 2015-09-10 | Conrad N. Wood | Systems and Methods for Storage of Data in a Virtual Storage Device |
CN107533518A (en) * | 2015-01-20 | 2018-01-02 | 乌尔特拉塔有限责任公司 | Distributed index for fault tolerant object memory construction |
CN107766355A (en) * | 2016-08-17 | 2018-03-06 | 腾讯科技(深圳)有限公司 | Hierarchical data management method, hierarchical data management system and instantaneous communication system |
CN108388604A (en) * | 2018-02-06 | 2018-08-10 | 平安科技(深圳)有限公司 | User right data administrator, method and computer readable storage medium |
US20180285002A1 (en) * | 2016-05-11 | 2018-10-04 | Hitachi, Ltd. | Data storage system, process and computer program for such data storage system for reducing read and write amplifications |
CN109062936A (en) * | 2018-06-15 | 2018-12-21 | 中国平安人寿保险股份有限公司 | A kind of data query method, computer readable storage medium and terminal device |
CN109299348A (en) * | 2018-11-28 | 2019-02-01 | 北京字节跳动网络技术有限公司 | A kind of data query method, apparatus, electronic equipment and storage medium |
US10210167B1 (en) * | 2012-05-07 | 2019-02-19 | Amazon Technologies, Inc. | Multi-level page caching for distributed object store |
CN109902105A (en) * | 2019-02-28 | 2019-06-18 | 携程旅游网络技术(上海)有限公司 | For the data query system of micro services framework, method, equipment and storage medium |
CN110765295A (en) * | 2019-09-06 | 2020-02-07 | 中国平安财产保险股份有限公司 | Graph database-based query method and device, computer equipment and storage medium |
CN111125169A (en) * | 2019-11-28 | 2020-05-08 | 江苏苏宁物流有限公司 | Data query method, control device, medium and computer equipment |
CN111309785A (en) * | 2020-02-14 | 2020-06-19 | 广州极晟网络技术有限公司 | Spring framework-based database access method and device, computer equipment and medium |
-
2020
- 2020-08-20 CN CN202010843817.0A patent/CN114077619B/en active Active
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6058394A (en) * | 1997-08-29 | 2000-05-02 | International Business Machines Corporation | Manager server selects an agent server to execute query based on availability of the server connections to data source and target |
US6161125A (en) * | 1998-05-14 | 2000-12-12 | Sun Microsystems, Inc. | Generic schema for storing configuration information on a client computer |
US10210167B1 (en) * | 2012-05-07 | 2019-02-19 | Amazon Technologies, Inc. | Multi-level page caching for distributed object store |
US20150095341A1 (en) * | 2013-09-27 | 2015-04-02 | International Business Machines Corporation | System and a method for hierarchical data column storage and efficient query processing |
US20150254126A1 (en) * | 2014-03-07 | 2015-09-10 | Conrad N. Wood | Systems and Methods for Storage of Data in a Virtual Storage Device |
CN107533518A (en) * | 2015-01-20 | 2018-01-02 | 乌尔特拉塔有限责任公司 | Distributed index for fault tolerant object memory construction |
US20180285002A1 (en) * | 2016-05-11 | 2018-10-04 | Hitachi, Ltd. | Data storage system, process and computer program for such data storage system for reducing read and write amplifications |
CN107766355A (en) * | 2016-08-17 | 2018-03-06 | 腾讯科技(深圳)有限公司 | Hierarchical data management method, hierarchical data management system and instantaneous communication system |
CN108388604A (en) * | 2018-02-06 | 2018-08-10 | 平安科技(深圳)有限公司 | User right data administrator, method and computer readable storage medium |
CN109062936A (en) * | 2018-06-15 | 2018-12-21 | 中国平安人寿保险股份有限公司 | A kind of data query method, computer readable storage medium and terminal device |
CN109299348A (en) * | 2018-11-28 | 2019-02-01 | 北京字节跳动网络技术有限公司 | A kind of data query method, apparatus, electronic equipment and storage medium |
CN109902105A (en) * | 2019-02-28 | 2019-06-18 | 携程旅游网络技术(上海)有限公司 | For the data query system of micro services framework, method, equipment and storage medium |
CN110765295A (en) * | 2019-09-06 | 2020-02-07 | 中国平安财产保险股份有限公司 | Graph database-based query method and device, computer equipment and storage medium |
CN111125169A (en) * | 2019-11-28 | 2020-05-08 | 江苏苏宁物流有限公司 | Data query method, control device, medium and computer equipment |
CN111309785A (en) * | 2020-02-14 | 2020-06-19 | 广州极晟网络技术有限公司 | Spring framework-based database access method and device, computer equipment and medium |
Also Published As
Publication number | Publication date |
---|---|
CN114077619B (en) | 2024-08-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR101871383B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
CN109492013B (en) | Data processing method, device and system applied to database cluster | |
CN107357896A (en) | Expansion method, device, system and the data base cluster system of data-base cluster | |
CN110888847B (en) | Recycle bin system and file recycling method | |
CN114741335B (en) | Cache management method, device, medium and equipment | |
CN111652685A (en) | Information processing method, apparatus, computer equipment, and computer-readable storage medium | |
CN111291421A (en) | Block chain data authorization method, electronic device and computer readable storage medium | |
CN110674082B (en) | Method and device for removing online document, electronic equipment and computer readable medium | |
CN110633324B (en) | Method, apparatus, electronic device and computer readable medium for synchronizing data | |
CN118210829A (en) | Data processing method and device | |
CN114077619B (en) | Data query method, device, electronic equipment and storage medium | |
CN111090530A (en) | Distributed cross-interprocess communication bus system | |
CN113553206B (en) | Data event execution method and device, electronic equipment and computer readable medium | |
CN114077639B (en) | Data writing method, device, electronic equipment and storage medium | |
CN113760929A (en) | Data synchronization method and device, electronic equipment and computer readable medium | |
CN114077587A (en) | Rule engine based business processing method, rule engine, medium and device | |
CN113342837B (en) | Data transmission method, device, electronic equipment and computer readable medium | |
CN116938961B (en) | System and method for data synchronization and data reading method | |
CN112181975A (en) | Method and apparatus for creating a database in a data warehouse | |
CN115174599B (en) | Business archiving node creation and data processing method, device, equipment and medium | |
CN114268540B (en) | Rule engine optimization method, device and equipment | |
CN115686378B (en) | Data storage method, device, electronic device and storage medium | |
CN110262756B (en) | Method and device for caching data | |
CN114329563A (en) | Database processing method, device, equipment and medium | |
CN119311669A (en) | A task processing method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |