CN113761016B

CN113761016B - Data query method, device, equipment and storage medium

Info

Publication number: CN113761016B
Application number: CN202110053164.0A
Authority: CN
Inventors: 鲁一锋
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2021-01-15
Filing date: 2021-01-15
Publication date: 2025-03-18
Anticipated expiration: 2041-01-15
Also published as: CN113761016A

Abstract

The embodiment of the present application provides a data query method, device, equipment and storage medium, which obtains a user's data query request, which includes: index constraint parameters including a target database identifier and a field value. Since the search engine stores data information of the target database stored according to the index constraint parameters, according to the above index constraint parameters, a data query is performed in the search engine to determine the target data at the target query field value in the target database corresponding to the target database identifier, and finally outputs the target data corresponding to the data query request. In this technical solution, the data in the database is consistent with the data in the search engine, thereby improving the retrieval efficiency when the search function of the search engine is used to perform data query.

Description

Data query method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of big data, in particular to a data query method, a device, equipment and a storage medium.

Background

With rapid development of network technology, internet applications based on Web environments are becoming more and more widespread, and various applications are erected on a Web platform in the process of enterprise informatization, so that development requirements of a large number of Web application systems are met, and data of the Web application systems need to be stored through a database on the Web platform.

In the prior art, when a database on a Web platform stores data of a Web application system, the data storage is generally completed through a large number of forms. Thus, when a data query is subsequently performed, a large number of query interfaces need to be invoked to perform accurate and fuzzy queries of data in a form of a database based on various conditions.

In the process of realizing the invention, the inventor finds that at least the following problems exist in the prior art, namely when inquiring in a form of a database, particularly the fuzzy inquiry of multiple fields aiming at large data volume, the problem of low data inquiry efficiency exists due to the large data volume and the relatively large number of fields in fuzzy matching.

Disclosure of Invention

The embodiment of the application provides a data query method, a device, equipment and a storage medium, which are used for solving the problem of low data query efficiency in the existing database.

According to a first aspect of the present application, an embodiment of the present application provides a data query method, including:

Acquiring a data query request of a user, wherein the data query request comprises index constraint parameters comprising a target database identifier and a target query field value;

according to the index constraint parameters, carrying out data query in a search engine, determining target data of a target database identifier corresponding to a target query field value in a target database, wherein the search engine stores data information of the target database stored according to the index constraint parameters;

And outputting target data corresponding to the data query request.

In one possible design of the first aspect, the index constraint parameter further comprises a target lookup data table identification;

And according to the index constraint parameter, performing data query in a search engine to determine target data at a target query field value in the target database corresponding to the target database identifier, wherein the method comprises the following steps:

positioning a data table set corresponding to the target database in the search engine according to the target database identifier;

positioning a target query data table in a data table set corresponding to the target database according to the target query data table identifier;

and inquiring in the target inquiry data table to determine target data at the target inquiry field value.

Optionally, the index constraint parameter further comprises a target query field identifier;

The querying in the target query data table, determining the target data at the target query field value, includes:

inquiring in the target inquiry data table according to the target inquiry field identification, and positioning to a target inquiry field;

And inquiring in the target inquiry field to determine target data corresponding to the target inquiry field value.

In another possible design of the first aspect, before the obtaining the data query request of the user, the method further includes:

acquiring a data processing request aiming at the target database, wherein the data processing request comprises data to be processed and the index constraint parameters;

Inserting the data to be processed into the target database based on the pre-configured database table structure and the index constraint parameter, and generating a database log;

Transmitting the database log to a message service module through a message middleware;

Processing the database log by using the message service module to obtain data updating information of the target database;

and updating the data information of the target database in the search engine based on the data updating information.

Optionally, the inserting the data to be processed into the target database based on the preconfigured database table structure and the index constraint parameter, to generate a database log, includes:

Determining a target processing data table and a target processing field in the target processing data table according to the index constraint parameter;

and dynamically storing the data to be processed into the target database based on the configuration information of the database table structure, and generating the database log.

Optionally, the processing the database log by using the message service module to obtain data update information of the target database includes:

Converting the database log into target processing data represented by an object by using the message service module;

according to the message type of the database log, packaging the target processing data into an event corresponding to the message type;

Determining an event processing unit corresponding to the event according to the type of the event;

And processing the target processing data by using an event processing unit corresponding to the event to obtain the data updating information of the target database.

According to a second aspect of the present application, an embodiment of the present application provides a data query apparatus, including:

The acquisition module is used for acquiring a data query request of a user, wherein the data query request comprises index constraint parameters comprising a target database identifier and a target query field value;

The processing module is used for carrying out data query in a search engine according to the index constraint parameters, determining target data at a target query field value in the target database corresponding to the target database identifier, and storing data information of the target database stored according to the index constraint parameters in the search engine;

and the output module is used for outputting the target data corresponding to the data query request.

In one possible design of the second aspect, the index constraint parameter further includes a target lookup data table identifier, and the processing module is specifically configured to:

the processing module is configured to query in the target query data table, and determine target data at the target query field value, where the target data is specifically:

The processing module is specifically configured to query in the target query data table according to the target query field identifier, locate a target query field, query in the target query field, and determine target data corresponding to the target query field value.

In another possible design of the second aspect, the obtaining module is further configured to obtain a data processing request for the target database, where the data processing request includes data to be processed and the index constraint parameter;

the processing module is further configured to:

Optionally, the processing module is configured to insert the data to be processed into the target database based on the preconfigured database table structure and the index constraint parameter, so as to generate a database log, which specifically is:

The processing module is specifically configured to determine a target processing data table and a target processing field in the target processing data table according to the index constraint parameter, dynamically store the data to be processed in the target database based on configuration information of the database table structure, and generate the database log.

Optionally, the processing module is configured to process the database log by using the message service module to obtain data update information of the target database, where the data update information specifically includes:

the processing module is specifically configured to:

According to a third aspect of the present application, an embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor implementing the method according to the first aspect and each possible design described above when executing the computer program.

According to a fourth aspect of the present application, embodiments of the present application provide a computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of the first aspect and each possible design described above.

According to a fifth aspect of the present application, embodiments of the present application provide a computer program product comprising a computer program stored in a readable storage medium, the computer program being readable by at least one processor of an electronic device from the readable storage medium, the at least one processor executing the computer program causing the electronic device to perform the method of the first aspect and each possible design described above.

According to the data query method, the device, the equipment and the storage medium, the data query request of the user is obtained, the data query request comprises index constraint parameters comprising the target database identification and the field value, and because the search engine stores the data information of the target database stored according to the index constraint parameters, the data query is carried out in the search engine according to the index constraint parameters, the target data of the target database identification corresponding to the target query field value in the target database is determined, and finally the target data corresponding to the data query request is output. In the technical scheme, the data in the database is consistent with the data in the search engine, so that the search efficiency is improved when the search function of the search engine is adopted for data query.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1A is a schematic diagram of an application scenario of a data query method according to an embodiment of the present application;

Fig. 1B is a schematic diagram of another application scenario of a data query method according to an embodiment of the present application;

fig. 2 is a schematic flow chart of a first embodiment of a data query method according to an embodiment of the present application;

fig. 3 is a schematic flow chart of a second embodiment of a data query method according to the embodiment of the present application;

fig. 4 is a schematic flow chart of a third embodiment of a data query method according to the embodiment of the present application;

FIG. 5 is a diagram illustrating a database table structure according to an embodiment of the present application;

fig. 6 is a schematic flow chart of a fourth embodiment of a data query method according to the embodiment of the present application;

fig. 7 is a schematic diagram of the structure of a message service module in the present embodiment;

fig. 8 is a schematic structural diagram of an embodiment of a data query device according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an embodiment of an electronic device according to an embodiment of the present application.

Specific embodiments of the present disclosure have been shown by way of the above drawings and will be described in more detail below. These drawings and the written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the disclosed concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

Firstly, explanation is made on nouns according to embodiments of the present application:

the database is a computer software system for organizing, storing and managing data according to a data structure, and is a collection of large data which is stored in a computer for a long time, organized, sharable and uniformly managed. The types of the databases include a relational database and a non-relational database. The storage format of the relational database can intuitively reflect the relation among the entities, and common relational databases comprise Mysql, sqlServer and the like. A non-relational database refers to a distributed, non-relational, data storage system that does not guarantee compliance with the ACID principle, where ACID refers to an abbreviation of four basic elements for the correct execution of a database transaction, including atomicity (Atomicity), consistency (Consistency), isolation (Isolation), persistence (Durability).

In an embodiment of the application, the database includes, but is not limited to, SQLSERVER, MYSQL, ORACLE, REDIS, HBASE or the like storage systems.

The search engine is a search technology for searching out the appointed information from the Internet by applying a specific strategy according to the requirements of the user and a certain algorithm and feeding back the appointed information to the user. The searching mode is one key link of the search engine and can be roughly divided into four types of full-text search engines, meta search engines, vertical search engines and catalog search engines, and the full-text search engines, the meta search engines, the vertical search engines and the catalog search engines have the characteristics and are applicable to different search environments.

The full-text search engine is a search mode for capturing all related articles on the Internet by utilizing a crawler program to index. The computer indexing program scans each word in the article, establishes an index for each word, indicates the number and position of the word in the article, and when the user inquires, the searching program searches according to the index established in advance and feeds the searched result back to the searching mode of the user. General network users are applicable to full text search engines. The searching mode is convenient and simple, and all relevant information is easy to obtain. However, the searched information is too complicated, so that the user needs to browse and screen out the required information one by one.

Currently mainstream full text search engines include, but are not limited to ELASTICSEARCH, SOLR, LUCENE.

Message queue middleware, which is an important component in large-scale systems, has gradually become a core means of communication inside enterprise systems, has a series of functions of loose coupling, asynchronous message, flow peak clipping, reliable delivery, broadcasting, flow control, final consistency and the like, and has become one of the main means of asynchronous remote procedure call (remote procedure call, RPC). Currently, common message queue middleware has ActiveMQ, rabbitMQ, zeroMQ, kafka, metaMQ, rocketMQ and the like.

As can be seen from the foregoing background, when a query is performed in a form of a database, particularly, for a fuzzy query of multiple fields with large data volume, the problem of low data query efficiency will occur due to the large data volume and the relatively large number of fields with fuzzy matching.

Furthermore, by configuring the database table structure, http interfaces with dimensions of addition, modification, deletion, inquiry and the like can be provided to improve development efficiency and development quality.

When the search engine stores the data information of the target database stored according to the index constraint parameters, and the electronic equipment acquires the data query request of the user, the target data corresponding to the data query request can be determined by carrying out data query in the search engine according to the index constraint parameters containing the target database identification in the data query request. That is, when the data information of the target database is mapped to the search engine, the database can be ensured to be consistent with the data of the search engine, so that when the fuzzy query of the database is replaced by adopting the retrieval function of the search engine, excellent multi-field fuzzy query performance can be ensured to be still provided under the condition of extremely large data volume (in the billions), and the retrieval efficiency is improved.

Before describing the technical scheme of the application, an application scenario of the technical scheme of the application is first described. Fig. 1A is a schematic diagram of an application scenario of a data query method according to an embodiment of the present application. As shown in fig. 1A, the scene diagram may include a user terminal 11 and an electronic device 12. The electronic device 12 may include, among other things, a configuration module 121, a database 122, message middleware 123, a message service module 124, and a search engine 125.

Optionally, when a user has a data processing request for a database, the user terminal 11 may be operated to submit the data processing request to the electronic device 12, and the configuration module 121 of the electronic device 12 may obtain the data processing request, dynamically update the data information stored in the database 122 according to a preset database structure, the index constraint parameter carried in the data processing request, and the data to be processed, so as to generate a database log, and transmit the database log to the message service module 124 through the message middleware 123, where the message service module 124 processes the received database log, and updates the data update information of the database 122 to the search engine 125.

Accordingly, when a user has a data query request, the user may send the data query request to the electronic device 12 through the user terminal 11, so that the search engine 125 of the electronic device 12 may directly obtain the data query request, and perform data query in the search engine based on the index constraint parameter carrying the data to be queried in the data query request, so as to directly determine the target data corresponding to the data query request, and execute the data export process.

Fig. 1B is a schematic diagram of another application scenario of the data query method according to the embodiment of the present application. Fig. 1B differs from fig. 1A in that the user may directly operate the electronic device 12, e.g., the electronic device 12 has a user operation interface 120. Thus, when a user has a data processing request or a data query request for a database, the user operation interface 120 may be directly operated, so that the electronic device 12 obtains the data processing request or the data query request through the user operation interface 120.

It will be appreciated that the electronic device 12 of fig. 1B may include the same components as the electronic device of fig. 1A, and in the application scenario illustrated in fig. 1B, the electronic device 12 may also include a configuration module 121, a database 122, message middleware 123, a message service module 124, and a search engine 125. The functions of each component may be described in the application scenario shown in fig. 1A, and will not be described herein.

It should be noted that, fig. 1A and fig. 1B are only schematic diagrams of two exemplary application scenarios provided in the embodiments of the present application, and devices included in each application scenario may be set according to actual requirements, which is not described herein again.

Optionally, the data query method provided by the embodiment of the present application is explained by using the electronic device in the scene schematic shown in fig. 1A or fig. 1B as the execution body. Typically, the electronic device is a server, or may be a data storage device dedicated to storing data, for example, a terminal device, which is not limited by the embodiment of the present application.

The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 2 is a flowchart of a first embodiment of a data query method according to an embodiment of the present application. As shown in fig. 2, the data query method may include the steps of:

s201, acquiring a data query request of a user, wherein the data query request comprises index constraint parameters comprising a target database identifier and a field value.

As one example, when a user has a data query requirement, the user may directly operate the electronic device to issue a data query request. For example, when the electronic device has a user operation interface, a user can send out a data query request by operating the user operation interface, and when the electronic device has a voice receiving function, the electronic device can also send out the data query request by voice so as to acquire the data query request by utilizing the voice receiving function.

As another example, when the electronic device is connected to the user terminal, the user may send a data query request to the electronic device through the user terminal, so that the electronic device obtains the data query request.

The method for the electronic device to obtain the data query request is not limited in this embodiment, and may be determined according to actual requirements, which is not described herein.

In practical application, because the index constraint parameters are stored according to the formats of the identification of the database, the identification of the data table (also called as the alias of the data table), the identification of the field (also called as the field alias) (optional) and the field value, different databases, different fields and different field values can be located based on different index constraint parameters, and then the data information in different databases is determined. Therefore, in this embodiment, in order to enable the user to accurately locate the target data to be queried, the data query request sent by the user needs to carry index constraint information, and the index constraint information may include the target database identifier and the target query field value.

S202, according to the index constraint parameters, data query is carried out in a search engine, target data of a target database identifier corresponding to a target query field value in the target database is determined, and the search engine stores data information of the target database stored according to the index constraint parameters.

The search engine of the electronic device stores data information of different databases stored according to different index constraint parameters, and data in the search engine can be stored according to an inverted index mode, so when the electronic device executes data retrieval, the index constraint parameters are firstly analyzed to determine a target database identifier and a target query field value at least included in the index constraint parameters, then full-text retrieval is performed in the search engine based on the target database identifier and the target query field value, and further target data corresponding to a data query request is obtained.

The specific implementation of this step may be referred to as description in the embodiment shown in fig. 3 below, and will not be described here.

S203, outputting target data corresponding to the data query request.

As an example, when the electronic device has a user operation interface, target data corresponding to the data query request may be displayed on the user operation interface so as to be presented to the user.

As another example, the electronic device may also send the target data corresponding to the data query request to the user terminal, so that the user terminal presents the determined target data to the user.

According to the data query method provided by the embodiment of the application, the data query request of the user is obtained, the data query request comprises index constraint parameters comprising the target database identification and the field value, and the data information of the target database stored according to the index constraint parameters is stored in the search engine, so that the data query is performed in the search engine according to the index constraint parameters, the target data of the target database identification corresponding to the target query field value in the target database is determined, and finally the target data corresponding to the data query request is output. In the technical scheme, the data in the database is consistent with the data in the search engine, so that the search efficiency is improved when the search function of the search engine is adopted for data query.

Illustratively, in one possible design of the embodiment of the present application, the index constraint parameter further includes a target query data table identifier and a target query field value. At this time, the above S202 may be implemented by the embodiment shown in fig. 3 as follows.

Fig. 3 is a schematic flow chart of a second embodiment of a data query method according to an embodiment of the present application. As shown in fig. 3, S202 may be implemented by:

S301, positioning a data table set corresponding to a target database in a search engine according to the target database identification.

In this embodiment, the search engine of the electronic device stores data of a plurality of databases, each database has an identifier, and the data of each database is stored in the search engine according to index constraint parameters in formats of the identifier of the database, the identifier of the data table (also called an alias of the data table), the identifier of the field (also called a field alias) (optional), and the field value, so that the electronic device obtains a data query request, firstly parses the data query request into a target database identifier, and then locates a data table set corresponding to the target database in the search engine based on the target database identifier.

S302, positioning the target query data table in the data table set corresponding to the target database according to the target query data table identification.

Optionally, the data of each database may be stored in a plurality of data tables, so after determining the target query data table identifier from the index constraint parameter, search query may be performed in the data table set corresponding to the target database, so as to locate the target query data table to be queried.

In one example of the present application, the electronic device may query directly in the target query data table to determine target data at the target query field value. For example, when the index constraint parameter does not include the target query field identifier, the electronic device may directly query in the target query data table based on the target query field value, directly locate at the target query field value, and thereby obtain the target data at the target query field value.

In another example of the present application, when the index constraint parameter further includes a target query field identifier, performing a query in a target query data table to determine target data at a target query field value may be implemented by the following steps;

s303, inquiring in the target inquiry data table according to the target inquiry field identification, and positioning to the target inquiry field.

In this step, when the index constraint parameter includes the target query field identifier, the electronic device may locate, according to the target query field identifier, a target query field in the target query data table, thereby laying a foundation for obtaining target data.

S304, inquiring in the target inquiry field to determine target data corresponding to the target inquiry field value.

For example, the data in the search engine is stored according to an index constraint parameter (Key), and different keys correspond to different data index information, so when the electronic device locates the target query field, the electronic device can locate the position where the target query field value is located, and thus the target data corresponding to the target query field value is obtained.

According to the data query method provided by the embodiment of the application, if the index constraint parameter comprises a target database identifier, a target query data table identifier and a target field value, the data table set corresponding to the target database can be positioned in a search engine according to the target database identifier, then the target query data table is positioned in the data table set corresponding to the target database according to the target query data table identifier, and the target data at the target query field value can be determined by querying in the target query data table, and further, if the index constraint parameter also comprises the target query field identifier, the target query is performed in the target query data table according to the target query field identifier, the target query field is positioned, and then the target data corresponding to the target query field value is determined by querying in the target query field. According to the technical scheme, the target data can be queried in the search engine based on the index constraint parameters, so that a fuzzy query mode in a database is avoided, and the data query efficiency is improved.

For example, in one possible design of the embodiment of the present application, the electronic device may be capable of executing the data query in the search engine based on the index constraint parameter carried in the data query request, where relevant data information is stored in the search engine, and thus, before S201, the technical solution further needs to include a data processing procedure, for example, operations of data storage, updating, deleting, and so on.

Optionally, fig. 4 is a schematic flow chart of a third embodiment of the data query method provided in the embodiment of the present application. As shown in fig. 4, before the step S201, the method may further include the steps of:

s401, acquiring a data processing request aiming at a target database, wherein the data processing request comprises data to be processed and index constraint parameters.

As one example, when a user has a data manipulation requirement (e.g., manipulation of data storage, updating, deleting, etc.) for a target database, a data processing request may be issued by manipulating an electronic device. As another example, when a user operates a web application on a terminal device, certain data may also be generated, and a data processing request is generated accordingly, so as to update a database corresponding to the web application based on the data.

It will be appreciated that in order for the electronic device to determine the object to process, the data processing request will include the data to be processed. In order to ensure the processing position of the data to be processed, the data processing request also comprises an index constraint parameter, wherein the index constraint parameter carries the identification, the field value and the like of the data table.

S402, inserting the data to be processed into a target database based on the preconfigured database table structure and the index constraint parameters, and generating a database log.

In the embodiment of the application, the electronic equipment is provided with a configuration module, and the core function of the configuration module is to store the database table structure so as to facilitate the data processing of the electronic equipment on the database, such as query, export, insertion, modification, deletion and the like.

For example, when a user needs to store data in a database, the electronic device may dynamically store the data to be processed in the database according to a pre-configured database table structure when acquiring the data to be processed submitted by the user, thereby generating a database log.

Illustratively, this S402 may be implemented by:

A1, determining a target processing data table and a target processing field in the target processing data table according to index constraint parameters;

a2, dynamically storing the data to be processed into a target database based on configuration information of a database table structure, and generating the database log.

Optionally, after the configuration module of the electronic device obtains the data processing request, an index constraint parameter is extracted from the data processing request, then a data table identifier and a field identifier are extracted from the index constraint parameter, so that a target processing data table is found and positioned to a target processing field corresponding to the data table, finally, based on a database table structure, data to be processed in the data processing request is stored in the target processing data table and the target processing field in the target processing data table, further, data storage is realized, and correspondingly, a database log is generated by the target database.

Fig. 5 is a schematic diagram illustrating a database table structure according to an embodiment of the present application. As shown in fig. 5, the configuration module is centered on the data table in the database, and configures information such as field information, query condition, query structure, and export task of the data table.

Wherein, the data table refers to the name of the data table for configuring the interface in the record database and has a specified alias (i.e. identification);

The field refers to a field name, a field alias, a field type and the like corresponding to the data table in the record database;

the export task refers to a task of recording user export data;

the query condition refers to the condition names needed by the export task and the query task, and comprises field identification, alias of the query condition field and the like;

the query result refers to a field identifier, a field alias, etc. which are required to be returned by the export task and the query task.

In this embodiment, the data table shown in fig. 5 is configured to dynamically process the data in the data table, or some simple configurations may be used to quickly provide http interfaces with dimensions of addition, modification, deletion, query, etc., so that the interaction requirements between different systems (which may be systems with different functions, different development languages, different servers, etc., but all based on http services) can be better solved, thereby laying a foundation for subsequent data query.

S403, transmitting the database log to the message service module through the message middleware.

In this embodiment, when the data in the database is changed, the database log is generated and output, so that, in order to make the data of the target database stored in the search engine consistent with the data in the target database, the electronic device may extract the database log into the message middleware after generating the database log, and then transmit the message middleware to the message service module, so that the message service module processes the database log and stores the processed database log in the search engine.

Optionally, the message middleware is also called a message queue middleware, is used for data communication between applications, has a series of functions of loose coupling, asynchronous message, flow peak clipping, reliable delivery, broadcasting, flow control, final consistency and the like, and is a core means of enterprise internal communication. Therefore, in the embodiment, the message service module can be ensured to perform data analysis and processing smoothly by using the message middleware to transmit the database log.

S404, processing the database log by using the message service module to obtain the data updating information of the target database.

Optionally, because the JSON has a concise and clear hierarchical structure, the JSON becomes an ideal data exchange language, and is a lightweight data exchange format, which stores and represents data in a text format completely independent of a programming language, and has the functions of being easy for people to read and write, and simultaneously being easy for a machine to parse and generate, and effectively improving network transmission efficiency. Therefore, after the message service module obtains the database log, the database log can be parsed into JSON data, and the JSON data can be stored in the search engine.

It can be understood that, in this embodiment, JSON data generated by processing the database log by the message service module is data update information of the target database, which is not described herein.

S405, updating the data information of the target database in the search engine based on the data updating information.

In this step, after obtaining the data update information of the target database, the electronic device may update the data update information into the search engine, so that the data in the search engine is consistent with the data of the target database.

For example, the message service module receives a database log (the database log may be a data insertion log, a data update log, or a data deletion log, which is not limited in this embodiment) from the message middleware, restores the database log into real updated data information, and writes the real updated data information into the search engine, so that the search engine provides accurate retrieval service.

According to the data query method provided by the embodiment of the application, the data to be processed is inserted into the target database based on the pre-configured database table structure and index constraint parameters by acquiring the data processing request aiming at the target database, the database log is generated, the database log is transmitted to the message service module through the message middleware, the message service module is used for processing the database log to obtain the data updating information of the target database, and finally the data information of the target database in the search engine is updated based on the data updating information. According to the technical scheme, the database log generated by the database is mapped to the search engine, so that the database is consistent with the data of the search engine, and excellent data query performance can be ensured under the condition of extremely large data volume (in the billions) when the fuzzy query of the database is replaced by the search function of the search engine later.

Further, in the embodiment of the present application, the message service module is mainly configured to receive the database log from the message middleware, restore the database log to real data, thereby obtaining data update information of the target database, and write the data update information into the search engine to provide the search service.

Fig. 6 is a schematic flow chart of a fourth embodiment of the data query method according to the embodiment of the present application, which is illustrated on the basis of the embodiment shown in fig. 4. As shown in fig. 6, the step S404 may be implemented as follows:

S601, converting the database log into target processing data represented by the object by using a message service module.

In embodiments of the present application, data can be stored and represented entirely independently of the programming language due to the text format of the object representation. Thus, the electronic device, upon receiving the database log, using the message service module, may convert it into target processing data, e.g., JSON-formatted data, for the object representation, thereby performing different processing operations based on the message type of the database log.

S602, packaging target processing data into events corresponding to the message types according to the message types of the database logs.

Optionally, in this embodiment, the message service module may analyze the database log to determine a message type of the database log, and further package the target processing data obtained by conversion in S601 into different events based on the message type of the database log, so as to transfer to different processing units for processing according to the event type.

S603, determining an event processing unit corresponding to the event according to the type of the event.

Alternatively, in an embodiment of the present application, the message service module may include event processing units for processing different event types, so that the electronic device may determine an event processing unit for processing an event according to the event types, for example, an insert event, an update event, a delete event, etc., for example, an insert event processing unit, an update event processing unit, a delete event processing unit, etc.

S604, processing the target processing data by using an event processing unit corresponding to the event to obtain the data updating information of the target database.

Optionally, after determining the event processing unit corresponding to the event, the message service module may start the corresponding event processing unit to perform data processing, for example, process the insert event by using the insert event processing unit, process the update event by using the update event processing unit, process the delete event by using the delete event processing unit, and so on, and when the event processing unit processes the target processing data, obtain the data update information of the target database.

Fig. 7 is a schematic diagram illustrating the structure of a message service module in this embodiment. As shown in fig. 7, the message service module may include a message receiving unit, an event processing unit route, an event encapsulation unit, an event registration unit, an event processing unit, and the like. The event processing units can be further divided into different types of event processing units according to the event types. For example, when the types of events include an insert event, an update event, and a delete event, the event processing unit may include an insert event processing unit, an update event processing unit, and a delete event unit, etc.

In the schematic structure shown in fig. 7, when the electronic device is operating normally, the message service module is started, and all event processing sheets included in the message service module are registered in the event registration unit, so that the event registration unit manages all event processing units.

Optionally, the message receiving unit of the message service module is connected with the message middleware, and can receive the database log from the message middleware, and transmit the database log to the event processing unit for data conversion by routing to obtain target processing data represented by the object, the event processing unit transmits the target processing data to the event packaging unit by routing, the event packaging unit packages the target processing data into an event corresponding to the message type, determines a task identifier corresponding to the event, such as inserting the event, updating the event or deleting the event, and the like, and transmits the event to the event processing unit by routing through the event processing unit according to the task identifier, and distributes the event to the event processing unit corresponding to the task identifier for processing to obtain the data updating information of the target database.

According to the data query method provided by the embodiment of the application, the message service module is utilized to convert the database log into the target processing data represented by the object, the target processing data is packaged into the event corresponding to the message type according to the message type of the database log, the event processing unit corresponding to the event is determined according to the type of the event, and finally the event processing unit corresponding to the event is utilized to process the target processing data, so that the data update information of the target database is obtained. In the technical scheme, the message service module can ensure the precision and the processing efficiency of data processing by distinguishing the event from the event processing unit.

The following are examples of the apparatus of the present application that may be used to perform the method embodiments of the present application. For details not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the method of the present application.

Fig. 8 is a schematic structural diagram of an embodiment of a data query device according to an embodiment of the present application. As described with reference to fig. 8, the data query apparatus may include:

an obtaining module 801, configured to obtain a data query request of a user, where the data query request includes an index constraint parameter including a target database identifier and a target query field value;

A processing module 802, configured to perform data query in a search engine according to the index constraint parameter, determine that the target database identifier corresponds to target data at a target query field value in a target database, where the search engine stores data information of the target database stored according to the index constraint parameter;

and the output module 803 is configured to output target data corresponding to the data query request.

In one possible design of the embodiment of the present application, the index constraint parameter further includes a target lookup data table identifier;

the processing module 802 is specifically configured to:

The processing module 802 is configured to query in the target query data table, and determine target data at the target query field value, specifically:

The processing module 802 is specifically configured to perform a query in the target query data table according to the target query field identifier, locate a target query field, perform a query in the target query field, and determine target data corresponding to the target query field value.

In another possible design of the embodiment of the present application, the obtaining module 801 is further configured to obtain a data processing request for the target database, where the data processing request includes data to be processed and the index constraint parameter;

the processing module 802 is further configured to:

Optionally, the processing module 802 is configured to insert the data to be processed into the target database based on the preconfigured database table structure and the index constraint parameter, so as to generate a database log, specifically:

The processing module 802 is specifically configured to determine a target processing data table and a target processing field in the target processing data table according to the index constraint parameter, dynamically store the data to be processed in the target database based on configuration information of the database table structure, and generate the database log.

Optionally, the processing module 802 is configured to process the database log by using the message service module to obtain data update information of the target database, specifically:

the processing module 802 is specifically configured to:

The device provided by the embodiment of the application can be used for executing the technical scheme described in the embodiment of the method, and the implementation principle and the technical effect are similar and are not repeated here.

It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. The modules can be realized in the form of software which is called by the processing element, in the form of hardware, in the form of software which is called by the processing element, and in the form of hardware. For example, the processing module may be a processing element that is set up separately, may be implemented in a chip of the above-mentioned apparatus, or may be stored in a memory of the above-mentioned apparatus in the form of program codes, and the functions of the above-mentioned processing module may be called and executed by a processing element of the above-mentioned apparatus. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.

Fig. 9 is a schematic structural diagram of an embodiment of an electronic device according to an embodiment of the present application. As shown in fig. 9, the electronic device may include a processor 901, a memory 902, a communication interface 903, and a system bus 904, where the memory 902 and the communication interface 903 are connected to the processor 901 through the system bus 904 and perform communication between them, the memory 902 is used to store computer instructions, the communication interface 903 is used to communicate with other devices, and the processor 901 implements the technical solutions described in the above method embodiments when executing the computer instructions.

Optionally, in an embodiment of the present application, the electronic device may further include a user operation interface 905, where the user operation interface 905 may be used to receive a data query request or other data processing request from a user.

In fig. 9, the processor 901 may be a general-purpose processor including a central processing unit CPU, a network processor (network processor, NP), etc., or may be a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

The memory 902 may include random access memory (random access memory, RAM), read-only memory (RAM), and non-volatile memory (non-volatile memory), such as at least one disk memory.

The communication interface 903 is used to enable communication between the database access apparatus and other devices (e.g., clients, read-write libraries, and read-only libraries).

The system bus 904 may be a peripheral component interconnect (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, or the like. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

According to an embodiment of the present application, there is further provided a computer readable storage medium having stored therein computer instructions which, when executed on a computer, cause the computer to perform the technical solution as described in the method embodiment above.

According to the embodiment of the application, the embodiment of the application also provides a chip for running the instruction, and the chip is used for executing the technical scheme described in the embodiment of the method.

According to an embodiment of the present application, there is also provided a computer program product including a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read the computer program, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any of the embodiments described above.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of querying data, comprising:

outputting target data corresponding to the data query request;

Before the acquiring the data query request of the user, the method further comprises:

inserting the data to be processed into the target database based on a pre-configured database table structure and the index constraint parameters, and generating a database log;

and updating the data information of the target database in the search engine based on the data updating information so as to enable the data information in the search engine to be consistent with the data of the target database.

2. The method of claim 1, wherein the index constraint parameter further comprises a target look-up table identification;

3. The method of claim 2, wherein the index constraint parameter further comprises a target query field identification;

4. The method of claim 1, wherein the inserting the data to be processed into the target database based on the pre-configured database table structure and the index constraint parameter to generate a database log comprises:

5. The method according to claim 1, wherein the processing the database log with the message service module to obtain the data update information of the target database includes:

6. A data query device, comprising:

The output module is used for outputting target data corresponding to the data query request;

the acquisition module is further used for acquiring a data processing request aiming at the target database, wherein the data processing request comprises data to be processed and the index constraint parameters;

the processing module is further configured to:

7. The apparatus of claim 6, wherein the index constraint parameter further comprises a target lookup table identifier, and wherein the processing module is configured to:

8. The apparatus of claim 7, wherein the index constraint parameter further comprises a target query field identification;

9. The apparatus according to claim 6, wherein the processing module is configured to insert the data to be processed into the target database based on the pre-configured database table structure and the index constraint parameter, and generate a database log, specifically:

10. The apparatus of claim 6, wherein the processing module is configured to process the database log by using the message service module to obtain the data update information of the target database, specifically:

the processing module is specifically configured to:

11. An electronic device comprising a processor, a memory and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of the preceding claims 1-5 when executing the computer program.

12. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-5.

13. A computer program product comprising a computer program for implementing the method according to any of claims 1-5 when the computer program is executed by a processor.