[go: up one dir, main page]

CN115470229B - Data table processing method and device, electronic equipment and storage medium - Google Patents

Data table processing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115470229B
CN115470229B CN202211045460.7A CN202211045460A CN115470229B CN 115470229 B CN115470229 B CN 115470229B CN 202211045460 A CN202211045460 A CN 202211045460A CN 115470229 B CN115470229 B CN 115470229B
Authority
CN
China
Prior art keywords
time
sql
data table
processing
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211045460.7A
Other languages
Chinese (zh)
Other versions
CN115470229A (en
Inventor
崔雪征
周萌
杨剑锋
李崇
张金龙
李娟�
贾小龙
单东旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kunlun Digital Technology Co ltd
China National Petroleum Corp
Original Assignee
Kunlun Digital Technology Co ltd
China National Petroleum Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kunlun Digital Technology Co ltd, China National Petroleum Corp filed Critical Kunlun Digital Technology Co ltd
Priority to CN202211045460.7A priority Critical patent/CN115470229B/en
Publication of CN115470229A publication Critical patent/CN115470229A/en
Application granted granted Critical
Publication of CN115470229B publication Critical patent/CN115470229B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2477Temporal data queries
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Fuzzy Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The disclosure provides a data table processing method and device, electronic equipment and a storage medium, and belongs to the technical field of the Internet. The method comprises the following steps: generating a time parameter according to the time dimension type and the time period; acquiring a first SQL file, wherein the first SQL file is used for executing a processing request and is formed by splicing a plurality of SQL statements, and codes used for representing time in the SQL statements are expressed by adopting time variables; replacing the time variable of each SQL statement included in the first SQL file with a time parameter to obtain a second SQL file; and writing a processing result generated by executing the second SQL file and aiming at the target index into the target data table. According to the method and the device, the SQL sentences used for executing the same processing request are written into the same SQL file, and then the time variables in the SQL sentences are replaced in a unified mode, so that the problem that the time variables in the SQL sentences are not modified uniformly is solved, and the accuracy of the processed data table is improved.

Description

Data table processing method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for processing a data table, an electronic device, and a storage medium.
Background
In the field of internet technology, due to business requirements, calculation results of the same index in a data table at different time granularities of week, month, year and the like need to be obtained. Generally speaking, the computation logic of the same index at different time granularities is the same. In practical application, the calculation of the same index at different time granularities usually obtains an SQL file of the index at a certain time granularity, where the SQL file includes at least one SQL statement, and then based on the time at the current time granularity, a technician manually modifies each SQL statement in the SQL file to obtain a modified SQL file, and further based on each SQL statement in the modified SQL file, calculates the index in a data table to obtain a processed data table.
However, the above method is prone to modification inconsistency, resulting in low accuracy of the processed data table.
Disclosure of Invention
The embodiment of the disclosure provides a data table processing method and device, electronic equipment and a storage medium, which can improve the accuracy of a processed data table. The technical scheme is as follows:
in a first aspect, a method for processing a data table is provided, where the method includes:
receiving a processing request aiming at a target index in a target data table, wherein the processing request comprises a time dimension type and a time period;
generating a time parameter according to the time dimension type and the time period;
acquiring a first SQL file, wherein the first SQL file is used for executing the processing request and is formed by splicing a plurality of SQL statements, and codes used for representing time in the SQL statements are expressed by adopting time variables;
replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
and writing a processing result generated by executing the second SQL file and aiming at the target index into the target data table.
In another embodiment of the present disclosure, before writing the processing result generated by executing the second SQL file and aiming at the target index into the target data table, the method further includes
The SQL sentences included in the second SQL file are segmented to obtain a plurality of time-replaced SQL sentences;
executing each SQL statement after time replacement to obtain a processing result corresponding to each SQL statement after time replacement;
the writing, into the target data table, a processing result for the target index generated by executing the second SQL file includes:
and writing the corresponding processing result into the target database according to the type of each SQL statement after time replacement.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
and when the type of the SQL statement after the time replacement is a writing type, writing a processing result corresponding to the SQL statement after the time replacement into the target database.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
when the type of the SQL statement after the time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after the time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, and before writing the processing result generated by executing the second SQL file and directed to the target indicator into the target data table, the method further includes:
modifying a time dimension type field corresponding to the target index into the time dimension type, and modifying the time period field into the time period;
and under the condition that the modification of the two time fields corresponding to the target index is completed, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
In a second aspect, there is provided a data table processing apparatus, the apparatus comprising:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a processing request aiming at a target index in a target data table, and the processing request comprises a time dimension type and a time period;
the generating module is used for generating a time parameter according to the time dimension type and the time period;
the acquisition module is used for acquiring a first SQL file, the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL sentences, and codes used for representing time in the SQL sentences are expressed by adopting time variables;
the replacing module is used for replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
and the writing module is used for writing the processing result which is generated by executing the second SQL file and aims at the target index into the target data table.
In another embodiment of the present disclosure, the apparatus further comprises
The segmentation module is used for segmenting the SQL sentences included in the second SQL file to obtain a plurality of time-replaced SQL sentences;
the execution module is used for executing each SQL statement subjected to time replacement to obtain a processing result corresponding to each SQL statement subjected to time replacement;
and the writing module is used for writing the corresponding processing result into the target database according to the type of each SQL statement after time replacement.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
and when the type of the SQL statement after the time replacement is a writing type, writing a processing result corresponding to the SQL statement after the time replacement into the target database.
In another embodiment of the present disclosure, the writing, according to the type of the SQL statement after each time replacement, a corresponding processing result into the target database includes:
when the type of the SQL statement after the time replacement is a view type, generating a temporary view based on a processing result corresponding to the SQL statement after the time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, and before writing the processing result generated by executing the second SQL file and directed to the target indicator into the target data table, the method further includes:
modifying a time dimension type field corresponding to the target index into the time dimension type, and modifying the time period field into the time period;
and under the condition that the two time fields corresponding to the target index are modified, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
In a third aspect, an electronic device is provided, which includes a processor and a memory, where at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the data table processing method according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the data table processing method according to the first aspect.
In a fifth aspect, a computer program product is provided, the computer program product comprising computer program code stored in a computer-readable storage medium, the computer program code being read from the computer-readable storage medium by a processor of an electronic device, the processor executing the computer program code to cause the electronic device to perform the data table processing method according to the first aspect.
The technical scheme provided by the embodiment of the disclosure has the following beneficial effects:
the method comprises the steps of writing target indexes needing to be processed into a target data table, writing SQL sentences used for executing the same processing request into the same SQL file, and when processing results of the target indexes in a certain time period under a certain time dimension type need to be obtained, uniformly replacing time variables of all the SQL sentences in the SQL file, so that the problem that the time variables in all the SQL sentences are not uniformly modified is solved, and the accuracy of the processed data table is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is apparent that the drawings in the description below are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings may be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a method for processing a data table according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of another method of data table processing provided by embodiments of the present disclosure;
FIG. 3 is a flow chart of another method of processing a data table provided by an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of a data table processing apparatus according to an embodiment of the present disclosure;
fig. 5 shows a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
It is to be understood that the terms "each," "a plurality," and "any" and the like, as used in the embodiments of the present disclosure, are intended to encompass two or more, each referring to each of the corresponding plurality, and any referring to any one of the corresponding plurality. For example, the plurality of words includes 10 words, and each word refers to each of the 10 words, and any word refers to any one of the 10 words.
Information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals to which the present disclosure relates are all authorized by the user or sufficiently authorized by various parties, and the collection, use, and processing of the relevant data requires compliance with relevant laws and regulations and standards in the relevant countries and regions.
The embodiment of the present disclosure provides a data table processing method, which is implemented by taking an electronic device as an example, where the electronic device has a relatively high computing capability, the electronic device may be a terminal, such as a notebook computer, a desktop computer, and the like, and the electronic device may also be a server, such as an individual physical server, a cluster or distributed system formed by multiple physical servers, and the like. Referring to fig. 1, a method flow provided by the embodiment of the present disclosure includes:
101. a processing request for a target metric in a target data table is received.
In the embodiment of the present disclosure, when a processing result of a target index in a target data table needs to be obtained due to a business requirement, a user may send a processing request for the target index in the target data table to an electronic device. The target data table is a data table for bearing the target index and the processing result of the target index. The target data table includes a plurality of indexes, the index refers to a field name of a certain field in the target data table, and the index may be gender, age, salary and the like. The processing request is used for requesting the electronic equipment to process data related to the target indexes in the target database in the time period according to the time dimension type, the processing request comprises the time dimension type, the time period and the like, and the time dimension type comprises the types of year, month, week, day and the like.
102. And generating a time parameter according to the time dimension type and the time period.
In the embodiment of the present disclosure, the processing request received by the electronic device is actually an SQL statement, and the SQL statement needs to be executed by a scheduler, which is a program for executing an SQL file. In response to a received processing request for a target index in the target data table, the electronic device calls a scheduling program, and then generates a time parameter according to the time dimension type and the time period.
Specifically, the electronic device divides a time period according to a time dimension type to obtain a time parameter of the time period in the time dimension type. For example, if the time dimension type is week, and the time period is from 1 month 1 to 1 month 21, the electronic device divides the time period from 1 month 1 to 1 month 21 according to the week, and obtains the time parameters as: no. 1-No. 7, no. 1-No. 8-No. 1-No. 14, and No. 1-No. 14-No. 1-No. 21.
103. And acquiring a first SQL file.
In the embodiment of the present disclosure, according to different service types, the electronic device splices SQL statements that process the same service in advance, and places the spliced SQL statements in one SQL file. Usually, these SQL files are stored in the hard disk, and when a processing request for a target index in a target data table is received, in response to the processing request, the electronic device obtains a first SQL file corresponding to the service type from the hard disk, and then writes the first SQL file into the memory, so as to facilitate subsequent processing of the first SQL file. The first SQL file is used for executing a processing request aiming at a target index in a target data table and is formed by splicing a plurality of SQL sentences.
In the embodiment of the present disclosure, the code for representing time in the SQL statements included in the SQL file is represented by a time variable. Generally, the names of time variables of different time dimension types are consistent, generally, only a few time variables are needed, and for special time, the time variables can be represented by time functions. Because the code used for representing the time in each SQL statement included in the first SQL file is represented by a time variable and has no exact execution time, the electronic device cannot execute the first SQL file, and the time variable in the first SQL file needs to be modified to a time constant.
104. And replacing the time variable of each SQL statement included in the first SQL file with a time parameter to obtain a second SQL file.
Based on the acquired time variable, the electronic equipment identifies the time variable in each SQL statement in the first SQL file, and then replaces the time variable of each SQL statement with the time parameter in a unified manner to obtain a second SQL file. The electronic device can execute the second SQL file because the code representing time in each SQL statement in the second SQL file is changed from the time variable to the time constant. By executing the second SQL file, the processing result of the target index may be obtained.
105. And writing the processing result generated by executing the second SQL file and aiming at the target index into the target data table.
In the embodiment of the present disclosure, before writing the processing result generated by executing the second SQL file and aiming at the target index into the target data table, the electronic device will perform the following operations:
the first step is that the electronic equipment divides SQL sentences included in the second SQL file to obtain a plurality of time-substituted SQL sentences.
In the embodiment of the present disclosure, because the SQL statements in the second SQL file are spliced together, and the operations executed by different SQL statements are different, in order to execute different operations based on different SQL statements, the electronic device needs to segment the SQL statements included in the second SQL file. When the electronic device splits the second SQL file, a symbol (for example, an english comma) may be preset as a splitting point to split the second SQL file. And segmenting the second SQL file to obtain a plurality of time-replaced SQL sentences, wherein the plurality of time-replaced SQL sentences are independent from each other and can be executed independently.
And secondly, the electronic equipment executes each time-substituted SQL statement to obtain a processing result corresponding to each time-substituted SQL statement.
Based on each time-substituted SQL statement, the electronic device may obtain a processing result corresponding to each time-substituted SQL statement by executing each time-substituted SQL statement, and then write a processing result for the target index generated by executing the second SQL file into the target data table.
In the disclosed embodiment, the SQL statement has different types, including a write type and a view type. Generally, write-type SQL statements begin with Insert and view-type SQL statements begin with @. Therefore, different types of SQL sentences can be distinguished according to the initial characters of the different types of SQL sentences. For example, for any SQL statement, @ v _ org select org _ id, org _ name form i _ org where dt = '2022-02-22', the type of the SQL statement can be identified as the view type according to the initial character of the SQL statement.
Because the operations executed by different types of SQL statements are different, the forms of the obtained processing results are also different, and the processing modes of the electronic device are different for different forms of processing results, the electronic device can write the corresponding processing results into the target database in different modes according to the types of the SQL statements replaced at each time.
In a possible implementation manner, when the type of the SQL statement after the time replacement is the write type, the electronic device may directly write the processing result corresponding to the SQL statement after the time replacement into the target database.
In another possible implementation manner, when the type of the SQL statement after time replacement is a view type, the processing result corresponding to the SQL statement after time replacement is generally an intermediate processing result, and the SQL statement after time replacement often needs to read the intermediate processing result to generate a processing result corresponding to the SQL statement after time transformation, so that the electronic device does not directly write the processing result corresponding to the SQL statement after time replacement into the target data table, but generates a temporary view based on the processing result corresponding to the SQL statement after time replacement. The data in the temporary view is the execution result of the temporally replaced SQL statement, for example, if the temporally replaced SQL statement is "select org _ id, org _ name from i _ org where dt = '2022-02-22'", then the data in the temporary view is the execution result of "select org _ id, org _ name from i _ org where dt = '2022-02-22'". Because the temporary view is stored in the memory and does not need to be displayed and created, the subsequent table building and maintenance work is saved, and the processing efficiency of the data table is improved.
Further, when the SQL statements replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL statements replaced at other times, the electronic device writes the processing results corresponding to the SQL statements replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, where the time dimension type field is used to distinguish different time granularities, and tinyint types may be used; the time period field can adopt a date type, so that different databases can be partitioned conveniently.
In another embodiment of the present disclosure, to avoid confusion of processing results of different time granularities corresponding to the target index, before writing the processing result generated by executing the second SQL file and directed to the target index into the target data table, the electronic device further modifies the time dimension type field corresponding to the target index into a time dimension type and modifies the time period field into a time period, so that, when the modification of the two time fields corresponding to the target index is completed, the electronic device executes the operation of writing the processing result generated by the second SQL file and directed to the target index into the target data table.
In another embodiment of the disclosure, in response to a processing request for a target index in a target data table, an electronic device writes an operation involved in a processing process into a log corresponding to the target data table during the processing process of the target index. When the abnormal record log is identified, the electronic equipment also sends the abnormal content and the context to a technician so that the technician can trace the processing process conveniently. When the electronic device sends the abnormal content to the technician, the abnormal content can be sent by adopting a network, a mail, an instant messaging tool and the like, and the embodiment of the disclosure does not limit the sending mode of the abnormal content.
By adopting the method provided by the embodiment of the disclosure, a set of SQL codes can be used, the calculation of the same index of the time granularity of a plurality of days, weeks, months and the like can be realized, and compared with the mode that the time codes in the SQL sentences need to be manually modified one by one in the related art, the method is more convenient to debug, and is more convenient to change particularly when bugs or business changes are repaired. In addition, the generated temporary view does not need to be displayed and maintained, and is friendly to both development and execution.
Fig. 2 shows an overall flow of a data table processing method provided by the embodiment of the present disclosure, and referring to fig. 3, the processing flow includes the following steps:
1. when a processing request aiming at a target index in a target data table is received, the electronic equipment calls a scheduling program, and the scheduling program generates time corresponding to a time variable;
2. the scheduling degree reads the SQL file;
3. the scheduling program replaces the time variable of each SQL statement in the SQL file by specific time;
4. the processed SQL file is divided into single SQL sentences, and the SQL sentences are analyzed and recombined according to different types;
5. and writing the processing result into the target data table by executing the split SQL statement.
Fig. 3 shows different processing procedures based on different types of SQL statements, and referring to fig. 3, the electronic device determines the type of the SQL statement according to the symbol at the beginning of the SQL statement. When the type of the SQL statement is a writing type, writing a processing result of the SQL statement into a target data table; and when the type of the SQL statement is the view type, generating a Spark temporary view, and further writing the processing result of other SQL statements based on the temporary view into the target data table.
According to the method provided by the embodiment of the disclosure, the target indexes to be processed are written into the target data table, the SQL sentences used for executing the same processing request are written into the same SQL file, and when the processing result of the target indexes in a certain time period under a certain time dimension type needs to be obtained, the time variables of all the SQL sentences in the SQL file are replaced in a unified manner, so that the problem that the time variables in all the SQL sentences are not modified uniformly is solved, and the accuracy of the processed data table is improved.
Referring to fig. 4, an embodiment of the present disclosure provides a data table processing apparatus, including:
a receiving module 401, configured to receive a processing request for a target indicator in a target data table, where the processing request includes a time dimension type and a time period;
a generating module 402, configured to generate a time parameter according to the time dimension type and the time period;
an obtaining module 403, configured to obtain a first SQL file, where the first SQL file is used to execute a processing request, the first SQL file is formed by splicing a plurality of SQL statements, and a code in the SQL statement for representing time is represented by a time variable;
a replacing module 404, configured to replace a time variable of each SQL statement included in the first SQL file with a time parameter, to obtain a second SQL file;
and a writing module 405, configured to write a processing result for the target index, generated by executing the second SQL file, into the target data table.
In another embodiment of the present disclosure, the apparatus further comprises
The segmentation module is used for segmenting the SQL sentences included in the second SQL file to obtain a plurality of time-replaced SQL sentences;
the execution module is used for executing each time-replaced SQL statement to obtain a processing result corresponding to each time-replaced SQL statement;
and the writing module is used for writing the corresponding processing result into the target database according to the type of the SQL statement replaced at each time.
In another embodiment of the present disclosure, writing the corresponding processing result into the target database according to the type of the SQL statement replaced at each time includes:
and when the type of the SQL statement after time replacement is a write-in type, writing a processing result corresponding to the SQL statement after time replacement into the target database.
In another embodiment of the present disclosure, writing the corresponding processing result into the target database according to the type of the SQL statement replaced at each time includes:
when the type of the SQL statement after time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target database.
In another embodiment of the present disclosure, each indicator in the target data table corresponds to two time fields, where the two time fields include a time dimension type field and a time period field, and before writing a processing result generated by executing the second SQL file and directed to the target indicator into the target data table, the method further includes:
modifying a time dimension type field corresponding to the target index into a time dimension type, and modifying a time period field into a time period;
and under the condition that the modification of the two time fields corresponding to the target index is completed, executing the operation of writing the processing result generated by the second SQL file and aiming at the target index into the target data table.
To sum up, the apparatus provided in this embodiment of the present disclosure writes a target index to be processed into a target data table, writes an SQL statement for executing a same processing request into a same SQL file, and when a processing result of the target index in a certain time period under a certain time dimension type needs to be obtained, performs unified replacement on a time variable of each SQL statement in the SQL file, overcomes a problem that the time variable in each SQL statement is not uniformly modified, and improves accuracy of the processed data table.
Fig. 5 shows a block diagram of an electronic device 500 according to an exemplary embodiment of the disclosure. In general, the electronic device 500 includes: a processor 501 and a memory 502.
The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 501 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in a wake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 501 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.
Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 502 is used to store at least one instruction for execution by processor 501 to implement the database processing methods provided by method embodiments in the present disclosure.
In some embodiments, the electronic device 500 may further optionally include: a peripheral interface 503 and at least one peripheral. The processor 501, memory 502 and peripheral interface 503 may be connected by a bus or signal lines. Various peripheral devices may be connected to the peripheral interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: a power supply 504.
The peripheral interface 503 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502, and peripheral interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral interface 503 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
The power supply 504 is used to power the various components in the electronic device 500. The power source 504 may be alternating current, direct current, disposable or rechargeable. When the power supply 504 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charge technology.
Those skilled in the art will appreciate that the configuration shown in fig. 5 is not intended to be limiting of the electronic device 500 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.
In an exemplary embodiment, there is also provided a computer readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of the electronic device 500 to perform the above-described database processing method. Alternatively, the storage medium may be a non-transitory computer readable storage medium, for example, a CD-ROM (Compact Disc Read-Only Memory), a ROM, a RAM (Random Access Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.
The embodiment of the present disclosure provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor, so as to implement the above data table processing method.
The disclosed embodiments provide a computer program product comprising computer program code stored in a computer readable storage medium, from which a processor of an electronic device reads the computer program code, the processor executing the computer program code, causing the electronic device to execute the above-mentioned data table processing method.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is intended to be exemplary only and not to limit the present disclosure, and any modification, equivalent replacement, or improvement made without departing from the spirit and scope of the present disclosure is to be considered as the same as the present disclosure.

Claims (7)

1. A method for processing a data table, the method comprising:
receiving a processing request aiming at a target index in a target data table, wherein the processing request comprises a time dimension type and a time period, and the processing request is used for requesting to process data related to the target index in the target data table in the time period according to the time dimension type;
dividing the time period according to the time dimension type to obtain a time parameter of the time period under the time dimension type;
acquiring a first SQL file, wherein the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL statements, and codes used for representing time in the SQL statements are expressed by adopting time variables;
replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
the SQL sentences included in the second SQL file are segmented to obtain a plurality of time-substituted SQL sentences;
executing each SQL statement after time replacement to obtain a processing result corresponding to each SQL statement after time replacement;
and writing the corresponding processing result into the target data table according to the type of each SQL statement after time replacement.
2. The method according to claim 1, wherein writing the corresponding processing result into the target data table according to the type of each time-substituted SQL statement comprises:
and when the type of the SQL statement after the time replacement is a write-in type, writing a processing result corresponding to the SQL statement after the time replacement into the target data table.
3. The method according to claim 1, wherein writing the corresponding processing result into the target data table according to the type of each time-substituted SQL statement comprises:
when the type of the SQL statement after the time replacement is the view type, generating a temporary view based on a processing result corresponding to the SQL statement after the time replacement;
and when the SQL sentences replaced at other times read the data in the temporary view to generate processing results corresponding to the SQL sentences replaced at other times, writing the processing results corresponding to the SQL sentences replaced at other times into the target data table.
4. The method according to any one of claims 1 to 3, wherein each indicator in the target data table corresponds to two time fields, the two time fields include a time dimension type field and a time period field, and before writing the corresponding processing result into the target data table according to the type of each time-replaced SQL statement, the method further includes:
modifying a time dimension type field corresponding to the target index into the time dimension type, and modifying the time period field into the time period;
and under the condition that the modification of the two time fields corresponding to the target index is completed, executing the operation of writing the corresponding processing result into the target data table according to the type of the SQL statement after each time replacement.
5. A data table processing apparatus, the apparatus comprising:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a processing request aiming at a target index in a target data table, the processing request comprises a time dimension type and a time period, and the processing request is used for requesting to process data related to the target index in the target data table in the time period according to the time dimension type;
the generating module is used for dividing the time period according to the time dimension type to obtain a time parameter of the time period under the time dimension type;
the acquisition module is used for acquiring a first SQL file, the first SQL file is used for executing the processing request, the first SQL file is formed by splicing a plurality of SQL sentences, and codes used for representing time in the SQL sentences are expressed by adopting time variables;
the replacing module is used for replacing the time variable of each SQL statement included in the first SQL file with the time parameter to obtain a second SQL file;
the segmentation module is used for segmenting the SQL sentences included in the second SQL file to obtain a plurality of time-replaced SQL sentences;
the execution module is used for executing each SQL statement after time replacement to obtain a processing result corresponding to each SQL statement after time replacement;
and the writing module is used for writing the corresponding processing result into the target data table according to the type of each SQL statement after the time replacement.
6. An electronic device, comprising a processor and a memory, wherein at least one program code is stored in the memory, and wherein the at least one program code is loaded and executed by the processor to implement the data table processing method according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that at least one program code is stored in said storage medium, which is loaded and executed by a processor to implement the data table processing method according to any one of claims 1 to 4.
CN202211045460.7A 2022-08-30 2022-08-30 Data table processing method and device, electronic equipment and storage medium Active CN115470229B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211045460.7A CN115470229B (en) 2022-08-30 2022-08-30 Data table processing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211045460.7A CN115470229B (en) 2022-08-30 2022-08-30 Data table processing method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115470229A CN115470229A (en) 2022-12-13
CN115470229B true CN115470229B (en) 2023-04-14

Family

ID=84368417

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211045460.7A Active CN115470229B (en) 2022-08-30 2022-08-30 Data table processing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115470229B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823680A (en) * 2014-02-24 2014-05-28 广州精攻网络科技有限公司 Development method and device for game service logic engine
CN109299132A (en) * 2018-11-29 2019-02-01 中国人民财产保险股份有限公司 SQL data processing method, system and electronic equipment

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10210240B2 (en) * 2017-06-30 2019-02-19 Capital One Services, Llc Systems and methods for code parsing and lineage detection
CN107908679B (en) * 2017-10-26 2020-11-27 平安科技(深圳)有限公司 Script statement conversion method and device and computer readable storage medium
CN111414377B (en) * 2020-03-05 2023-10-24 微民保险代理有限公司 Method, device and storage medium for processing structured query statement
CN112380180A (en) * 2020-11-17 2021-02-19 平安普惠企业管理有限公司 Data synchronization processing method, device, equipment and storage medium
CN113391818A (en) * 2021-03-26 2021-09-14 广州海量数据库技术有限公司 Compiled file generation method and device
CN113760864B (en) * 2021-04-21 2025-05-16 北京沃东天骏信息技术有限公司 Data model generation method and device
CN113076333A (en) * 2021-04-25 2021-07-06 中国建设银行股份有限公司 Service abnormity monitoring method and device
CN113918594A (en) * 2021-09-27 2022-01-11 苏宁易购集团股份有限公司 Method and device for generating structured query statement and computer equipment
CN113900944B (en) * 2021-10-12 2025-07-18 北京沃东天骏信息技术有限公司 A logical verification method and device applied to Flink SQL

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823680A (en) * 2014-02-24 2014-05-28 广州精攻网络科技有限公司 Development method and device for game service logic engine
CN109299132A (en) * 2018-11-29 2019-02-01 中国人民财产保险股份有限公司 SQL data processing method, system and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Thi-Hai-Yen Vuong ; .Learning to Transform Vietnamese Natural Language Queries into SQL Commands.IEEE.2019,第1-5页. *

Also Published As

Publication number Publication date
CN115470229A (en) 2022-12-13

Similar Documents

Publication Publication Date Title
CN108460068B (en) Method, device, storage medium and terminal for importing and exporting report
WO2019169725A1 (en) Test data generation method, device, and apparatus, and computer readable storage medium
JP7795018B2 (en) Providing application error data for use by third-party library development systems
CN110825731A (en) Data storage method and device, electronic equipment and storage medium
CN110019116A (en) Data traceability method, apparatus, data processing equipment and computer storage medium
CN114398282A (en) Test script generation method, device, device and storage medium
CN110490724A (en) The storage method and device of account data
CN110727677B (en) Method and device for tracing blood relationship of table in data warehouse
CN113609823A (en) Method and device for editing questionnaire logic, terminal equipment and storage medium
CN114416174A (en) Metadata-based model reconstruction method, device, electronic device and storage medium
CN110633258B (en) Log insertion method, device, computer device and storage medium
CN111859985A (en) AI customer service model testing method, device, electronic equipment and storage medium
CN115470229B (en) Data table processing method and device, electronic equipment and storage medium
CN111127223A (en) Insurance product testing method and device and storage medium
CN115129749B (en) Complex query optimization method, device and medium
CN118606177A (en) Code scoring method, system, electronic device and storage medium
US20200167152A1 (en) Identification of a partial code to be refactored within a source code
CN109840213B (en) Test data creating method, device, terminal and storage medium for GUI test
CN115543936A (en) Method, device and equipment for constructing model files in batches and storage medium
CN116661758A (en) Method, device, electronic equipment and medium for optimizing log framework configuration
CN110309038B (en) Performance test method and device, electronic equipment and computer readable storage medium
CN112416983B (en) Data processing method and device and computer readable storage medium
CN115409005A (en) Data supervision method and device and electronic equipment
CN113626281A (en) Slow SQL statement tracking method and device, electronic equipment and storage medium
CN117874211B (en) Intelligent question-answering method, system, medium and electronic equipment based on SAAS software

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant