CN114547109A - Database information screening method, system, storage medium and electronic equipment - Google Patents

Abstract

Embodiments of the present invention provide a database information screening method, system, storage medium and electronic device. Wherein, the method includes: acquiring a sampling data table, when the sampling data table has at least one record information, searching and obtaining a field type with at least one field information in the sampling data table, obtaining a target field type, and screening a template based on the target information for the target field type. Information detection is performed for each field information under the field type, and the number of field information that matches the target information screening template is obtained. When the ratio of the number to the total number of field information under the target field type is greater than the preset ratio, the target field is obtained. The position information of the field file under the type in the database is obtained, and the first position information is obtained; the field file includes at least one field information, and the first target information is obtained by searching the database based on the first position information. The invention can efficiently realize the screening of database information, so as to meet the requirement of large-scale batch processing of database information.

Description

Database information screening method, system, storage medium and electronic device

technical field

The invention relates to the technical field of data processing, in particular to a database information screening method, system, storage medium and electronic device.

Background technique

The information stored in the database will inevitably involve some sensitive information, such as ID number, bank card number, mobile phone number, bank card track information, bank card verification code, bank card validity period, password and other information. If these sensitive information is leaked, it will have adverse effects. Therefore, it is necessary to filter out the plaintext sensitive information stored in the database from the database as soon as possible, so as to facilitate the encryption of the plaintext sensitive information.

At present, when screening sensitive information in a database, only a small amount of data can be scanned for sensitive information at a time, which cannot meet the needs of large-scale batch processing of database information. Therefore, how to efficiently implement database information screening is very necessary to prevent the leakage of sensitive information.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a database information screening method, system, storage medium and electronic device, which can efficiently realize database information screening and meet the needs of large-scale batch processing of database information. The specific technical solutions are as follows:

The invention provides a database information screening method, comprising:

Obtain a sampling data table; the sampling data table is generated based on the sampling information of the database;

When there is at least one piece of record information in the sampling data table, look up and obtain a field type with at least one field information from the sampling data table, and obtain a target field type;

Perform information detection on each field information under the target field type based on the target information screening template, and obtain the number of field information matching the target information screening template;

When the ratio of the number to the total number of field information under the target field type is greater than a preset ratio, acquiring the position information of the field file under the target field type in the database, and obtaining the first position information; The field file includes at least one of the field information;

The first target information is obtained by searching the database based on the first location information.

Optionally, performing information detection on each field information under the target field type based on the target information screening template, specifically including:

When the target information screening template is keyword information, obtain the field name of the field information under the target field type; determine whether the field name contains the keyword information; if so, detect and obtain the field The field information corresponding to the name matches the target information screening template;

When the target information screening template is character combination information, it is judged whether the field information under the target field type contains the character combination information; if it does, it is detected that the field information under the target field type and the The target information filter template matches.

Optionally, also include:

When the ratio of the number to the total number of field information under the target field type is greater than 0 and less than a preset ratio, obtain the position information of the field information under the target field type in the database, and obtain the second position information;

The second target information is obtained by searching the database based on the second location information.

Optionally, also include:

A first information screening report table is generated based on the first target information parameter; the first target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, and The username corresponding to the sampling data table, the table name of the sampling data table, the field name of the field information matching the target information screening template, and the number of field information matching the target information screening template , one or more of the name of the first target information, the preset ratio, and the first position information;

A second information screening report table is generated based on the second target information parameter; the second target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, and The username corresponding to the sampling data table, the table name of the sampling data table, the field name of the field information matching the target information screening template, and the number of field information matching the target information screening template , one or more of the name of the second target information, the preset ratio, and the second position information.

Optionally, also include:

When there is at least one piece of record information in the sampling data table, look up and obtain a field type that does not have field information from the sampling data table, and obtain a reference field type;

A third information screening report table is generated based on the third target information parameter; the third target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, and One or more of the user name corresponding to the sampling data table, the table name of the sampling data table, the field name corresponding to the reference field type, and the number of non-empty record information;

When there is no record information in the sampling data table, a fourth information screening report table is generated based on the fourth target information parameter; the fourth target information parameter is the instance name of the database, the IP address of the database , one or more of the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the table name of the sampling data table, and the number of non-empty record information.

Optionally, the method for generating the sampling data table specifically includes:

Obtain all the data tables corresponding to the user name in the oracle database, and obtain the first data table set;

Select a data table in the described first data table set, calculate the ratio of the storage size of a piece of record information in the selected data table and the storage size of the selected data table to obtain the storage ratio;

Determine the multiple of the sampling quantity, so that the sampling ratio calculated based on the multiple is within the preset sampling value range; the preset sampling value range is the sampling value range of the sampling function of the oracle database; the sampling The ratio is the product of the multiple and the sampling number multiplied by the storage ratio;

Based on the sampling function of the oracle database, the record information of the multiple sampling quantity is collected from the selected data table, and the first sampling record information is obtained;

The sampling data table is generated based on the recording information of the sampling quantity selected from the first sampling recording information.

Optionally, the method for generating the sampling data table specifically includes:

Obtain all the data tables corresponding to the user name in the mysql database, and obtain the second data table set;

Select a data table in the second data table set, derive the record information of the preset number from the selected data table based on the data export instruction of the mysql database, and obtain the second record information;

The sampling data table is generated based on the second record information.

The present invention also provides a database information screening system, comprising:

a data acquisition module for acquiring a sampling data table; the sampling data table is generated based on the sampling information of the database;

a field determination module, configured to search and obtain a field type with at least one field information from the sampled data table when there is at least one piece of record information in the sampled data table, and obtain a target field type;

an information detection module, configured to perform information detection on each field information under the target field type based on a target information screening template, and obtain the number of field information that matches the target information screening template;

A first location determination module, configured to acquire the location information of the field file under the target field type in the database when the ratio of the number to the total number of field information under the target field type is greater than a preset ratio , obtain the first location information; the field file includes at least one of the field information;

A data extraction module, configured to search and obtain first target information from the database based on the first location information.

The present invention also provides a computer-readable storage medium, where a program is stored on the computer-readable storage medium, and when the program is executed by a processor, the above database information screening method is implemented.

The present invention also provides an electronic device, comprising:

at least one processor, and at least one memory and bus connected to the processor;

The processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the above database information screening method.

In a database information screening method, system, storage medium and electronic device provided by the embodiments of the present invention, the field information matching the target information screening template is searched from the sampling data table, and the location information of the field information in the database is obtained, based on The location information is searched from the database to obtain the target information. Through the method of screening target information by field, the present invention can efficiently realize the screening of database information, so as to meet the needs of large-scale batch processing of database information.

Of course, it is not necessary for any product or method to implement the present invention to simultaneously achieve all of the advantages described above.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a flowchart of a database information screening method provided in an embodiment of the present invention;

2 is a block diagram of a database information screening system provided by an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

An embodiment of the present invention provides a database information screening method, as shown in FIG. 1 , the method includes:

S101. Obtain a sampling data table. The sampling data table is generated based on the sampling information of the database.

Optionally, in an optional embodiment of the present invention, the specific implementation of the acquired sampling data table may be: acquiring a set of all data tables stored in the database, and randomly selecting a non-null data in the set. Extract data from the table, and import the extracted data into a blank sampling data table to obtain a non-empty sampling data table.

Optionally, in another optional embodiment of the present invention, the type of the above-mentioned database may be a relational database, such as a mySQL database and an Oracle database.

S102. When there is at least one piece of record information in the sampling data table, search and obtain a field type with at least one field information in the sampling data table, and obtain the target field type.

The above-mentioned field type may be the column name of the column where the field information is located in the above-mentioned acquired sampling data table.

S103: Perform information detection on each field information under the target field type based on the target information screening template, and obtain the number of field information matching the target information screening template.

Optionally, in an optional embodiment of the present invention, the target information screening template may be a screening rule preset according to field information. For example: password rules, bank card validity rules and verification code rules, etc. Wherein, the above-mentioned filtering rule may be a set including a plurality of field information texts.

Exemplarily, the target information screening templates can be as follows:

1) The ID number screening template can be any 1-digit number starting with the number 1 or 5 or 6 and then splicing 1-5, or it can be any 1-digit number starting with the number 2 or 4 and then splicing 1-3, or To splicing any 1-digit number in the range of 1-7 starting with the number 3, continue splicing 0 and then splicing any 1-9 digit, or continue splicing any 1-6 digit and then splicing any 1-digit number in the range of 0-9 Any 1-digit number, or continue to splicing 70, or continue to splicing 90, then splicing 0 and then splicing any 1-9 digit, or then splicing 1-9 1-digit and then splicing 0-9 any 1-digit number, The middle splicing range is 19000101-20991231, and finally splicing any 3 digits and ending with any 1 digit or uppercase X or lowercase x.

2) The bank card number screening template can be spliced with any 2 digits starting with the number 62, and 2 groups can contain connectors - splicing any 4 digits, can include connectors - splicing any 3 or 4 digits, you can choose 1 group that can contain Connector - Concatenate any 1-3 digits to end.

3) The mobile phone number screening template can be a combination that can contain a plus sign or 00 at the beginning and splicing the number 86, the middle can be the number 13 or 18 and then any 1-digit number from 0-9, and the middle can also be the number 14 spliced with the number 5 or 7 or 9, the number 15 or 19 can be spliced with a non-4 number in the middle, the number 166 can also be in the middle, and the number 17 can also be spliced with a non-2, non-4 and non-9 number, and finally any 8 digits Number ends.

4) The track information screening template can be spliced with a 16-digit number starting with the letter B and ending with ^ plus any single character except a newline, or it can be directly spliced with an equal sign at the beginning of the 16-digit number = plus except for a newline It can end with any single character of , or it can start with the number 99 and spliced 16 digits in the middle and end with an equal sign = plus any single character except the newline character.

5) The password rule filtering template can be the uppercase conversion match with the column name keyword PWd or PaSSwD or kL and contain 6-18 uppercase and lowercase letters and numbers.

6) The bank card screening template can be a capital conversion match where the column name keyword is EXP or fld14, and the first four digits are 1800-3999 and the last two digits are 01-12, or the first two digits are 01-12. The four digits are a combination of 1800-3999.

7) The bank card verification code screening template can match the uppercase conversion with the column name keyword CvN, and contains 3 digits.

The specific types of the above target information screening templates can be adjusted according to specific application scenarios, and are not limited too much.

Optionally, in another optional embodiment of the present invention, the data construction structure of the target information screening template may include: the sequence number of the screening rule, the regular expression, the sensitivity threshold coefficient, the name and the column name keyword rule. Wherein, the above-mentioned sensitive threshold coefficient can be used to calculate the following preset ratio. The above regular expressions and column name keyword rules can be adjusted based on actual application scenarios, and are not limited too much.

S104, when the ratio of the number to the total number of field information under the target field type is greater than the preset ratio, obtain the position information of the field file under the target field type in the database, and obtain the first position information; the field file includes at least one field information .

Wherein, the above-mentioned field file may be a non-empty data table in the above-mentioned database that includes field information under the above-mentioned target field type. The above-mentioned first location information may include: the address of the server where the database is located, the database address, the database instance name, and the like.

Optionally, in an optional embodiment of the present invention, when the ratio of the number to the total number of field information under the target field type is greater than the preset ratio, the field information corresponding to the number may be determined as the target information screening template. Sensitive data corresponding to the corresponding filtering rules in .

S105. Search and obtain first target information from a database based on the first location information.

The present invention obtains the position information of the field information in the database by searching the field information matching the target information screening template from the sampling data table, and obtains the target information from the database based on the position information. Through the method of screening target information by field, the present invention can efficiently realize the screening of database information, so as to meet the needs of large-scale batch processing of database information.

Optionally, perform information detection on each field information under the target field type based on the target information screening template, specifically including:

When the target information screening template is keyword information, obtain the field name of the field information under the target field type; determine whether the field name contains keyword information; if so, detect the field information corresponding to the field name and the target information screening template match.

When the target information filter template is character combination information, it is judged whether the field information under the target field type contains character combination information; if so, it is detected that the field information under the target field type matches the target information filter template.

Optionally, the above method further includes:

When the ratio of the number to the total number of field information under the target field type is greater than 0 and less than the preset ratio, obtain the position information of the field information under the target field type in the database, and obtain the second position information;

The second target information is obtained by searching the database based on the second position information.

Optionally, in an optional embodiment of the present invention, after obtaining the second target information, a temporary result is generated according to the second target information, the field file where the field information corresponding to the second target information is located, and the field information. file and store the temporary result file.

Optionally, the above method further includes:

Generate a first information screening report table based on the first target information parameter; the first target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, and the sampling data The table name of the table, the field name of the field information matching the target information screening template, the number of field information matching the target information screening template, the name of the first target information, the preset ratio, the first position information one or more.

Generate a second information screening report table based on the second target information parameter; the second target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the sampling data The table name of the table, the field name of the field information that matches the target information screening template, the number of field information that matches the target information screening template, the name of the second target information, the preset ratio, and the second position information. one or more.

Optionally, the above method further includes:

When there is at least one record information in the sampling data table, look up and obtain the field type without field information from the sampling data table, and obtain the reference field type;

Generate a third information screening report table based on the third target information parameter; the third target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the sampling data One or more of the table name of the table, the field name corresponding to the reference field type, and the number of non-null record information;

When there is no record information in the sampling data table, a fourth information screening report table is generated based on the fourth target information parameter; the fourth target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, One or more of the user name corresponding to the sampling data table, the table name of the sampling data table, and the number of non-empty record information.

Optionally, the generation method of the sampling data table, specifically including:

Obtain all the data tables corresponding to the user name in the oracle database, and obtain the first data table set;

Select a data table in the first data table set, calculate the ratio of the storage size of a record information in the selected data table to the storage size of the selected data table, and obtain the storage ratio;

Determine the multiple of the sampling number, so that the sampling ratio calculated based on the multiple is within the preset sampling value range; the preset sampling value range is the sampling value range of the sampling function of the oracle database; the sampling ratio is the product of the multiple and the sampling number Multiply by the stored ratio;

Based on the sampling function of the oracle database, the record information of the multiple sampling quantity is collected from the selected data table, and the first sampling record information is obtained;

The sampling data table is generated based on the record information of the number of samples selected from the first sample record information.

Wherein, the above-mentioned user name is a user identification name that can modify the data table in the above-mentioned database.

Optionally, in an optional embodiment of the present invention, the above-mentioned specific implementation of obtaining all the data tables corresponding to the user names in the oracle database to obtain the first data table set may be:

Gets a user-mode list of ranges and user-name modifiable data tables.

According to the range of the user mode list, by querying the relational view between the user and the data table in the database, the data table range in each user mode in the data is obtained circularly. The data table range is the above-mentioned first data table set.

Optionally, in another optional embodiment of the present invention, the above-mentioned sampling function based on the oracle database collects the record information of the multiple sampling quantity from the selected data table, and the implementation process of obtaining the first sampling record information can be: Based on the sampling function, the data is sampled according to the above-mentioned sampling ratio, and then the first sampling record information in the obtained sampling data is determined as the above-mentioned first sampling record information. For example: when the number of samples is 5000, the first 5000 records are taken from the 10000 records in the data table as the above record information.

The above sampling function may be a sampling function sample(ratio) that comes with the oracle database.

Optionally, the generation method of the sampling data table, specifically including:

Obtain all the data tables corresponding to the user name in the mysql database, and obtain the second data table set;

Select a data table in the second data table set, derive the record information of the preset number from the selected data table based on the data export instruction of the mysql database, and obtain the second record information;

A sample data table is generated based on the second record information.

Corresponding to the embodiment of the above-mentioned database information screening method, the present invention also provides a database information screening system, as shown in Figure 2, comprising:

The data acquisition module 201 is used for acquiring the sampling data table. The sampling data table is generated based on the sampling information of the database.

The field determination module 202 is configured to, when the sampled data table has at least one piece of record information, search and obtain a field type with at least one field information from the sampled data table to obtain the target field type.

The information detection module 203 performs information detection on each field information under the target field type based on the target information screening template, and obtains the number of field information matching the target information screening template.

The first location determination module 204 is configured to obtain location information of the field files of the target field type in the database when the ratio of the number to the total number of field information of the target field type is greater than a preset ratio to obtain the first location information. The field file includes at least one field information.

The data extraction module 205 searches and obtains the first target information from the database based on the first position information.

Optionally, the above information detection module 203 is specifically set as:

When the target information filter template is keyword information, obtain the field name of the field information under the target field type. Determine whether the field name contains keyword information. If it contains, it is detected that the field information corresponding to the field name matches the target information filtering template.

When the target information filtering template is character combination information, it is judged whether the field information under the target field type contains character combination information. If it contains, it is detected that the field information under the target field type matches the target information filtering template.

Optionally, the above database information screening system further includes:

The second position determination module is configured to obtain the position information of the field information under the target field type in the database when the ratio of the number to the total number of field information under the target field type is greater than 0 and less than the preset ratio, and obtain the second position information.

The second target information is obtained by searching the database based on the second position information.

Optionally, the above database information screening system further includes:

The first report generation sub-module generates a first information screening report table based on the first target information parameter. The first target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the table name of the sampling data table, and the field matching the target information filtering template. One or more of the field name of the information, the number of field information matching the target information screening template, the name of the first target information, a preset ratio, and the first position information.

The second report generation sub-module generates a second information screening report table based on the second target information parameter. The second target information parameters are the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the table name of the sampling data table, and the field matching the target information filtering template One or more of the field name of the information, the number of field information matching the target information screening template, the name of the second target information, the preset ratio, and the second position information.

Optionally, the above-mentioned field determination module 202 is also set to:

When there is at least one record information in the sampling data table, look up and obtain the field type without field information in the sampling data table, and obtain the reference field type.

A third information screening report table is generated based on the third target information parameter. The third target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the table name of the sampling data table, the field name corresponding to the reference field type, One or more of the number of non-null record information.

When there is no record information in the sampling data table, a fourth information screening report table is generated based on the fourth target information parameter. The fourth target information parameter is the instance name of the database, the IP address of the database, the sampling batch to which the sampling data table belongs, the user name corresponding to the sampling data table, the table name of the sampling data table, and the number of non-empty record information. one or more.

Optionally, the above database information screening system further includes:

The data table generation module is used to obtain all the data tables corresponding to the user names in the oracle database, and obtain the first data table set.

A data table is selected from the first data table set, and the ratio of the storage size of a piece of record information in the selected data table to the storage size of the selected data table is calculated to obtain the storage ratio.

Determine the multiple of the sampling quantity, so that the sampling ratio calculated based on the multiple is within the preset sampling value range. The preset sampling value range is the sampling value range of the sampling function of the oracle database. The sampling ratio is the product of the multiple and the number of samples multiplied by the storage ratio.

Based on the sampling function of the oracle database, the record information of the multiple sampling quantity is collected from the selected data table to obtain the first sampling record information.

The sampling data table is generated based on the record information of the number of samples selected from the first sample record information.

Optionally, the above data table generation module is also set to:

Obtain all the data tables corresponding to the user name in the mysql database, and obtain the second data table set.

A data table is selected from the second data table set, and a preset number of record information is derived from the selected data table based on a data export instruction of the mysql database to obtain second record information.

A sample data table is generated based on the second record information.

An embodiment of the present invention provides a computer-readable storage medium, where a program is stored on the computer-readable storage medium, and when the program is executed by a processor, any one of the above database information screening methods is implemented.

An embodiment of the present invention provides an electronic device, including:

at least one processor, and at least one memory and bus connected to the processor;

The processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the database information screening method as described above.

The above electronic device, as shown in FIG. 3 , the electronic device 30 includes at least one processor 301, and at least one memory 302 and a bus 303 connected to the processor 301; Communication; the processor 301 is used for calling the program instructions in the memory 302 to execute the above-mentioned database information screening method. The electronic device herein can be a server, a PC, a PAD, a mobile phone, and the like.

The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program initialized with the steps included in the above database information screening method.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, systems and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

In a typical configuration, a device includes one or more processors (CPUs), memory, and a bus. Devices may also include input/output interfaces, network interfaces, and the like.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM), the memory including at least one memory chip. Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It will be appreciated by those skilled in the art that the embodiments of the present application may be provided as a method, a system or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture or apparatus that includes the element.

Each embodiment in this specification is described in a related manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the partial description of the method embodiment.

The above are merely examples of the present application, and are not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims (10)

1. A database information screening method is characterized by comprising the following steps:

acquiring a sampling data table; the sampling data table is generated based on sampling information of a database;

when at least one piece of record information exists in the sampling data table, searching and obtaining a field type with at least one piece of field information from the sampling data table to obtain a target field type;

performing information detection on each field information under the target field type based on a target information screening template to obtain the number of the field information matched with the target information screening template;

when the ratio of the number to the total number of the field information under the target field type is greater than a preset ratio, acquiring the position information of the field file under the target field type in the database to obtain first position information; the field file comprises at least one field information;

and searching the database based on the first position information to obtain first target information.

2. The database information screening method according to claim 1, wherein the information detection of each field information under the target field type based on the target information screening template specifically includes:

when the target information screening template is the keyword information, acquiring the field name of the field information under the target field type; judging whether the field name contains the keyword information or not; if yes, detecting to obtain that the field information corresponding to the field name is matched with the target information screening template;

when the target information screening template is character combination information, judging whether the field information under the target field type contains the character combination information; and if so, detecting that the field information under the target field type is matched with the target information screening template.

3. The database information screening method according to claim 1, further comprising:

when the ratio of the number to the total number of the field information under the target field type is greater than 0 and smaller than a preset ratio, acquiring the position information of the field information under the target field type in the database to obtain second position information;

and searching the database to obtain second target information based on the second position information.

4. The database information screening method according to claim 3, further comprising:

generating a first information screening report table based on the first target information parameter; the first target information parameter is one or more of an instance name of the database, an IP address of the database, a sampling batch to which the sampling data table belongs, a user name corresponding to the sampling data table, a table name of the sampling data table, a field name of field information matched with the target information screening template, the number of the field information matched with the target information screening template, a name of first target information, a preset ratio and first position information;

generating a second information screening report table based on the second target information parameter; the second target information parameter is one or more of an instance name of the database, an IP address of the database, a sampling batch to which the sampling data table belongs, a user name corresponding to the sampling data table, a table name of the sampling data table, a field name of field information matched with the target information screening template, the number of field information matched with the target information screening template, a name of second target information, a preset ratio and second position information.

5. The database information screening method according to claim 1, further comprising:

when at least one piece of record information exists in the sampling data table, searching and obtaining a field type without field information from the sampling data table to obtain a reference field type;

generating a third information screening report table based on the third target information parameter; the third target information parameter is one or more of an instance name of the database, an IP address of the database, a sampling batch to which the sampling data table belongs, a user name corresponding to the sampling data table, a table name of the sampling data table, a field name corresponding to the reference field type, and the number of non-empty record information;

when the sampling data table does not have the record information, generating a fourth information screening report table based on the fourth target information parameter; the fourth target information parameter is one or more of an instance name of the database, an IP address of the database, a sampling batch to which the sampling data table belongs, a user name corresponding to the sampling data table, a table name of the sampling data table, and the number of non-empty record information.

6. The database information screening method according to claim 1, wherein the method for generating the sampling data table specifically includes:

acquiring all data tables corresponding to the user name in an oracle database to obtain a first data table set;

selecting a data table from the first data table set, and calculating the ratio of the storage size of a piece of recording information in the selected data table to the storage size of the selected data table to obtain a storage ratio;

determining a multiple of the sampling number so that a sampling proportion calculated based on the multiple is within a preset sampling value range; the preset sampling value range is the sampling value range of the sampling function of the oracle database; the sampling proportion is the product of the multiple and the sampling number multiplied by the storage ratio value;

collecting record information of the multiple sampling number from the selected data table based on the sampling function of the oracle database to obtain first sampling record information;

generating the sampling data table based on the record information of the sampling number selected from the first sampling record information.

7. The database information screening method according to claim 1, wherein the method for generating the sampling data table specifically includes:

acquiring all data tables corresponding to the user name in the mysql database to obtain a second data table set;

selecting a data table from the second data table set, and deriving a preset number of record information from the selected data table based on a data derivation instruction of the mysql database to obtain second record information;

and generating the sampling data table based on the second record information.

8. A database information screening system, comprising:

the data acquisition module is used for acquiring a sampling data table; the sampling data table is generated based on sampling information of a database;

the field determining module is used for searching and obtaining a field type with at least one piece of field information from the sampling data table to obtain a target field type when at least one piece of record information exists in the sampling data table;

the information detection module is used for carrying out information detection on each field information under the target field type based on a target information screening template to obtain the number of the field information matched with the target information screening template;

the first position determining module is used for acquiring the position information of the field file in the target field type in the database when the ratio of the number to the total number of the field information in the target field type is greater than a preset ratio to obtain first position information; the field file comprises at least one field information;

and the data extraction module is used for searching and obtaining first target information from the database based on the first position information.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program which, when executed by a processor, implements the database information screening method according to any one of claims 1 to 7.

10. An electronic device, comprising:

at least one processor, and at least one memory, bus connected with the processor;

the processor and the memory complete mutual communication through the bus; the processor is used for calling the program instructions in the memory to execute the database information screening method of any one of claims 1 to 7.

Images