CN118332603B - Data isolation and safe sharing method and system for big data environment - Google Patents
Data isolation and safe sharing method and system for big data environment Download PDFInfo
- Publication number
- CN118332603B CN118332603B CN202410748382.XA CN202410748382A CN118332603B CN 118332603 B CN118332603 B CN 118332603B CN 202410748382 A CN202410748382 A CN 202410748382A CN 118332603 B CN118332603 B CN 118332603B
- Authority
- CN
- China
- Prior art keywords
- data
- grouping
- information
- encryption code
- verification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a data isolation and safe sharing method and system for a big data environment, which relate to the technical field of data processing, and the method comprises the following steps: grouping data attribute information based on the task and attribute type set to generate first and second grouping results; according to the second grouping result, the data to be isolated is stored in blocks, static verification rules are configured, task-level data index trees are constructed on the basis of the block storage result of the first grouping result, and dynamic verification rules are configured; when the retrieval tag is a data tag, carrying out data sharing based on a static verification rule; and when the label is of a task type, carrying out data sharing based on the dynamic verification rule and the task-level data index tree. The application can solve the technical problems that the existing data storage and calling modes using the data type as a label have low data calling efficiency and poor data sharing security and cannot meet the current data processing requirement, and can achieve the effect of meeting the data processing requirement in a big data environment.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a data isolation and safe sharing method and system for a big data environment.
Background
In the existing data processing, the common data isolation and calling modes are to store data by taking the data type as a label, and when a user performs scheduling for a certain task, the user needs to perform multi-party data verification, so that the data calling efficiency is low, and meanwhile, when the data calling verification is performed, the same data verification mode is not reasonable, so that the security of data sharing is poor.
At present, due to the diversification of data types and the large data volume in a big data environment, the existing data storage and calling modes using the data types as tags have the technical problems that the data calling efficiency is low, the data sharing security is poor, and the current data processing requirements cannot be met.
Disclosure of Invention
The application aims to provide a data isolation and safe sharing method and system for a big data environment, which are used for solving the technical problems that the existing data storage and calling modes using data types as labels have lower data calling efficiency and poorer data sharing safety and cannot meet the current data processing requirements due to the diversification of the data types and larger data quantity in the big data environment.
In view of the above, the present application provides a data isolation and secure sharing method and system for a big data environment.
In a first aspect, the present application provides a data isolation and secure sharing method for a big data environment, the method being implemented by a data isolation and secure sharing system for a big data environment, wherein the method includes: obtaining basic information of data to be isolated, wherein the basic information of the data to be isolated comprises data attribute information; grouping the data attribute information based on a task type set to generate a first data attribute grouping result; grouping the data attribute information based on an attribute type set to generate a second data attribute grouping result; traversing the first data attribute grouping result and configuring a dynamic verification rule; traversing the second data attribute grouping result and configuring a static verification rule; according to the second data attribute grouping result, the data to be isolated is stored in blocks, the static verification rule is configured, the task-level data index tree is constructed based on the block storage result of the first data attribute grouping result, and the dynamic verification rule is configured; when the retrieval tag is a data tag, matching the data attribute type to carry out data sharing based on the static verification rule; and when the retrieval tag is of a task type, carrying out data sharing based on the dynamic verification rule and the task-level data index tree.
In a second aspect, the present application further provides a data isolation and secure sharing system for a big data environment, for performing a data isolation and secure sharing method for a big data environment according to the first aspect, wherein the system comprises: the data base information to be isolated is used for obtaining data base information to be isolated, wherein the data base information to be isolated comprises data attribute information; the task type grouping module is used for grouping the data attribute information based on a task type set to generate a first data attribute grouping result; the attribute type grouping module is used for grouping the data attribute information based on an attribute type set to generate a second data attribute grouping result; the dynamic verification rule configuration module is used for traversing the first data attribute grouping result and configuring a dynamic verification rule; the static verification rule configuration module is used for traversing the second data attribute grouping result and configuring a static verification rule; the data storage module is used for storing the data to be isolated in blocks according to the second data attribute grouping result, configuring the static verification rule, constructing a task-level data index tree based on the block storage result of the first data attribute grouping result, and configuring the dynamic verification rule; the static verification module is used for carrying out data sharing on the basis of the static verification rule when the retrieval tag is a data tag and the matching data attribute type; and the dynamic verification module is used for carrying out data sharing based on the dynamic verification rule and the task-level data index tree when the retrieval label is of a task type.
One or more technical schemes provided by the application have at least the following technical effects or advantages:
Obtaining basic information of data to be isolated, wherein the basic information of the data to be isolated comprises data attribute information; grouping the data attribute information based on a task type set to generate a first data attribute grouping result; grouping the data attribute information based on an attribute type set to generate a second data attribute grouping result; traversing the first data attribute grouping result and configuring a dynamic verification rule; traversing the second data attribute grouping result and configuring a static verification rule; according to the second data attribute grouping result, the data to be isolated is stored in blocks, the static verification rule is configured, the task-level data index tree is constructed based on the block storage result of the first data attribute grouping result, and the dynamic verification rule is configured; when the retrieval tag is a data tag, matching the data attribute type to carry out data sharing based on the static verification rule; and when the retrieval tag is of a task type, carrying out data sharing based on the dynamic verification rule and the task-level data index tree. That is, by grouping data attribute information based on task type and attribute type and performing data storage according to the data grouping result, an adapted verification rule is set at the same time; namely, aiming at the grouping result corresponding to the task type, configuring a dynamic verification rule, constructing a task-level data index tree to facilitate data retrieval, and aiming at the attribute type grouping result, configuring a static verification rule; then when data call is carried out, when the data attribute is taken as a retrieval tag to carry out retrieval, carrying out data sharing based on a static verification rule; when the task type is used as a retrieval tag for retrieval, data sharing is performed based on the dynamic verification rule and the task-level data index tree, so that multiple times of verification during data scheduling by the task tag can be avoided, the data calling efficiency is improved, meanwhile, the dynamic verification rule is set for calling verification in a targeted mode, and the safety of data sharing can be improved; the method can improve the flexibility and accuracy of data storage mode, calling mode and verification rule setting, thereby improving the data storage and calling efficiency, improving the safety of data sharing and achieving the technical effect of meeting the data processing requirement in a big data environment.
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent. It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the application or the technical solutions of the prior art, the following brief description will be given of the drawings used in the description of the embodiments or the prior art, it being obvious that the drawings in the description below are only exemplary and that other drawings can be obtained from the drawings provided without the inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for data isolation and secure sharing for big data environments according to the present application;
FIG. 2 is a schematic flow chart of a dynamic verification rule configuration in a data isolation and secure sharing method for big data environment according to the present application;
FIG. 3 is a schematic diagram of a data isolation and secure sharing system for large data environments according to the present application.
Reference numerals illustrate:
The system comprises a data basic information obtaining module 11 to be isolated, a task type grouping module 12, an attribute type grouping module 13, a dynamic verification rule configuration module 14, a static verification rule configuration module 15, a data storage module 16, a static verification module 17 and a dynamic verification module 18.
Detailed Description
The application provides a data isolation and safe sharing method and system for a big data environment, which solve the technical problems that the existing data storage and calling modes using data types as labels have lower data calling efficiency and poorer data sharing safety and cannot meet the current data processing requirements due to the diversification of data types and larger data quantity in the big data environment. The method can improve the flexibility and accuracy of data storage mode, calling mode and verification rule setting, thereby improving the data storage and calling efficiency, improving the safety of data sharing and achieving the technical effect of meeting the data processing requirement in a big data environment.
In the following, the technical solutions of the present application will be clearly and completely described with reference to the accompanying drawings, and it should be understood that the described embodiments are only some embodiments of the present application, but not all embodiments of the present application, and that the present application is not limited by the exemplary embodiments described herein. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application. It should be further noted that, for convenience of description, only some, but not all of the drawings related to the present application are shown.
Referring to fig. 1, the present application provides a data isolation and secure sharing method for a big data environment, wherein the method is applied to a data isolation and secure sharing system for a big data environment, and the method specifically includes the following steps:
step one: and obtaining basic information of the data to be isolated, wherein the basic information of the data to be isolated comprises data attribute information.
In particular, first, data base information to be isolated is obtained, wherein the data base information to be isolated comprises data attribute information, and the data attribute is description or identification about data characteristics, describes certain characteristics or attributes of data, and in data management and analysis, the data attribute information is critical to understanding and processing the data, and generally comprises information such as data type, data length, value range, data item name, data item number and the like.
Step two: and based on the task type set, grouping the data attribute information to generate a first data attribute grouping result.
Step three: and grouping the data attribute information based on the attribute type set to generate a second data attribute grouping result.
Specifically, a task type set and an attribute type set are then acquired, wherein the task type set can set task types according to specific application scenarios or service requirements, such as: user management tasks, sales tasks, production tasks, etc.; the attribute type set comprises a plurality of data attributes, which can be defined according to actual conditions, and comprises data types, service characteristics, use frequency and the like, wherein the data types comprise integers, floating point numbers, character strings, dates, boolean values and the like. Then grouping the data attribute information according to the task type set, namely grouping the data attribute information corresponding to the same task type into one type to obtain a first data attribute grouping result; and on the other hand, based on the attribute type set, grouping the data attribute information, and grouping the data attribute information belonging to the same attribute type into one type to obtain a second data attribute grouping result.
Step four: and traversing the first data attribute grouping result and configuring a dynamic verification rule.
Step five: and traversing the second data attribute grouping result and configuring a static verification rule.
Specifically, performing traversal analysis on the first data attribute grouping result, and configuring an adaptive dynamic verification rule according to the data attribute and the task type of each first data attribute grouping, wherein the dynamic verification rule comprises requirements on data format, data range, data uniqueness and the like, such as: the matched verification rules can be set according to the importance of the task type and the data privacy, wherein the higher the importance of the task type is, the higher the data privacy is, and the higher the corresponding verification level is. And then performing traversal analysis on the second data attribute grouping result, configuring an adaptive static verification rule, namely writing a corresponding static verification rule for each data attribute or data attribute combination, such as: whether the data is an integer, whether the string length meets the requirements, and the like.
Step six: and according to the second data attribute grouping result, carrying out block storage on the data to be isolated, configuring the static verification rule, constructing a task-level data index tree based on the block storage result of the first data attribute grouping result, and configuring the dynamic verification rule.
And then according to the second data attribute grouping result, carrying out block storage on the data to be isolated, namely carrying out block storage on the data to be isolated according to the attribute type, and configuring the static verification rule, wherein the block storage is a data storage technology for dispersedly storing the data on a plurality of physical devices or servers, the data is divided into a plurality of data blocks by utilizing the principle of block storage, and the data blocks are stored on different devices, when the data is required to be accessed, the data blocks can be read from different devices according to the requirement, and the method has the advantages of high reliability, high availability, high expansibility, high performance and the like, and is suitable for application scenes requiring massive data processing and requiring high performance and high reliability. On the other hand, based on the block storage result of the first data attribute grouping result, constructing a task-level data index tree and configuring the dynamic verification rule, firstly, selecting a proper index type, such as a B tree, a B+ tree, a hash index and the like, according to an application scene and requirements; the hierarchical structure of the index tree and the filling data are then set according to the task type, such as: it is determined which data attributes or fields can be used as nodes of the tree and the hierarchy and association relationship between them, and the attributes which are frequently queried need to be selected as upper nodes in consideration of query performance. By constructing the task-level data index tree, the data calling efficiency and accuracy in the task type based retrieval can be improved.
Step seven: and when the retrieval tag is a data tag, the matching data attribute type performs data sharing based on the static verification rule.
Step eight: and when the retrieval tag is of a task type, carrying out data sharing based on the dynamic verification rule and the task-level data index tree.
Specifically, when data attribute is taken as a retrieval tag to carry out data calling, firstly matching the data attribute type, then calling a corresponding static verification rule according to an attribute type matching result to carry out data calling verification, and carrying out data sharing on calling data after verification is passed; when the task type is used as a retrieval tag to carry out data call, firstly, carrying out data call verification according to the dynamic verification rule, and after the dynamic verification is passed, carrying out data retrieval by using the task-level data index tree, and simultaneously carrying out data sharing on the retrieved data. The method can improve the flexibility and accuracy of data storage mode, calling mode and verification rule setting, thereby improving the data storage and calling efficiency, improving the safety of data sharing and achieving the technical effect of meeting the data processing requirement in a big data environment.
The data isolation and safe sharing method for the big data environment is applied to a data isolation and safe sharing system for the big data environment, and can solve the technical problems that the existing data storage and calling modes using the data types as labels have lower data calling efficiency and poorer data sharing safety and cannot meet the current data processing requirements due to the fact that the data types are diversified and the data quantity is large in the big data environment. First, obtaining basic information of data to be isolated, wherein the basic information of the data to be isolated comprises data attribute information. And then, grouping the data attribute information based on the task type set to generate a first data attribute grouping result. And then, grouping the data attribute information based on the attribute type set to generate a second data attribute grouping result. Next, traversing the first data attribute grouping result, and configuring a dynamic verification rule. And traversing the second data attribute grouping result and configuring a static verification rule. Further, according to the second data attribute grouping result, the data to be isolated is stored in blocks, the static verification rule is configured, based on the first data attribute grouping result, a task-level data index tree is constructed, and the dynamic verification rule is configured. And finally, when the retrieval tag is a data tag, matching the data attribute type and carrying out data sharing based on the static verification rule. And when the retrieval tag is of a task type, carrying out data sharing based on the dynamic verification rule and the task-level data index tree. Grouping data attribute information based on task types and attribute types, storing data according to data grouping results, and setting an adaptive verification rule; namely, aiming at the grouping result corresponding to the task type, configuring a dynamic verification rule, constructing a task-level data index tree to facilitate data retrieval, and aiming at the attribute type grouping result, configuring a static verification rule; then when data call is carried out, when the data attribute is taken as a retrieval tag to carry out retrieval, carrying out data sharing based on a static verification rule; when the task type is used as a retrieval tag for retrieval, data sharing is performed based on the dynamic verification rule and the task-level data index tree, so that multiple times of verification during data scheduling by the task tag can be avoided, the data calling efficiency is improved, meanwhile, the dynamic verification rule is set for calling verification in a targeted mode, and the safety of data sharing can be improved; the method can improve the flexibility and accuracy of data storage mode, calling mode and verification rule setting, thereby improving the data storage and calling efficiency, improving the safety of data sharing and achieving the technical effect of meeting the data processing requirement in a big data environment.
Further, traversing the first data attribute grouping result, and configuring a dynamic verification rule, as shown in fig. 2, a fourth step of the present application includes:
And obtaining a first grouping task type label of the grouping result of the first data attribute.
And configuring a first encryption code for the first grouping task type label according to an encryption database.
And carrying out search isolation encryption on the first grouping task type tag according to the first encryption code, and sending a first decryption code corresponding to the first encryption code to an authorized user through a short message.
And updating the first encryption code through the encryption database every time a preset updating period is met, and generating a second encryption code.
Specifically, first, randomly selecting a first grouping task type label from the first data attribute grouping result, wherein the first grouping task type label is any one of a plurality of task type labels; and then configuring a first encryption code for the first grouping task type tag based on an encryption database, wherein the encryption database comprises a plurality of encryption codes, the encryption database can be set according to actual conditions, and the first encryption code is any encryption code in the encryption database. And then performing search isolation encryption on the first packet task type tag according to the first encryption code, wherein the search isolation encryption is to allow a search operation to be performed on encrypted data, and meanwhile keep isolation and encryption states of the data, such as: homomorphic encryption, differential privacy, etc.; and sending a first decryption code corresponding to the first encryption code to an authorized user through a short message. A preset update period is configured, wherein the preset update period is a self-defined time period and can be set according to actual conditions, for example: 24 hours; and updating the first encryption code through the encryption database to obtain a second encryption code when the preset updating period is met.
Further, the application also comprises the following steps:
And constructing a first high-dimensional coordinate according to the first encryption code.
And randomly selecting a to-be-selected encryption code from the encryption database, and constructing a second high-dimensional coordinate.
And calculating characteristic distance information of the first high-dimensional coordinates and the second high-dimensional coordinates.
And counting the selected frequency information of the encryption code to be selected in a preset time zone.
And when the characteristic distance information is larger than or equal to a characteristic distance threshold value and the selected frequency information is smaller than or equal to a selected frequency threshold value, setting the encryption code to be selected as the second encryption code.
Specifically, the method of obtaining the second encryption code includes that first, a first high-dimensional coordinate is constructed according to the first encryption code, for example: a high-dimensional space can be defined, wherein each dimension represents a feature type, and then the first encryption code is mapped into the high-dimensional space to obtain a first high-dimensional coordinate; and randomly selecting a to-be-selected encryption code from the encryption database, wherein the to-be-selected encryption code is any encryption code except the first encryption code in the encryption database, and performing high-dimensional mapping on the to-be-selected encryption code to construct a second high-dimensional coordinate.
Then in a high-dimensional space, calculating characteristic distance information of the first high-dimensional coordinate and the second high-dimensional coordinate, wherein the characteristic distance is used for measuring similarity or difference between the first high-dimensional coordinate and the second high-dimensional coordinate, the distance calculation can be carried out through Euclidean algorithm or Manhattan algorithm, and the characteristic distance information is inversely proportional to the similarity between the first high-dimensional coordinate and the second high-dimensional coordinate, namely, the smaller the characteristic distance is, the higher the similarity is; and counting the selected frequency information of the to-be-selected encryption code in a preset time zone, wherein the preset time zone is a longer time period and can be set according to actual conditions, such as: for 30 days.
Acquiring a characteristic distance threshold value and a selected frequency threshold value, wherein the characteristic distance threshold value and the selected frequency threshold value can be set according to actual conditions; and then respectively judging the characteristic distance information and the selected frequency information according to the characteristic distance threshold and the selected frequency threshold, and setting the encryption code to be selected as the second encryption code when the characteristic distance information is larger than or equal to the characteristic distance threshold and the selected frequency information is smaller than or equal to the selected frequency threshold. By evaluating the encryption codes to be selected from the two layers of similarity and selection frequency, the accuracy of the evaluation of the encryption codes to be selected can be improved, and the accuracy and reliability of the setting of the second encryption code are improved.
Further, the application also comprises the following steps:
extracting a neighborhood encryption code set with the feature distance smaller than the feature distance threshold value from the encryption database.
And counting the neighborhood encryption code selection frequency set of the neighborhood encryption code set in the preset time zone, and calculating the average value of the neighborhood encryption code selection frequency set to obtain neighborhood selection frequency information.
And when the neighborhood selected frequency information is smaller than or equal to a selected frequency threshold value, setting the to-be-selected encryption code as the second encryption code.
Specifically, firstly, extracting a plurality of encryption codes with the feature distance smaller than the feature distance threshold value from the encryption database, setting the plurality of encryption codes as neighborhood encryption codes, and constructing a neighborhood encryption code set; counting a neighborhood encryption code selection frequency set of the neighborhood encryption code set in the preset time zone, carrying out selection frequency average value calculation on the neighborhood encryption code selection frequency set, and setting an average value calculation result as neighborhood selection frequency information; when the neighborhood selected frequency information is smaller than or equal to the selected frequency threshold, the to-be-selected encryption code is set to be the second encryption code, so that the accuracy and reliability of setting of the second encryption code can be further improved.
And carrying out search and isolation encryption on the first grouping task type label according to the second encryption code, and sending a second decryption code corresponding to the second encryption code to an authorized user through a short message.
Specifically, the first grouping task type label is further searched and isolated for encryption according to the second encryption code, and after encryption is completed, a second decryption code corresponding to the second encryption code is sent to an authorized user through a short message, so that the security of data encryption storage is improved.
Further, when the search tag is a data tag, the matching data attribute type performs data sharing based on the static verification rule, and the seventh step of the application includes:
and when the retrieval tag is a data tag, activating the static verification rule according to the data attribute type to verify the static input verification code, and generating static verification result information.
And when the static verification result information passes, sending the data information corresponding to the data attribute type to a request user side for sharing.
And when the static verification result information does not pass, generating a repeated verification instruction, and sending the repeated verification instruction to the request user side for repeated verification.
Specifically, when the search tag is a data tag, firstly, the data tag is matched with a data attribute type, and then the static verification rule is activated according to the data attribute type to verify the static input verification code, so that static verification result information is obtained. When the static verification result information passes, sending the data information corresponding to the data attribute type to a request user side for sharing; and when the static verification result information is not passed, generating a repeated verification instruction, sending the repeated verification instruction to the request user terminal for repeated verification, and performing request locking on the request user terminal until the repeated verification number threshold is not passed.
Further, when the search tag is a task type, data sharing is performed based on the dynamic verification rule and the task level data index tree, and the step eight of the present application includes:
And when the search tag is of a task type, activating the dynamic verification rule to verify the dynamic input verification code, and generating dynamic verification result information.
And when the dynamic verification result information is passed, indexing is carried out on the task-level data index tree based on the task type, a cross-block extraction data packet is obtained and sent to a request user side for sharing.
And when the dynamic verification result information does not pass, generating a repeated verification instruction, and sending the repeated verification instruction to the request user side for repeated verification.
Specifically, when the search tag is of a task type, the dynamic verification rule is activated to verify the dynamic input verification code, and dynamic verification result information is generated. When the dynamic verification result information is passing, indexing is carried out on the task-level data index tree based on the task type, and a cross-block extraction data packet is obtained according to a data index result and is sent to a request user side for sharing; and when the dynamic verification result information is not passed, generating a repeated verification instruction, and sending the repeated verification instruction to the request user terminal for repeated verification, and if the dynamic verification result information is not passed when the dynamic verification result information is not passed, and carrying out request locking on the request user terminal.
Further, the application also comprises the following steps:
and when the repeated verification times are greater than or equal to the repeated verification times threshold, carrying out request locking on the request user terminal for a preset time.
Specifically, a repeated verification number threshold is configured, and the repeated verification number threshold can be set according to actual conditions; when static verification or dynamic verification is performed, if the number of repeated verification is greater than or equal to the threshold value of the number of repeated verification, and the number of repeated verification is not yet passed, the request is performed on the request user terminal for locking a preset time length, wherein the preset time length can be set in a user-defined manner according to actual conditions, such as: 10 minutes.
In summary, the data isolation and secure sharing method for big data environment provided by the application has the following technical effects:
1. grouping data attribute information based on task types and attribute types, storing data according to data grouping results, and setting an adaptive verification rule; namely, aiming at the grouping result corresponding to the task type, configuring a dynamic verification rule, constructing a task-level data index tree to facilitate data retrieval, and aiming at the attribute type grouping result, configuring a static verification rule; then when data call is carried out, when the data attribute is taken as a retrieval tag to carry out retrieval, carrying out data sharing based on a static verification rule; when the task type is used as a retrieval tag for retrieval, data sharing is performed based on the dynamic verification rule and the task-level data index tree, so that multiple times of verification during data scheduling by the task tag can be avoided, the data calling efficiency is improved, meanwhile, the dynamic verification rule is set for calling verification in a targeted mode, and the safety of data sharing can be improved; the method can improve the flexibility and accuracy of data storage mode, calling mode and verification rule setting, thereby improving the data storage and calling efficiency, improving the safety of data sharing and achieving the technical effect of meeting the data processing requirement in a big data environment.
2. By evaluating the encryption codes to be selected from the two layers of similarity and selection frequency, the accuracy of the evaluation of the encryption codes to be selected can be improved, and the accuracy and reliability of the setting of the second encryption code are improved.
In a second embodiment, based on the same inventive concept as the data isolation and secure sharing method for a big data environment in the foregoing embodiment, the present application further provides a data isolation and secure sharing system for a big data environment, referring to fig. 3, where the system includes:
The data to be isolated basic information obtaining module 11 is configured to obtain data to be isolated basic information, where the data to be isolated basic information includes data attribute information.
The task type grouping module 12 is configured to group the data attribute information based on a task type set, and generate a first data attribute grouping result.
And the attribute type grouping module 13 is used for grouping the data attribute information based on the attribute type set to generate a second data attribute grouping result.
The dynamic verification rule configuration module 14 is configured to traverse the first data attribute grouping result and configure a dynamic verification rule.
And the static verification rule configuration module 15 is used for traversing the second data attribute grouping result and configuring a static verification rule.
And the data storage module 16 is configured to store the data to be isolated in blocks according to the second data attribute grouping result, configure the static verification rule, construct a task-level data index tree based on the block storage result of the first data attribute grouping result, and configure the dynamic verification rule.
And the static verification module 17 is used for carrying out data sharing on the basis of the static verification rule when the retrieval tag is a data tag and the matching data attribute type.
And the dynamic verification module 18 is used for carrying out data sharing based on the dynamic verification rule and the task level data index tree when the retrieval label is of a task type.
Further, the dynamic verification rule configuration module 14 in the system is further configured to:
And obtaining a first grouping task type label of the grouping result of the first data attribute.
And configuring a first encryption code for the first grouping task type label according to an encryption database.
And carrying out search isolation encryption on the first grouping task type tag according to the first encryption code, and sending a first decryption code corresponding to the first encryption code to an authorized user through a short message.
And updating the first encryption code through the encryption database every time a preset updating period is met, and generating a second encryption code.
And carrying out search and isolation encryption on the first grouping task type label according to the second encryption code, and sending a second decryption code corresponding to the second encryption code to an authorized user through a short message.
Further, the dynamic verification rule configuration module 14 in the system is further configured to:
And constructing a first high-dimensional coordinate according to the first encryption code.
And randomly selecting a to-be-selected encryption code from the encryption database, and constructing a second high-dimensional coordinate.
And calculating characteristic distance information of the first high-dimensional coordinates and the second high-dimensional coordinates.
And counting the selected frequency information of the encryption code to be selected in a preset time zone.
And when the characteristic distance information is larger than or equal to a characteristic distance threshold value and the selected frequency information is smaller than or equal to a selected frequency threshold value, setting the encryption code to be selected as the second encryption code.
Further, the dynamic verification rule configuration module 14 in the system is further configured to:
extracting a neighborhood encryption code set with the feature distance smaller than the feature distance threshold value from the encryption database.
And counting the neighborhood encryption code selection frequency set of the neighborhood encryption code set in the preset time zone, and calculating the average value of the neighborhood encryption code selection frequency set to obtain neighborhood selection frequency information.
And when the neighborhood selected frequency information is smaller than or equal to a selected frequency threshold value, setting the to-be-selected encryption code as the second encryption code.
Further, the static verification module 17 in the system is also configured to:
and when the retrieval tag is a data tag, activating the static verification rule according to the data attribute type to verify the static input verification code, and generating static verification result information.
And when the static verification result information passes, sending the data information corresponding to the data attribute type to a request user side for sharing.
And when the static verification result information does not pass, generating a repeated verification instruction, and sending the repeated verification instruction to the request user side for repeated verification.
Further, the dynamic verification module 18 in the system is also configured to:
And when the search tag is of a task type, activating the dynamic verification rule to verify the dynamic input verification code, and generating dynamic verification result information.
And when the dynamic verification result information is passed, indexing is carried out on the task-level data index tree based on the task type, a cross-block extraction data packet is obtained and sent to a request user side for sharing.
And when the dynamic verification result information does not pass, generating a repeated verification instruction, and sending the repeated verification instruction to the request user side for repeated verification.
Further, the dynamic verification module 18 in the system is also configured to:
and when the repeated verification times are greater than or equal to the repeated verification times threshold, carrying out request locking on the request user terminal for a preset time.
Various embodiments in the present disclosure are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, so that a data isolation and security sharing method and specific example for a big data environment in the foregoing embodiment are equally applicable to a data isolation and security sharing system for a big data environment in the present embodiment, and by the foregoing detailed description of a data isolation and security sharing method for a big data environment, those skilled in the art can clearly know a data isolation and security sharing system for a big data environment in the present embodiment, so that the description is omitted herein for brevity. For the system disclosed in the embodiment, since the system corresponds to the method disclosed in the embodiment, the description is simpler, and the relevant points refer to the description of the method section.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the present application and the equivalent techniques thereof, the present application is also intended to include such modifications and variations.
Claims (5)
1. A method of data isolation and secure sharing for a big data environment, comprising:
obtaining basic information of data to be isolated, wherein the basic information of the data to be isolated comprises data attribute information;
grouping the data attribute information based on a task type set to generate a first data attribute grouping result;
grouping the data attribute information based on an attribute type set to generate a second data attribute grouping result;
Traversing the first data attribute grouping result and configuring a dynamic verification rule;
traversing the second data attribute grouping result and configuring a static verification rule;
According to the second data attribute grouping result, the data to be isolated is stored in blocks, the static verification rule is configured, the task-level data index tree is constructed based on the block storage result of the first data attribute grouping result, and the dynamic verification rule is configured;
When the retrieval tag is a data tag, matching the data attribute type to carry out data sharing based on the static verification rule;
When the retrieval tag is of a task type, carrying out data sharing based on the dynamic verification rule and the task-level data index tree;
traversing the first data attribute grouping result, configuring a dynamic verification rule, including:
Obtaining a first grouping task type label of the first data attribute grouping result;
According to an encryption database, a first encryption code is configured for the first grouping task type label;
searching, isolating and encrypting the first grouping task type tag according to the first encryption code, and sending a first decryption code corresponding to the first encryption code to an authorized user through a short message;
Each time a preset updating period is met, updating the first encryption code through the encryption database to generate a second encryption code;
Searching, isolating and encrypting the first grouping task type tag according to the second encryption code, and sending a second decryption code corresponding to the second encryption code to an authorized user through a short message;
Updating the first encryption code through the encryption database to generate a second encryption code, including:
constructing a first high-dimensional coordinate according to the first encryption code;
randomly selecting a to-be-selected encryption code from the encryption database to construct a second high-dimensional coordinate;
calculating characteristic distance information of the first high-dimensional coordinates and the second high-dimensional coordinates;
Counting the selected frequency information of the encryption code to be selected in a preset time zone;
When the characteristic distance information is larger than or equal to a characteristic distance threshold value and the selected frequency information is smaller than or equal to a selected frequency threshold value, setting the encryption code to be selected as the second encryption code;
When the feature distance information is greater than or equal to a feature distance threshold and the selected frequency information is less than or equal to a selected frequency threshold, setting the to-be-selected encryption code as the second encryption code, and further including:
Extracting a neighborhood encryption code set with the characteristic distance from the encryption database to the to-be-selected encryption code being smaller than the characteristic distance threshold value;
Counting a neighborhood encryption code selection frequency set of the neighborhood encryption code set in the preset time zone, and calculating the average value of the neighborhood encryption code selection frequency set to obtain neighborhood selection frequency information;
And when the neighborhood selected frequency information is smaller than or equal to a selected frequency threshold value, setting the to-be-selected encryption code as the second encryption code.
2. The method of claim 1, wherein matching data attribute types for data sharing based on the static validation rules when the search tag is a data tag comprises:
When the retrieval tag is a data tag, activating the static verification rule according to the data attribute type to verify the static input verification code, and generating static verification result information;
when the static verification result information passes, sending the data information corresponding to the data attribute type to a request user side for sharing;
And when the static verification result information does not pass, generating a repeated verification instruction, and sending the repeated verification instruction to the request user side for repeated verification.
3. The method of claim 1, wherein when a search tag is a task type, performing data sharing based on the dynamic validation rule and the task level data index tree comprises:
When the search tag is of a task type, activating the dynamic verification rule to verify the dynamic input verification code, and generating dynamic verification result information;
When the dynamic verification result information is passed, indexing is carried out on the task-level data index tree based on the task type, a cross-block extraction data packet is obtained and sent to a request user side for sharing;
And when the dynamic verification result information does not pass, generating a repeated verification instruction, and sending the repeated verification instruction to the request user side for repeated verification.
4. A method as claimed in claim 2 or 3, further comprising: and when the repeated verification times are greater than or equal to the repeated verification times threshold, carrying out request locking on the request user terminal for a preset time.
5. A data isolation and secure sharing system for a big data environment, characterized by the steps for implementing the method of any of claims 1 to 4, comprising:
The data base information to be isolated is used for obtaining data base information to be isolated, wherein the data base information to be isolated comprises data attribute information;
the task type grouping module is used for grouping the data attribute information based on a task type set to generate a first data attribute grouping result;
The attribute type grouping module is used for grouping the data attribute information based on an attribute type set to generate a second data attribute grouping result;
The dynamic verification rule configuration module is used for traversing the first data attribute grouping result and configuring a dynamic verification rule;
the static verification rule configuration module is used for traversing the second data attribute grouping result and configuring a static verification rule;
The data storage module is used for storing the data to be isolated in blocks according to the second data attribute grouping result, configuring the static verification rule, constructing a task-level data index tree based on the block storage result of the first data attribute grouping result, and configuring the dynamic verification rule;
The static verification module is used for carrying out data sharing on the basis of the static verification rule when the retrieval tag is a data tag and the matching data attribute type;
and the dynamic verification module is used for carrying out data sharing based on the dynamic verification rule and the task-level data index tree when the retrieval label is of a task type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410748382.XA CN118332603B (en) | 2024-06-12 | 2024-06-12 | Data isolation and safe sharing method and system for big data environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410748382.XA CN118332603B (en) | 2024-06-12 | 2024-06-12 | Data isolation and safe sharing method and system for big data environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118332603A CN118332603A (en) | 2024-07-12 |
| CN118332603B true CN118332603B (en) | 2024-08-16 |
Family
ID=91780410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410748382.XA Active CN118332603B (en) | 2024-06-12 | 2024-06-12 | Data isolation and safe sharing method and system for big data environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118332603B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118656522B (en) * | 2024-08-16 | 2024-11-29 | 广东广宇科技发展有限公司 | Multi-element data isolation processing method |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118070328A (en) * | 2024-02-26 | 2024-05-24 | 中国邮政储蓄银行股份有限公司 | Encryption method, device, storage medium and system for data |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104811448A (en) * | 2015-04-21 | 2015-07-29 | 成都汇智远景科技有限公司 | Safe data storage method |
| US11144663B2 (en) * | 2016-12-30 | 2021-10-12 | Robert Bosch Gmbh | Method and system for search pattern oblivious dynamic symmetric searchable encryption |
| AU2020433394A1 (en) * | 2020-03-04 | 2022-09-15 | Yijun Du | System and method for utilizing search trees and tagging data items for data collection managing tasks |
-
2024
- 2024-06-12 CN CN202410748382.XA patent/CN118332603B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118070328A (en) * | 2024-02-26 | 2024-05-24 | 中国邮政储蓄银行股份有限公司 | Encryption method, device, storage medium and system for data |
Non-Patent Citations (1)
| Title |
|---|
| 云环境中基于信任分散策略的数据共享方案;张光华 等;计算机应用研究;20180331;35(03);正文第0-5节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118332603A (en) | 2024-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108874927B (en) | Intrusion Detection Method Based on Hypergraph and Random Forest | |
| Mouratidis et al. | Partially materialized digest scheme: an efficient verification method for outsourced databases | |
| US10402386B2 (en) | Method and apparatus for generating index for encrypted field in database | |
| Chen et al. | Continuous subgraph pattern search over certain and uncertain graph streams | |
| CN118332603B (en) | Data isolation and safe sharing method and system for big data environment | |
| CN113704252B (en) | Rule engine decision tree implementation method, device, computer equipment and computer readable storage medium | |
| CN108829880B (en) | Method for configuration management of optical network terminal equipment | |
| CN113901159A (en) | A Local Encryption and Decryption Method for Vector Data Network Transmission Based on Multilevel Spatial Index | |
| Guo et al. | A provably secure and efficient range query scheme for outsourced encrypted uncertain data from cloud-based Internet of Things systems | |
| Hassani et al. | I-hastream: density-based hierarchical clustering of big data streams and its application to big graph analytics tools | |
| CN109359481B (en) | Anti-collision search reduction method based on BK tree | |
| Guo et al. | Luxgeo: Efficient and security-enhanced geometric range queries | |
| CN106469218A (en) | A kind of Boolean expression storage based on bitmap, matching process and system | |
| Whang et al. | Disinformation techniques for entity resolution | |
| Xue et al. | Cuckoo-filter based privacy-aware search over encrypted cloud data | |
| CN108304467A (en) | For matched method between text | |
| CN111797130A (en) | Disambiguation-based enterprise entity unique ID (identity) creation method and system | |
| CN113849538A (en) | Intelligent extraction method and system based on fuzzy search multiple options | |
| CN112837026A (en) | Method, device and equipment for analyzing delivery cycle dependence and storage medium | |
| Jaiswal et al. | Uninterpreted schema matching with embedded value mapping under opaque column names and data values | |
| CN114510465B (en) | Data management method, device and computer readable storage medium | |
| CN110046180A (en) | It is a kind of for positioning the method, apparatus and electronic equipment of similar case | |
| CN114116715B (en) | Storage construction and retrieval method for secret state knowledge graph for protecting confidentiality of data | |
| CN112100220B (en) | System for realizing real-time monitoring of illegal account group | |
| US20240119178A1 (en) | Anonymizing personal information for use in assessing fraud risk |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |