CN111291107B - A progressive immersive visual data analysis method based on virtual reality technology - Google Patents

Abstract

The invention discloses a progressive immersion type visual data analysis method based on a virtual reality technology, which comprises the following steps of: when a client sends a message to a server through a WebSocket, a related database is selected to run a GSP progressive sequence mode mining algorithm, the mined data information is transmitted to a Unity client through the WebSocket, and visual presentation of the mined data is realized through a virtual reality technology; during running GSP progressive sequence pattern mining algorithm, related interactive operation is carried out on data, and steering operation is carried out on the algorithm, so that needed visual data is obtained. According to the characteristics of the virtual reality technology, the progressive immersion type visual analysis system can realize visual analysis and interaction of data, can give better data analysis environment and more excellent interactive immersion type experience to data analysis staff, and is beneficial to the rapid data analysis and mining of the data analysis staff.

Description

A progressive immersive visual data analysis method based on virtual reality technology

technical field

The invention relates to the field of visual data mining and analysis, in particular to a progressively immersive visual data analysis method based on virtual reality technology.

Background technique

In recent years, with the improvement of the ability of computers to continuously process data, the processing time of many massive data has been greatly reduced. Analysts bring delays. Based on incremental visualization, there are many emerging progressive visual analysis systems that aim to alleviate latency by exploiting partial intermediate results provided by the execution of progressive analysis algorithms. The basic idea of progressive visual analysis is that the analysis algorithm can be designed to produce meaningful partial results during execution, and then the progressive results can be combined with interactive visualization, allowing users to browse partial results immediately, after calculating new results Check them immediately, and perform new exploratory analyzes without waiting for previous analyzes to complete, which is recommended to avoid mandatory waiting times. This approach stands in stark contrast to traditional precomputation and acceleration methods, which assume that the analysis must be done before the results can be used for visual interaction. Progressive analysis systems allow users to interact with data mining algorithms by presenting the algorithm results as soon as they are discovered, or by providing the possibility to direct them to desired data or results. Altogether, progressive visual analytics promises to speed up the analysis for users by eliminating the time lag between when the user performs the interaction and when the analysis is computed.

Immersive analytics is also an emerging research field that aims to explore "the applicability and development of emerging user interface technologies to create more engaging and immersive experiences and seamless workflows for data analytics applications", Namely researching the use of Augmented Reality (AR) and Virtual Reality (VR) devices to visualize and analyze data in an immersive way. We can also define the use of compelling materialized analysis tools to support data understanding and decision making. Its goal is to remove barriers between people, data, and the tools they use to analyze it. At the same time, it is designed to support data understanding and decision-making everywhere and for everyone, whether working alone or collaboratively. Immersive analytics systems are often built on top of existing virtual/augmented reality environments. In the field of progressive immersive visual analytics, virtual/augmented reality environments control computing and track multiple human analytics behaviors through multiple input devices, thus providing rich opportunities for multimodal interaction.

The GSP algorithm is a sequential pattern mining algorithm. Sequential pattern mining is to find all sequences that meet the minimum support specified by the user from a data sequence (DataSequence) set S. Each such sequence is called a frequent sequence, or sequential pattern. The core idea of the GSP algorithm is: each time the database is scanned (pass), the candidate sequence is generated by using the large sequence generated in the previous scan, and their support is calculated while scanning, and the candidate sequence that satisfies the support is used as Large sequence for next scan. In the first scan, the frequent sequence pattern with a length of 1 is used as the initial seed set sequence. In addition, the GSP algorithm uses the Hash tree to store candidate sequences, which reduces the number of sequences to be scanned, and at the same time converts the representation method of the data sequence, so that it can effectively find out whether a candidate is a subsequence of the data sequence.

Contents of the invention

The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a progressively immersive visual data analysis method based on virtual reality technology. According to the characteristics of virtual reality technology, the progressively immersive visual analysis system can realize data analysis. Visual analysis and interaction. Compared with the traditional progressive visual analysis system, the immersive interactive environment gives data analysts a better data analysis environment and a better interactive immersive experience, which is beneficial for data analysts to perform rapid data analysis and dig.

The purpose of the present invention is achieved through the following technical solutions:

A progressive immersive visual data analysis method based on virtual reality technology, comprising the following steps:

S1. Use WebSocket to communicate messages between the client and the server, and use the sent messages to select the corresponding database and operation;

S2. Run the GSP progressive sequence pattern mining algorithm to obtain the data of customized content, and transmit the data to the client through WebSocket;

S3. Use the HTC VIVE device to realize the virtual reality immersive environment, and construct the coordinate system according to the attributes of the mining data;

S4. Use Unity's prefab to instantiate the data, the data is presented in the coordinate system, and the Handler handle is used to operate and interact with the data;

In step S1, the WebSocket realizes full-duplex communication between the client and the server: after the socket is created, it can respond to the socket through four events: onopen, onmessage, onclose and onerror; open the socket through a good address of the protocol The connection operation can send data to the server through the send() method, and can receive the data returned by the server through the onmessage event. After the data transmission is completed, the WebSocket will perform a close event to close the connection. It will be triggered when the connection, processing, receiving, and sending data fail. onerror event.

In step S2, the GSP progressive sequential pattern mining algorithm can design the analysis algorithm to generate meaningful partial results during the execution of data mining, and can give the partial progressive results to the user to immediately browse the visual results of partial content, and at the same time Perform the remaining data mining analysis.

In step S2, the GSP progressive sequence pattern mining algorithm is to use the large sequence generated during the previous scan to generate candidate sequences each time the database is scanned, and calculate their support (support) while scanning, satisfying The candidate sequence of the support degree is used as the large sequence for the next scan.

In step S3, the attributes of the mining data include the size (size), support (support) and number (number) of the pattern-based data.

In step S4, the use of Unity's prefab for data visualization is specifically: firstly create an object and a prefab, then import the mining data into Unity and instantiate the prefab, the data can be visualized, and the world coordinates and Camera coordinates Transformation and scale conversion, and then through the VR device, the data is displayed in the coordinate system of the VR environment for interaction.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The present invention proposes an effective solution for visual analysis of mining data from two aspects of progressive algorithm and immersive interaction based on virtual reality technology.

2. Compared with traditional interactive methods, the present invention provides an immersive interactive environment to provide data analysts with a better data analysis environment and better interactive experience, which is beneficial for data analysts to perform rapid data analysis and mining.

3. The present invention uses the Unity tool to be very suitable for cross-platform development, and provides a large number of plug-ins for easy development, wherein the VRTK used is an efficient interactive plug-in, which can help to quickly and easily build VR solutions in Unity. The creation process from prototyping ideas to building a complete solution helps users increase productivity.

Description of drawings

Figure 1 is a comparison diagram between progressive visual analysis algorithms and traditional non-progressive visual analysis algorithms.

Figure 2 is an example of the GSP sequential pattern mining algorithm and a pseudo-code diagram of the GSP algorithm.

Fig. 3 is a comparison flow chart of the progressive immersive visual data analysis system based on virtual reality technology of the present invention and the traditional progressive visual analysis system.

Fig. 4 is a comparison diagram of interactive data display methods between the progressive immersive visual data analysis system based on virtual reality technology and the traditional progressive visual analysis system.

FIG. 5 is a flowchart of a progressively immersive visual data analysis method based on virtual reality technology according to the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

As shown in Figure 5, a progressive immersive visual data analysis method based on virtual reality technology includes the following steps: when the client sends a message to the server through WebSocket, select the relevant database to run the GSP progressive sequential pattern mining algorithm, and The mined data information is transmitted to the Unity client through WebSocket, and the visual presentation of the mined data is realized through virtual reality technology. During the running of the GSP progressive sequential pattern mining algorithm, we can perform relevant interactive operations on the data and steering operations on the algorithm to obtain the visualized data we need.

As shown in Figure 1, the comparison between the progressive visual analysis algorithm and the traditional non-progressive visual analysis algorithm:

First, the data analyst selects the database to be analyzed, and selects the parameters of the data attributes for mining. For systems with progressive visual analysis algorithms, their purpose is to reduce waiting time by utilizing partial intermediate results provided by the execution of progressive analysis algorithms, The system is that analysis algorithms can be designed to produce meaningful partial results during execution, and progressive results can then be combined with interactive visualizations, allowing data analysts to browse partial results immediately, inspecting new results as soon as they are calculated, and perform new exploratory analyzes without waiting for previous analyzes to complete, which is recommended to avoid mandatory waiting times. This approach stands in stark contrast to traditional precomputation and acceleration methods, which assume that the analysis must be done before results can be obtained.

As shown in Figure 2, in step S2, the refinement steps of the GSP sequence pattern mining algorithm example and pseudo-code are as follows:

2.1) A sequence refers to arranging all the events related to the object in increasing order of time stamp to obtain a sequence s of the object. A sequence database contains one or more datasets of sequence data, as shown by S1 and S2 in Figure 2. The support degree of a sequence refers to the proportion of all data sequences containing s (an ordered list of events associated with a single data object (A/B/C in Figure 2)) of the sequence s, if the sequence If the support of s is greater than or equal to minsup, then s is said to be a sequence pattern (frequent sequence). Sequence pattern mining refers to finding all sequences whose support is greater than or equal to minsup given a sequence dataset Dataset and the minimum support minsup specified by the user.

2.2) The idea of the GSP sequence pattern mining algorithm is: 1. The sequence pattern C1 with a length of 1 is used as the initial seed set candidate k-frequent sequence pattern; 2. According to the seed set Ck with a length of k, through the connection operation and cutting The operation generates a candidate sequence pattern Ck+1 of length k+1, then scans the database, calculates the support of each candidate sequence pattern, and generates a sequence pattern Fk+1 of length k+1 as a new seed set. 3. Repeat the second step until no new sequence pattern or new candidate sequence pattern is generated.

Among them, Ck represents the candidate k-frequent sequential pattern, Fk represents the k-frequent sequential pattern, and UkFk represents the union of all k-frequent sequential patterns.

As shown in FIG. 3 , it is a comparison flow chart of the progressive immersive visual data analysis system based on virtual reality technology of the present invention and the traditional progressive visual analysis system. First of all, the previous steps are the same. The client sends using WebSocket technology to realize full-duplex communication between the client and the server. Select the database to be mined and analyzed through WebSocket message events, and then run the GSP sequence pattern mining algorithm to obtain the pattern data we want. During the mining and analysis process, user analysts can turn the algorithm and change the attributes of the mining pattern data required to obtain different data results. Table 1 is the specific explanation of the four message events in WebSocket.

Table 1

In the following steps, the traditional progressive visual analysis system presents the mined pattern data on a traditional web page, and data analysts use the mouse to perform interactive operations. The progressive immersive visual data analysis system we proposed based on virtual reality technology visualizes the pattern data transmitted through WebSocket, builds a virtual reality environment through HMD and VR equipment, and visualizes the data in the world coordinate system we defined. The refinement steps are as follows:

3.1) In order to visualize the data, we need points to represent the data. There are many ways to do this, but one of the more straightforward ways is to use a Sphere on Unity's built-in 3D assets and convert it into a so-called "prefab", which is essentially a template object that can be cloned and copy. Modify as needed.

3.2) To create a prefab, we need to convert the custom Sphere in Unity to a prefab so that we can create a copy of it for visualization as needed. Create a prefab object by right-clicking under Assets in the Project window and selecting "prefab" in the "Creat" window in the menu that opens.

3.3) Pass the mining data through WebSocket, and then instantiate the prefab. We need to associate the prefab we made with the script, and then guide the script to instantiate (clone). First we need to let the script know which prefab will be placed, to do this we need to declare a public GameObject variable inside the script, drag the prefab we created from the "Project" window into this field.

3.4) Then use the Instantiate method to instantiate and convert the position coordinates, continuously instantiate the passed mining pattern data, read the attribute value of the pattern data through the ToSingle method, and assign the attribute value to the coordinate value, and then transform the world through the Transform method The coordinate system is converted to the coordinate system value we constructed, and finally the visualized coordinate data is obtained.

As shown in Figure 4, interactive tasks are performed through the Handler handle, such as changing the attribute size of the mining mode data, performing steering operations on the mining algorithm, and viewing and analyzing the visualized data that has been mined. Among them, VRTK is mainly used to realize, and the refinement steps are as follows:

4.1) Import SteamVR and VRTK packages. Create a new scene, delete the built-in Camera, and create a new Plane. Create a new empty object, rename it to VRTK_SDK Manager, and add the component VRTK_SDK Manager. Create an empty object as a sub-object of VRTK_SDK Manager, rename it to VRTK_SDK Setup, and add the component VRTK_SDK Setup.

4.2) Add the prefab Camera_Rig as a child object of VRTK_SDK Setup. Select VRTK_SDKManager, select Setups and click "+", and drag VRTK_SDK Setup to the position of "None(VRTK_SDK Setup)". Create a new empty object and rename it to VRTK_Scripts. Create two empty objects under VRTK_Scripts and rename them as LeftController (for configuring the left handle) and RightController (right handle).

4.3) Select LeftController and RightController, set the left pointer and right pointer respectively. Then add some scripts that come with VRTK on the handle, which can control the interaction between the Handler handle and the object. When we need custom functions, we have to start writing scripts or inherit the existing VRTK scripts to expand functions. , some of the script functions used are shown in Table 2.

Table 2

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (5)

1. A progressive immersive visual data analysis method based on virtual reality technology, characterized in that, comprising the following steps: S1. Use WebSocket to communicate messages between the client and the server, and use the sent messages to select the corresponding database and operation; S2. Run the GSP progressive sequence pattern mining algorithm to obtain the data of customized content, and transmit the data to the client through WebSocket; S3. Use the HTC VIVE device to realize the virtual reality immersive environment, and construct the coordinate system according to the attributes of the mining data; S4. Use Unity's prefab to instantiate the data, the data is presented in the coordinate system, and the Handler handle is used to operate and interact with the data; Among them, the GSP progressive sequential pattern mining algorithm includes: Step 1. Use the sequence pattern C1 with a length of 1 as the initial seed set candidate k-frequent sequence pattern; Step 2. According to the seed set Ck of length k, generate a candidate sequence pattern Ck+1 of length k+1 through connection operation and cutting operation, then scan the database, calculate the support degree of each candidate sequence pattern, and generate a length of The sequence pattern Fk+1 of k+1 is used as a new seed set; Repeat step 2 until no new sequence patterns or new candidate sequence patterns are generated; Among them, Ck represents the candidate k-frequent sequence pattern, and Fk represents the k-frequent sequence pattern; In step S2, the GSP progressive sequence pattern mining algorithm uses the large sequence generated during the previous scan to generate candidate sequences each time the database is scanned, and calculates their support while scanning to meet the requirements of the support. The candidate sequence is used as the large sequence for the next scan. 2. according to the described progressive immersive visual data analysis method based on virtual reality technology of claim 1, it is characterized in that, in step S1, described WebSocket realizes full-duplex communication between client and server: After creating socket , can respond to the socket through four events of onopen, onmessage, onclose and onerror; open connection operation through the address of the agreement, can send data to the server through the send() method, and can receive the data returned by the server through the onmessage event, data After the transmission is completed, the WebSocket performs a close event to close the connection. If there is a failure to connect, process, receive, or send data, the onerror event is triggered. 3. The progressive immersive visual data analysis method based on virtual reality technology according to claim 1, characterized in that, in step S2, the GSP progressive sequence pattern mining algorithm can design the analysis algorithm as performing the data mining process Meaningful partial results can be generated in the system, and partial progressive results can be given to users to browse the visualization results of some content immediately, and perform the rest of the data mining analysis at the same time. 4. The progressively immersive visual data analysis method based on virtual reality technology according to claim 1, characterized in that, in step S3, the attributes of the mining data include size, support and quantity based on pattern data. 5. The progressively immersive visual data analysis method based on virtual reality technology according to claim 1, characterized in that, in step S4, the visualization of data using the prefab of Unity is specifically: first create object and prefab, and then Import the mining data into Unity and instantiate the prefab to visualize the data, go through the conversion of world coordinates and Camera coordinates and scale conversion, and then use the VR device to display the data in the coordinate system of the VR environment for interaction.

Images