CN118590489A - A method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud - Google Patents

Abstract

The present invention provides a method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud. By combining a multi-source heterogeneous data fusion algorithm, feature extraction and unsupervised fusion processing are performed on heterogeneous data such as text data, vibration data and audio data, and the fusion is reduced to fusion feature data with low data volume for transmission to the cloud; the fusion feature data of the vehicle-mounted edge device is analyzed and converted in the cloud, restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge device is stored, and corresponding device identifiers and corresponding data models are respectively set for different vehicle-mounted edge devices. The method of the present invention can reduce the amount of data transmitted when the heterogeneous data of the vehicle-mounted edge device is transmitted to the cloud, save the network bandwidth for transmission to the cloud, reduce its network load, help improve the data interaction efficiency within the Internet of Vehicles system, and enhance the overall perception ability of the Internet of Vehicles system to vehicle status information.

Description

A method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud

Technical Field

The present invention relates to the field of data input and processing of heterogeneous devices in an Internet of Things environment, and in particular to a method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud.

Background Art

With the continuous advancement of science and technology, the Internet of Things (IoT) has become an important bridge connecting the world, connecting various devices and systems intelligently through the Internet. In this digital age, connected vehicles (Connected Vehicles), as one of the important application areas of the Internet of Things, is gaining increasing attention and importance. Connected Vehicles technology connects cars to the Internet, which not only makes driving more intelligent and convenient, but also provides safer, more efficient and intelligent transportation solutions. In the development of connected vehicles, the combination of various sensors, communication technologies and data analysis technologies has provided vehicles with more intelligent functions, thereby promoting the upgrade and transformation of the entire transportation system.

In the field of Internet of Vehicles, the data generated by vehicles includes not only traditional on-board sensor data, but also multi-source heterogeneous data from multiple sources, such as audio, vibration, and text data. These data may come from different devices on the vehicle, which can be called on-board edge devices. The comprehensive use of data related to these on-board edge devices is crucial to achieving a smarter and safer transportation system. Audio data can be used to identify the ambient sound around the vehicle, vibration data can help monitor the status and health of the vehicle, and text data can contain information from on-board systems, navigation applications, and communication devices. By effectively integrating and analyzing these multi-source heterogeneous data, we can achieve more accurate vehicle health monitoring, smarter driver assistance systems, and more efficient traffic management. This comprehensive use of different types of data will bring more possibilities to the development of Internet of Vehicles technology and promote the development of intelligent transportation systems in a smarter and more sustainable direction.

As the scale of the Internet of Vehicles expands and the types of vehicle sensors increase, the amount of data collected is also growing exponentially, and is expected to exceed 40Gbit per second. If this amount of data is transmitted directly through the communication system, it will cause serious congestion, and heterogeneous device data will be difficult to manage efficiently. The Internet of Vehicles needs a more efficient system and method for transmitting heterogeneous device data or fusion features to the cloud.

At present, vehicle sensors mainly work independently, such as using cameras for image recognition and radars for speed and distance detection. Vehicles still lack the ability to effectively integrate multiple sensor data. The Internet of Vehicles needs to integrate multi-source heterogeneous data to reduce the amount of transmitted data, save network bandwidth, reduce network load, and improve the overall perception ability of the vehicle.

Traditional centralized computing and processing methods can no longer meet the needs of the Internet of Vehicles. Although the centralized processing and computing capabilities of the cloud are powerful, their communication transmission performance does not match the computing performance, which easily leads to data congestion problems. In order to meet these challenges, the Internet of Vehicles needs to shift to a more distributed computing mode, using edge computing to pre-process data, reduce data transmission volume, and improve real-time performance and efficiency. However, compared with the growth rate of sensor data volume, the development of edge acquisition computing devices is relatively slow, and the edge acquisition computing devices of a single vehicle are difficult to cope with the huge challenges of data fusion processing. In order to solve this problem, the Internet of Vehicles needs to share data and resources among the computing resources of equipment such as vehicle terminals, roadside units, and base stations. This collaborative computing mode can effectively meet the needs of massive data processing. By allocating data processing tasks to multiple edge acquisition computing devices, efficient data processing and analysis can be achieved, thereby ensuring that the Internet of Vehicles system can operate efficiently and provide more intelligent traffic solutions.

Summary of the invention

In view of the deficiencies of the above-mentioned prior art, the present invention provides a method for transmitting the data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud, so as to reduce the amount of data transmitted to the cloud for heterogeneous data of vehicle-mounted edge devices, save network bandwidth for transmission to the cloud, reduce its network load, and thereby help improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information.

In order to solve the above technical problems, the present invention adopts the following technical solutions:

A method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud comprises the following steps:

S1. Construct the device identification and data model corresponding to each vehicle-mounted edge device in the Internet of Things in the cloud; the device model corresponding to the vehicle-mounted edge device is used to identify the individual identity of the vehicle-mounted edge device; the data model corresponding to the vehicle-mounted edge device is used to record the working data of the corresponding vehicle-mounted edge device;

S2. Deploy edge collection and computing devices in the preset data collection area. Each edge collection and computing device can establish a data communication connection with multiple vehicle-mounted edge devices, and build a data transmission channel between the edge collection and computing device and the cloud, so that the working data collected by the vehicle-mounted edge device can be transmitted to the cloud through the edge collection and computing device;

S3, the vehicle-mounted edge device collects its working data and transmits it to the edge collection and computing device with which it establishes a data communication connection; the working data includes text data, vibration data and audio data of the vehicle-mounted edge device;

S4. The edge acquisition computing device performs feature extraction and unsupervised fusion processing on the text data, vibration data and audio data in the working data of the vehicle-mounted edge device to obtain the fused feature data of the vehicle-mounted edge device, and uploads it to the cloud through the data transmission channel;

S5. The cloud side parses and converts the fused feature data of the vehicle-mounted edge device, and obtains the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, which are stored as working parsed data in the data model corresponding to the vehicle-mounted edge device.

As a preferred solution, in step S1, a data storage space for the vehicle-mounted edge device and its components is set in the data model corresponding to the vehicle-mounted edge device, which is used to store the working data of the vehicle-mounted edge device and the working data of its components respectively.

As a preferred solution, step S2 includes:

S201, establishing a device access gateway between the cloud and the edge collection and computing device, each device access gateway is used as a data access point for one or more edge collection and computing devices;

S202. Establish a data access channel in the cloud to serve as a data transmission channel between the cloud and the device access gateway, so that the working data of the vehicle-mounted edge device collected by the edge collection and computing device can be uploaded to the cloud through the device access gateway and the data access channel.

As a preferred solution, step S4 includes:

S401, the vehicle-mounted edge device communicates data with the edge collection and computing device, and the edge collection and computing device collects working data of the vehicle-mounted edge device;

S402, the edge collection and computing device performs preprocessing of word segmentation, sentence segmentation, and stop word removal on the text data in the working data;

S403, the edge collection and calculation device performs word embedding feature extraction on the preprocessed text data to obtain a word feature vector of the text data;

S404, the edge acquisition and calculation device performs slicing and segmenting processing on the vibration data and the audio data in the working data in time sequence to obtain a vibration data slice vector and an audio data slice vector;

S405. The edge acquisition and computing device performs unsupervised dimensionality reduction and fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain fused feature data of the vehicle-mounted edge device.

As a preferred solution, in step S401, the vehicle-mounted edge device establishes a data connection with an edge acquisition and computing device that can establish data communication, obtains the data processing resource occupancy rate of the edge acquisition and computing device, and selects an edge acquisition and computing device whose data processing resource occupancy rate is less than a preset resource occupancy threshold to upload the working data of the vehicle-mounted edge device.

As a preferred solution, in step S402, the edge acquisition computing device uses a Bert model that has been pre-trained for text preprocessing to perform word segmentation, sentence segmentation, and stop word removal on the text data in the working data.

As a preferred solution, in step S403, the edge acquisition computing device uses a pre-trained Bert-base-chinese model to perform word embedding feature extraction on Chinese text data, and uses a pre-trained Bert-base-uncased model to perform word embedding feature extraction on English text data; the word feature vector representation of the obtained text data is dimensional matrix vector, a single sentence in the text data is represented as an n _i ×N _x- dimensional vector, and a single character is represented as a 1×N _x- dimensional vector, N _x represents the feature dimension of feature extraction, n _i represents the number of characters in the i-th sentence in the text data, and m represents the number of sentences obtained after the text data is segmented. As a result, the word feature vector contains not only the character information and semantic information of the global text in the text data, but also the position information of each sentence and character.

As a preferred solution, in step S404, the edge acquisition and computing device reads the vibration data and audio data in the working data respectively and performs normalization processing, and then slices and divides the vibration data and audio data according to the feature dimension _Nx of the preset feature extraction; the obtained vibration data slice vector is represented as an x×N _x -dimensional vector, and the audio data slice vector is represented as a y×N _x -dimensional vector, where x and y represent the number of slice division segments of the vibration data and the audio data respectively, so that the vibration data slice vector and the audio data slice vector not only contain the amplitude feature information of the vibration data and the audio data, but also contain the position information of each slice division segment.

As a preferred solution, in step S405, the edge acquisition computing device first performs splicing and fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain the working data feature matrix vectors, and then inputs the working data feature matrix vectors into the pre-trained Autoencoder self-encoding model for unsupervised dimensionality reduction encoding processing to obtain the fused feature data of the vehicle-mounted edge device; wherein the working data feature matrix vector is represented as dimensional matrix vector, _Nx represents the feature dimension of feature extraction, _ni represents the number of characters in the i-th sentence in the text data, m represents the number of sentences obtained after the text data is segmented, and x and y represent the number of slice segmentation fragments of vibration data and audio data respectively.

As a preferred solution, in step S5, the cloud first decodes the fused feature data of the vehicle-mounted edge device through the Autoencoder self-encoding process to obtain The working data feature matrix vector of the vehicle-mounted edge device is then decomposed to obtain dimensional word feature vectors of text data, x×N _x -dimensional vibration data slice vectors, and y×N _x- dimensional audio data slice vectors, and according to the position information contained in the word feature vectors of the text data, the vibration data slice vectors, and the audio data slice vectors, feature reorganization is performed on the word features of the text data and the slice segments of the vibration data and the audio data in position order, and then the respective reorganization results are converted into data formats according to the preset text data format, vibration data format, and audio data format to obtain text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, and finally the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device are used as the working parsing data of the corresponding vehicle-mounted edge device, and stored in the data model corresponding to the corresponding vehicle-mounted edge device.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention proposes a method for transmitting the data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud. In view of the problems that the amount of heterogeneous data of vehicle-mounted edge devices is large and easily causes communication congestion in cloud transmission, the method combines a multi-source heterogeneous data fusion algorithm to extract features and perform unsupervised fusion processing on heterogeneous data such as text data, vibration data and audio data, and then fuses and reduces their dimensions into fused feature data with low data volume for transmission to the cloud, thereby saving network bandwidth for cloud transmission and reducing its network load.

2. In the method for transmitting the fusion feature data of heterogeneous devices in the Internet of Vehicles to the cloud of the present invention, the fusion feature data of the vehicle-mounted edge device is analyzed and converted in the cloud and restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge device is stored. In addition, corresponding device identifiers and corresponding data models are respectively set for different vehicle-mounted edge devices, which facilitates the differentiation of vehicle-mounted edge devices for the stored heterogeneous data such as text, audio, vibration, etc., and provides data support for subsequent equipment operation and maintenance and other applications.

3. The method of the present invention can reduce the amount of heterogeneous data of vehicle-mounted edge devices when transmitting them to the cloud, save network bandwidth for cloud transmission, and reduce its network load. At the same time, it can also well ensure the storage integrity of future data of vehicle-mounted edge devices in the cloud, thereby helping to improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information, and has great value for technology promotion and application.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

FIG1 is a flow chart of a method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud according to the present invention.

FIG2 is an overall working diagram of an application example of the method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud according to the present invention.

FIG. 3 is a schematic diagram of a device model constructed in an embodiment of the present invention.

FIG. 4 is a schematic diagram showing the principle of a feature fusion algorithm for audio, text, and vibration data in an embodiment of the present invention.

FIG5 is a schematic diagram showing the principle of extracting sentence data using the Bert model in an embodiment of the present invention.

FIG. 6 is a schematic diagram of the structure of the AE model in an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

In view of the deficiencies in the prior art, the present invention mainly aims to solve the following problems:

(1) With the development of the Internet of Things and sensor technology, the amount of collected data is growing exponentially. Direct transmission through the communication system is likely to cause cloud transmission communication congestion. In addition, heterogeneous device data is difficult to manage efficiently and is difficult to operate manually.

(2) Due to the large amount of data and the non-uniform data structure, the data from multi-source heterogeneous devices is not fully utilized, resulting in poor overall perception of the vehicle-mounted devices by the Internet of Vehicles system.

(3) The processing and computing power of the on-board edge equipment is insufficient and it is difficult to meet the computing power requirements for more complex data fusion processing.

In view of the above technical problems, the present invention provides a method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud, the process of which is shown in FIG1 and includes the following steps:

S1. Construct the device identification and data model corresponding to each vehicle-mounted edge device in the Internet of Things in the cloud; the device model corresponding to the vehicle-mounted edge device is used to identify the individual identity of the vehicle-mounted edge device; the data model corresponding to the vehicle-mounted edge device is used to record the working data of the corresponding vehicle-mounted edge device;

S2. Deploy edge collection and computing devices in the preset data collection area. Each edge collection and computing device can establish a data communication connection with multiple vehicle-mounted edge devices, and build a data transmission channel between the edge collection and computing device and the cloud, so that the working data collected by the vehicle-mounted edge device can be transmitted to the cloud through the edge collection and computing device;

S3, the vehicle-mounted edge device collects its working data and transmits it to the edge collection and computing device with which it establishes a data communication connection; the working data includes text data, vibration data and audio data of the vehicle-mounted edge device;

S4. The edge acquisition computing device performs feature extraction and unsupervised fusion processing on the text data, vibration data and audio data in the working data of the vehicle-mounted edge device to obtain the fused feature data of the vehicle-mounted edge device, and uploads it to the cloud through the data transmission channel;

S5. The cloud side parses and converts the fused feature data of the vehicle-mounted edge device, and obtains the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, which are stored as working parsed data in the data model corresponding to the vehicle-mounted edge device.

The present invention proposes a method for transmitting the fusion features of data of heterogeneous devices in the Internet of Vehicles to the cloud. In view of the problems that the amount of heterogeneous data of vehicle-mounted edge devices is large and easily causes communication congestion in cloud transmission, the present invention combines a multi-source heterogeneous data fusion algorithm to extract features and perform unsupervised fusion processing on heterogeneous data such as text data, vibration data and audio data, and fusion and reduce their dimensions into fused feature data with low data volume for transmission to the cloud, thereby saving network bandwidth for cloud transmission and reducing its network load. On the other hand, the fused feature data of the vehicle-mounted edge devices is parsed and converted in the cloud and restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge devices is stored. In addition, corresponding device identifiers and corresponding data models are respectively set for different vehicle-mounted edge devices, so as to facilitate the distinction of vehicle-mounted edge devices for stored heterogeneous data such as text, audio, vibration, etc., and provide data support for subsequent equipment operation and maintenance applications. It can be seen that the method of the present invention can reduce the amount of heterogeneous data of vehicle-mounted edge devices when transmitting to the cloud, save network bandwidth for cloud transmission, and reduce its network load. At the same time, it can also well ensure the storage integrity of future data of vehicle-mounted edge devices in the cloud, thereby helping to improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information.

Next, the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud of the present invention is described in detail.

FIG2 shows a schematic diagram of an example of a method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud in a specific application. This example is also used to illustrate and explain the method below.

During the specific implementation, in step S1, the device identification mainly plays the role of identifying the identity of the vehicle-mounted edge device, and its specific data form can be diverse, for example, it can be a device ID or a visual device model. In the current application scenario, in order to better meet the high requirements for data visualization, a visual device model can be preferably used as the device identification. In the construction of the device model, it is possible to build specifications based on objects and adopt object-oriented concepts to enable heterogeneous device objects to quickly build device models of vehicle-mounted edge devices based on digital abstract models. As an abstract layer of edge devices, the device model shields the underlying terminal differences, standardizes the device's capability expression and interaction methods, forms a hierarchical object structure, and provides standardized support and guarantee for applications and other services. Use the device model to depict the device structure portrait, and use the data model to describe the device data map, and you can enter edge device related information in the cloud.

In the specific implementation of building a device model, in order to improve the efficiency of model reuse, hierarchical modeling can be performed according to device models, device objects, and device components. A device model represents a digital abstract model of a type of equipment and represents a specific device model; a device component represents a certain type of component that is of concern and monitored. The entire device model is subdivided into multiple subsystems, and each individual subsystem is further subdivided into more components. Hierarchical modeling describes the structural portrait of the device model through device components; a device object represents an instantiation of the device model. The device object inherits all the characteristics of the corresponding device model.

Taking the car as an example, hierarchical modeling is performed in the cloud according to the device model, device object, and device components, and the corresponding configuration information is entered to complete the device registration. The device model is a digital abstract model of a type of car, representing a specific car model; each device model is subdivided into multiple subsystems, such as a car engine, transmission, etc.; each subsystem consists of multiple components, such as a car transmission spindle, tires, etc. The device object represents an instantiation of the device model, such as a car with a unique ID.

In the process of building the device model, it is necessary to first build a data map for the device model, which can define the collection of all sampled data sets of the device model. By providing a visual device data model management function and customizing the basic data model, the structured definition of the data model can be realized, and finally the data packet model is bound to the device identifier (the above-mentioned device model or device ID, etc.). The data set is divided into the whole machine level and the component level. Each data set requires customized measurement point information. The data type can include vibration signals, audio information, text information, etc. The structural portrait and data map constitute a complete device model. As shown in Figure 3, the device model built for the speed change gear box system is shown, taking car A as an example.

In specific implementation, in step S1, a data storage space for the vehicle-mounted edge device and its components is set in the data model corresponding to the vehicle-mounted edge device, which is used to store the working data of the vehicle-mounted edge device and the working data of its components respectively. The data model can be regarded as a data repository for the device data map, which is used to store the vibration signal, audio information, text information, etc. of the vehicle-mounted edge device.

In specific implementation, step S2 includes:

S201, establishing a device access gateway between the cloud and the edge collection and computing device, each device access gateway is used as a data access point for one or more edge collection and computing devices;

S202. Establish a data access channel in the cloud to serve as a data transmission channel between the cloud and the device access gateway, so that the working data of the vehicle-mounted edge device collected by the edge collection and computing device can be uploaded to the cloud through the device access gateway and the data access channel.

A new device access gateway is built in the cloud. Each gateway is an access point (endpoint) of the device, representing an abstract device collection access terminal. It can manage several actual vehicle-mounted edge devices and is a unified entrance and exit between the data collection terminal and the platform data access terminal. When configuring the relevant information, it includes: gateway configuration, including the configuration of the MQTT protocol, including the gateway name, region, organization, gateway description, etc.; transmission configuration, including the message identifier of the transmission, the transmission cycle, the transmission delay time, the data expiration time, the data packet size, etc.; data encoding configuration, according to the needs of different devices, configure the corresponding encoding format, such as JSON, binary, text, etc.; data encryption configuration, according to the data requirements of different devices, data can be encrypted before transmission, supporting mainstream symmetric algorithms such as AES and DES, to ensure the network transmission security when the data is sent to the system data exchange layer.

When creating a new data access channel in the cloud, multiple data channels can be built under each gateway. Each channel can correspond to a data model. For example, if four data access channels are built, four data models can be corresponding to them in the specific design: building a car recording data model, including two data items: time and audio sampling data; building a car log data model, including two data items: time and content; building a gearbox transmission shaft vibration data model, including two data items: time and vibration sampling data; building a car audio and text vibration data fusion feature data model, including two data items: time fusion feature. And so on.

In specific implementation, in step S3, the working data collected by the vehicle-mounted edge device includes text data, vibration data, and audio data of the vehicle-mounted edge device. Taking a car as an example, the working data collected by a certain model of car may include its whole machine log data log.txt (text data), whole machine recording data audio.wav (audio data), and gearbox vibration data vib.dat (vibration data); for a specific parameter example, for example, the sampling frequency of collecting recording data and vibration data is 10240Hz, and the text content of the log data is "transmission gearbox drive shaft failure, serious abnormal noise".

In specific implementation, the flowchart of step S4 is shown in FIG4 , which includes the following steps:

S401, the vehicle-mounted edge device communicates data with the edge collection and computing device, and the edge collection and computing device collects working data of the vehicle-mounted edge device;

S402, the edge collection and computing device performs preprocessing of word segmentation, sentence segmentation, and stop word removal on the text data in the working data;

S403, the edge collection and calculation device performs word embedding feature extraction on the preprocessed text data to obtain a word feature vector of the text data;

S404, the edge acquisition and calculation device performs slicing and segmenting processing on the vibration data and the audio data in the working data in time sequence to obtain a vibration data slice vector and an audio data slice vector;

S405. The edge acquisition and computing device performs unsupervised dimensionality reduction and fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain fused feature data of the vehicle-mounted edge device.

In a specific implementation, the edge acquisition computing device can be designed to include a communication unit, a text preprocessing unit, a text feature extraction unit, a time series slicing processing unit and a fusion processing unit. Among them, the communication unit is used to communicate data with the vehicle-mounted edge device and collect the working data of the vehicle-mounted edge device; the text preprocessing unit is used to perform word segmentation, sentence segmentation, and stop word removal preprocessing on the text data in the working data; the text feature extraction unit is used to perform word embedding feature extraction on the preprocessed text data to obtain the word feature vector of the text data; the time series slicing processing unit is used to perform time series analysis on the vibration data and audio data in the working data, and perform slicing and segmentation processing according to the time series to obtain the vibration data slice vector and the audio data slice vector; the fusion processing unit is used to perform unsupervised fusion and dimensionality reduction processing on the word feature vector of the text data, the vibration data slice vector and the audio data slice vector to obtain the fusion feature data of the vehicle-mounted edge device.

During specific implementation, the details of step S401 in step S4 are as follows: the vehicle-mounted edge device establishes data connection with the edge collection and computing device that can establish data communication, obtains the data processing resource occupancy rate of the edge collection and computing device, and selects an edge collection and computing device whose data processing resource occupancy rate is less than a preset resource occupancy threshold, and uploads the working data of the vehicle-mounted edge device.

For example, car A determines that the amount of perceived data is too large, and communicates with the roadside unit through the 4G communication module. It selects an edge collection and computing device whose data processing resource occupancy rate is less than the preset resource occupancy threshold (that is, the communication load and computing load are small) to perform data collection and fusion processing. The data transmission process between them will include initiating communication, identity authentication, establishing connection, data transmission, data confirmation, collaborative computing and other processes.

In the specific implementation, the details of step S402 in step S4 are: the edge acquisition computing device uses the Bert model that has been pre-trained for text preprocessing to perform word segmentation, sentence segmentation, and stop word removal on the text data in the working data.

For example, the Bert model has a maximum sequence input limit of 512. The text data is first processed by sentence segmentation, and the text is segmented according to the context semantics and punctuation marks such as ".", "?", "!", ";" are used for auxiliary segmentation to obtain sentence texts with different lengths. Then, for sentence texts of different lengths, Chinese is segmented by characters, and English is segmented by words, and the continuous natural language text is divided into the smallest units (words or subwords) with semantic meaning. Finally, common words that contribute less to understanding the meaning of the text, namely stop words, are removed to reduce noise and redundant information in the text to improve the effect of text processing and analysis. For example, the whole machine log data log.txt of the aforementioned example "transmission gearbox transmission shaft failure, severe abnormal noise" is obtained after preprocessing as "change", "speed", "device", "tooth", "wheel", "box", "transmission", "motion", "shaft", "fault", "obstacle", "abnormal", "noise", "severe", and "heavy".

In the specific implementation, the details of step S403 in step S4 are as follows: the edge acquisition computing device uses the pre-trained Bert-base-chinese model to extract word embedding features for Chinese text data, and uses the pre-trained Bert-base-uncased model to extract word embedding features for English text data; the word feature vector representation of the obtained text data is dimensional matrix vector, a single sentence in the text data is represented as an n _i ×N _x- dimensional vector, and a single character is represented as a 1×N _x- dimensional vector, N _x represents the feature dimension of feature extraction, n _i represents the number of characters in the i-th sentence in the text data, and m represents the number of sentences obtained after the text data is segmented. As a result, the word feature vector contains not only the character information and semantic information of the global text in the text data, but also the position information of each sentence and character.

Use the open source pre-trained Bert-base-uncased model to process English data and the Bert-base-chinese model to process Chinese data, extract features from text data, and represent text data with word vectors through word vectors, position vectors, and text vectors.

For example, if the feature dimension of the feature extraction of the preset Bert model is N _x = 768, the output representation of a single character after the Bert model is 1×768 dimensions, and the output representation of a single sentence is n _i ×768 dimensions, where n _i represents the number of characters in the i-th sentence, and each sentence is normalized. The vector representation of a single text data obtained by concatenating the vector representations of m sentences through the concat method can be expressed as dimensional matrix vector, where m represents the number of sentences obtained after the text data is segmented.

As shown in Figure 5, the above-mentioned example of the whole machine log data log.txt "transmission gearbox transmission shaft failure, serious abnormal noise" is obtained after the Bert model to obtain the word feature vector of the 15×768-dimensional vector matrix (including 15 words: "change", "speed", "device", "tooth", "wheel", "box", "transmission", "drive", "shaft", "fault", "obstacle", "abnormal", "noise", "serious", and "serious"). As a vector representation of the text data of the whole machine log data log.txt, the word feature vector not only contains the character information and semantic information of the global text in the text data, but also contains the position information of each sentence and character, and has sufficient expression ability.

During specific implementation, the details of step S404 in step S4 are as follows: the edge acquisition and computing device reads the vibration data and audio data in the working data respectively and performs normalization processing, and then slices and divides the vibration data and audio data according to the feature dimension _Nx of the preset feature extraction; the obtained vibration data slice vector is represented as an x×N _x- dimensional vector, and the audio data slice vector is represented as a y×N _x -dimensional vector, where x and y represent the number of slice division segments of the vibration data and the audio data respectively, so that the vibration data slice vector and the audio data slice vector not only contain the amplitude feature information of the vibration data and the audio data, but also contain the position information of each slice division segment.

For example, the vibration data and audio data are read and normalized respectively, and then divided according to the cutting dimension 768 to obtain multiple data blocks, which are used as the vector representation x×768 of the vibration data and the vector representation y×768 of the audio data. After that, the vector representation of the vibration data and the vector representation of the audio data can be connected by concat to obtain a vector matrix of (x+y)×768 dimensions. For example, the audio audio.wav with a duration of 4s is parsed to obtain 4×10240 data points, and the vibration log.dat with a duration of 20s is parsed to obtain 20×10240 data points. After each is normalized, the data is cut according to 768, and finally the insufficient data is filled with data point 1, obtaining two data matrices of 54×768 and 280×768 dimensions respectively, and then concat is used to obtain a 334×768-dimensional audio-vibration joint data matrix.

In the specific implementation, the details of step S405 in step S4 are as follows: the edge acquisition computing device first performs splicing and fusion processing on the word feature vector of the text data, the vibration data slice vector and the audio data slice vector to obtain the working data feature matrix vector, and then inputs the working data feature matrix vector into the pre-trained Autoencoder self-encoding model for unsupervised dimensionality reduction encoding processing to obtain the fusion feature data of the vehicle-mounted edge device; wherein the working data feature matrix vector is represented as dimensional matrix vector, _Nx represents the feature dimension of feature extraction, _ni represents the number of characters in the i-th sentence in the text data, m represents the number of sentences obtained after the text data is segmented, and x and y represent the number of slice segmentation fragments of vibration data and audio data respectively.

For example, the vector representations of vibration data and audio data are concat- ed with the text representation vector to obtain a 349×768-dimensional working data feature matrix vector, which is used as the input of the Autoencoder model.

The Autoencoder model is a commonly used dimensionality reduction and fusion coding model, and its architecture is shown in Figure 6. In the Autoencoder model, the Encoder and Decoder each contain three one-dimensional convolutional layers. The three one-dimensional convolutional layers of the Encoder reduce the input matrix to 256 dimensions, 64 dimensions, and 32 dimensions in turn, and obtain a 349×32-dimensional feature matrix for the hidden layer output. The Decoder restores the hidden layer output back to the 349×768-dimensional input vector matrix in turn. Before applying it for vector dimensionality reduction and fusion coding, the Autoencoder model can be trained by minimizing the reconstruction error MSE between the encoder input and the decoder output to achieve the expected dimensionality reduction and fusion requirements. The calculation formula of the MSE loss function is Among them, _yi represents the model output encoding label during training, Indicates the target encoding label of model training, and K indicates the number of training samples. In the specific application implementation, the hyperparameters of the Autoencoder model during training are set as follows: the neural network optimizer is Adam, the learning rate is 1e ^-4 , the batch size is 10, the Dropout random inactivation rate is 0.4, and a total of 200 epochs are trained.

In the specific implementation, step S5 includes: the cloud first decodes the fused feature data of the vehicle-mounted edge device through the Autoencoder self-encoding, and parses it to obtain The working data feature matrix vector of the vehicle-mounted edge device is then decomposed to obtain dimensional word feature vectors of text data, x×N _x- dimensional vibration data slice vectors, and y×N _x- dimensional audio data slice vectors, and according to the position information contained in the word feature vectors of the text data, the vibration data slice vectors, and the audio data slice vectors, feature reorganization is performed on the word features of the text data and the slice segments of the vibration data and the audio data in position order, and then the respective reorganization results are converted into data formats according to the preset text data format, vibration data format, and audio data format to obtain text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, and finally the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device are used as the working parsing data of the corresponding vehicle-mounted edge device, and stored in the data model corresponding to the corresponding vehicle-mounted edge device.

The constructed data transmission channel is used to transmit the data features fused at the edge to the cloud, and key authentication parameters are provided; the edge uses the industrial gateway to establish communication with the cloud, and the fused data features are efficiently transmitted to the cloud; the original data or pre-processed heterogeneous data can be sent when communication resources are idle, and collected in the cloud data center for backup.

Based on the above data architecture layout between the cloud and the vehicle-mounted edge devices and the method of transmitting to the cloud, in order to achieve high compatibility data access between the vehicle-mounted edge devices and the cloud, the high compatibility access task of heterogeneous data of the vehicle-mounted edge devices is decoupled into data access and transmission, data analysis and processing, and data storage and management. Specifically, the overall data transmission process to the cloud can be summarized as follows:

Data access and transmission: Establish an industrial gateway based on the constructed device data model. The industrial data gateway is the network interface for data access to the IoT cloud platform, responsible for providing a network transmission channel for the collected data to be sent to the IoT cloud platform. Using Mosquitto open source message agent software and MQTT data transmission protocol, the collected data is received in the standard JSON format and sent to the Kafka message queue for asynchronous processing and current limiting and peak shaving.

Data parsing and processing: After the cloud receives the data transmitted by the device, it needs to be parsed and processed. According to the unified object model, the device data is parsed into standard formats of vibration data, audio data, and text data, and data cleaning, verification, conversion and other processing are performed to facilitate subsequent data storage and analysis. The original data in the internal message queue is decrypted, and the decrypted data is parsed into the specified data structure according to the data access model.

Data storage and management: The parsed heterogeneous device data can be stored in the MongoDB database in the cloud for subsequent query and analysis. The data storage and data management framework designed and developed based on the loosely coupled technical design concept can realize structured data and unstructured data storage, and can be integrated with third-party data platforms through data adaptation, such as the Hadoop big data platform.

In summary, the method of the present invention has the following technical advantages:

1. The present invention proposes a method for transmitting the data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud. In view of the problems that the amount of heterogeneous data of vehicle-mounted edge devices is large and easily causes communication congestion in cloud transmission, the method combines a multi-source heterogeneous data fusion algorithm to extract features and perform unsupervised fusion processing on heterogeneous data such as text data, vibration data and audio data, and then fuses and reduces their dimensions into fused feature data with low data volume for transmission to the cloud, thereby saving network bandwidth for cloud transmission and reducing its network load.

2. In the method for transmitting the fusion feature data of heterogeneous devices in the Internet of Vehicles to the cloud of the present invention, the fusion feature data of the vehicle-mounted edge device is analyzed and converted in the cloud and restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge device is stored. In addition, corresponding device identifiers and corresponding data models are set for different vehicle-mounted edge devices, which facilitates the differentiation of vehicle-mounted edge devices for the stored heterogeneous data such as text, audio, vibration, etc., and provides data support for subsequent equipment operation and maintenance and other applications.

3. The method of the present invention can reduce the amount of heterogeneous data of vehicle-mounted edge devices when transmitting them to the cloud, save network bandwidth for cloud transmission, and reduce its network load. At the same time, it can also well ensure the storage integrity of future data of vehicle-mounted edge devices in the cloud, thereby helping to improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information, and has great value for technology promotion and application.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims (10)

1. The cloud uploading method for data fusion characteristic transmission of the heterogeneous equipment of the Internet of vehicles is characterized by comprising the following steps of:

s1, constructing a device identifier and a data model corresponding to each vehicle-mounted side device in the cloud end Internet of things; the equipment model corresponding to the vehicle-mounted side equipment is used for identifying the individual identity of the vehicle-mounted side equipment; the data model corresponding to the vehicle-mounted side equipment is used for recording working data of the corresponding vehicle-mounted side equipment;

S2, arranging edge acquisition computing devices in a preset data acquisition area, wherein each edge acquisition computing device can be connected with a plurality of vehicle-mounted side end devices in a data communication mode, and a data transmission channel is formed between the edge acquisition computing devices and a cloud end, so that working data acquired by the vehicle-mounted side end devices can be transmitted to the cloud end through the edge acquisition computing devices;

S3, the vehicle-mounted side equipment collects working data and transmits the working data to an edge collection computing device which establishes data communication connection; the working data comprise text data, vibration data and audio data of the vehicle-mounted side-end equipment;

S4, the edge acquisition computing device performs feature extraction and unsupervised fusion processing on text data, vibration data and audio data in the working data of the vehicle-mounted side equipment to obtain fusion feature data of the vehicle-mounted side equipment, and the fusion feature data is uploaded to a cloud end through a data transmission channel;

S5, the cloud end analyzes and converts the fusion characteristic data of the vehicle-mounted side equipment, and the obtained text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted side equipment are stored as work analysis data into a data model corresponding to the vehicle-mounted side equipment.

2. The method for cloud computing transmission of data fusion features of heterogeneous equipment of the internet of vehicles according to claim 1, wherein in the step S1, a data storage space for the whole machine and parts thereof of the vehicle-mounted side equipment is provided in a data model corresponding to the vehicle-mounted side equipment, and the data storage space is used for respectively storing working data of the whole machine and working data of the parts thereof of the vehicle-mounted side equipment.

3. The method for cloud computing for data fusion feature transmission of heterogeneous internet of vehicles according to claim 1, wherein the step S2 comprises:

S201, establishing equipment access gateways between a cloud end and edge acquisition computing devices, wherein each equipment access gateway is used as a data access point of one or more edge acquisition computing devices;

S202, a data access channel is established in the cloud end and used as a data transmission channel between the cloud end and the equipment access gateway, so that working data of the vehicle-mounted side equipment acquired by the edge acquisition computing device can be uploaded to the cloud end through the equipment access gateway and the data access channel.

4. The method for cloud computing for data fusion feature transmission of heterogeneous internet of vehicles according to claim 1, wherein the step S4 comprises:

S401, carrying out data communication on the vehicle-mounted side equipment and an edge acquisition and calculation device, and acquiring working data of the vehicle-mounted side equipment through the edge acquisition and calculation device;

s402, preprocessing word segmentation, sentence segmentation and stop word removal of text data in the working data by an edge acquisition computing device;

s403, carrying out word embedding feature extraction on the preprocessed text data by the edge acquisition computing device to obtain word feature vectors of the text data;

S404, the edge acquisition computing device performs slice segmentation processing on vibration data and audio data in the working data according to time sequences to obtain vibration data slice vectors and audio data slice vectors;

And S405, performing unsupervised dimension reduction fusion processing on the word feature vector, the vibration data slice vector and the audio data slice vector of the text data by the edge acquisition computing device to obtain fusion feature data of the vehicle-mounted side equipment.

5. The method for cloud computing transmission of heterogeneous equipment data fusion features of the internet of vehicles according to claim 4, wherein in the step S401, the vehicle-mounted side equipment is in data connection with an edge acquisition computing device capable of establishing data communication, the data processing resource occupancy rate of the edge acquisition computing device is obtained, one edge acquisition computing device with the data processing resource occupancy rate smaller than a preset resource occupancy threshold is selected, and the working data of the vehicle-mounted side equipment is uploaded.

6. The method for cloud computing transmission of internet of vehicles heterogeneous equipment data fusion features according to claim 4, wherein in the step S402, the edge collection computing device performs word segmentation, sentence segmentation and stop word removal preprocessing on text data in the working data by using a Bert model which is trained in advance for text preprocessing.

7. The method for cloud computing for data fusion feature transmission of heterogeneous devices of internet of vehicles according to claim 4, wherein in the step S403, the edge collection computing device performs word embedding feature extraction on text data in chinese by using a pretrained Bert-base-chinese model, and performs word embedding feature extraction on text data in english by using a pretrained Bert-base-uncased model; the word feature vector of the obtained text data is characterized asThe method comprises the steps of representing a matrix vector of dimensions, representing a single sentence in text data as an N _i×N_x -dimensional vector, representing a single character as a 1 XN _x -dimensional vector, representing a feature dimension of feature extraction by N _x, representing the number of characters of an ith sentence in the text data by N _i, and representing the number of sentences obtained after the text data is divided into sentences by m, so that the word feature vector not only contains character information and semantic information of global characters in the text data, but also contains position information of each sentence and character.

8. The method for cloud computing transmission of data fusion features of heterogeneous equipment of the internet of vehicles according to claim 4, wherein in the step S404, the edge acquisition computing device reads vibration data and audio data in the working data respectively and performs normalization processing, and then performs slice segmentation processing on the vibration data and the audio data according to a feature dimension N _x extracted by a preset feature; the obtained vibration data slice vector is characterized as an xN _x -dimensional vector, the audio data slice vector is characterized as a yN _x -dimensional vector, and x and y respectively represent the slice segmentation segment numbers of the vibration data and the audio data, so that the vibration data slice vector and the audio data slice vector not only respectively contain amplitude characteristic information of the vibration data and the audio data, but also contain position information of each slice segmentation segment.

9. The method for cloud computing in the data fusion feature transmission of the heterogeneous equipment of the internet of vehicles according to claim 4, wherein in the step S405, the edge acquisition computing device performs splicing fusion processing on word feature vectors, vibration data slice vectors and audio data slice vectors of text data to obtain working data feature matrix vectors, and then inputs the working data feature matrix vectors into a pre-trained Autoencoder self-coding model to perform unsupervised dimension-reduction coding processing to obtain fusion feature data of the vehicle-mounted side equipment; wherein the working data feature matrix vector is characterized asThe matrix vector of the dimension, N _x, represents the feature dimension of feature extraction, N _i represents the number of characters of the ith sentence in the text data, m represents the number of sentences obtained after the text data is divided, and x and y represent the number of slice segmentation fragments of the vibration data and the audio data respectively.

10. The method for cloud computing based data fusion feature transmission of heterogeneous devices of internet of vehicles according to claim 9, wherein in step S5, the cloud end performs Autoencoder self-coding decoding processing on the fusion feature data of the vehicle-mounted side device, and analyzes the decoding processing to obtain the dataWorking data feature matrix vectors of the vehicle-mounted side-end equipment in dimension are decomposed to obtainAnd carrying out feature recombination on word features of the text data and slice segmentation fragments of the vibration data and the audio data according to position information contained in the word feature vector, the vibration data slice vector and the audio data slice vector of the text data, respectively, carrying out data format conversion processing on respective recombination results according to a preset text data format, a vibration data format and an audio data format, obtaining text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted side equipment, and finally, taking the text recovery data, the vibration recovery data and the audio recovery data of the vehicle-mounted side equipment as work analysis data of the corresponding vehicle-mounted side equipment, and storing the work analysis data of the corresponding vehicle-mounted side equipment into a data model corresponding to the corresponding vehicle-mounted side equipment.