CN118470639A - Park digital operation management system and method based on Internet of Things technology - Google Patents

Abstract

This application relates to the field of Internet of Things technology, and more specifically discloses a digital operation management system and method for a park based on Internet of Things technology, which realizes intelligent management and safety monitoring of vehicles in the park by identifying the target objects and obtaining the license plate information of the vehicles in the park monitoring video and matching them with the vehicle information in the database. Through feature vector extraction, fusion and classifier analysis, the system can timely identify abnormal vehicles and output alarm information, effectively improving the digital operation management level of the park.

Description

Park digital operation management system and method based on Internet of Things technology

Technical Field

The present application relates to the field of Internet of Things technology, and more specifically, to a digital operation management system and method for a park based on Internet of Things technology.

Background Art

Park refers to a standard building or group of buildings that are generally planned and constructed by the government (private enterprises in cooperation with the government), with complete water supply, power supply, gas supply, communications, roads, warehousing and other supporting facilities, reasonable layout and can meet the needs of production and scientific experiments in a specific industry, "including industrial parks, industrial parks, logistics parks, urban industrial parks, science and technology parks, creative parks, etc.". With the development, in order to solve the problems and needs of traditional parks, smart parks came into being. Smart parks use cloud computing, the Internet of Things and other technologies to sense, detect, analyze, control and integrate the key links of park operations, thereby improving the park's operating efficiency, reducing operating costs, and enhancing innovation, service and management capabilities.

However, research has found that various industrial parks, service parks, and logistics parks are emerging in an endless stream under the names of intelligence and smartness. The proliferation of "smart parks" does not seem to bring practical help to the operation and management of the parks, except for the redundant names and gimmicks. The effect is very low and the level of intelligence is not high. Moreover, many of them have only added temperature and humidity sensors, and added a certain degree of data visualization and park IT projects, and are also named smart parks.

Smart parks often have a large number of vehicles entering, such as visitor vehicles, vehicles transporting goods, etc. In order to ensure the safety management requirements within the park, with the help of smart video technology analysis, it is possible to timely detect and warn of abnormal external vehicles entering, and take emergency measures for abnormal vehicle incidents.

Therefore, a digital operation management system and method for a park based on Internet of Things technology is desired.

Summary of the invention

In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a digital operation management system and method for a park based on the Internet of Things technology, which realizes intelligent management and safety monitoring of park vehicles through video target recognition and license plate information matching based on the Internet of Things technology.

Accordingly, according to one aspect of the present application, a digital operation and management system for a park based on Internet of Things technology is provided, which includes:

The park vehicle information collection module is used to obtain the target vehicle monitoring video and obtain all vehicle information in the database;

The park vehicle information processing module is used to extract the license plate area enhancement feature vector from the target vehicle monitoring video, and encode all the vehicle information in the database to obtain the license plate information feature vector of each vehicle information;

The park vehicle information fusion module is used to fuse the license plate area enhancement feature vector and the license plate information feature vector of each vehicle information to obtain a classification feature vector based on each vehicle information in the database, and perform probability-driven feature adjustment on the classification feature vector based on each vehicle information in the database to obtain an optimized classification feature vector based on each vehicle information in the database;

The park vehicle information analysis module is used to pass the optimized classification feature vector based on each vehicle information in the database through the classifier to obtain the classification result corresponding to each vehicle information, and the classification result is used to indicate whether the license plate information of the target vehicle matches each vehicle in the database;

The park vehicle information output module is used to output warning information based on the classification result when the license plate information of the target object does not match all the vehicle information in the database.

According to another aspect of the present application, a digital operation and management method for a park based on Internet of Things technology is also provided, which includes:

Obtain surveillance video of the target vehicle and obtain all vehicle information in the database;

Extracting a license plate area enhancement feature vector from the target vehicle monitoring video, and encoding all vehicle information in the database to obtain a license plate information feature vector of each vehicle information;

The license plate region enhancement feature vector and the license plate information feature vector of each vehicle information are respectively fused to obtain a classification feature vector based on each vehicle information in the database, and probability-driven feature adjustment is performed on the classification feature vector based on each vehicle information in the database to obtain an optimized classification feature vector based on each vehicle information in the database;

Passing the optimized classification feature vectors based on each vehicle information in the database through a classifier to obtain classification results corresponding to each vehicle information, wherein the classification results are used to indicate whether the license plate information of the target vehicle matches each vehicle in the database;

Based on the classification result, when the license plate information of the target object does not match all the vehicle information in the database, an alarm message is output.

Compared with the existing technology, the digital operation management system and method of the park based on the Internet of Things technology provided by this application can realize intelligent management and safety monitoring of vehicles in the park by identifying the target objects and obtaining the license plate information of the vehicles in the park monitoring video and matching them with the vehicle information in the database. Through feature vector extraction, fusion and classifier analysis, the system can timely identify abnormal vehicles and output alarm information, effectively improving the digital operation management level of the park.

BRIEF DESCRIPTION OF THE DRAWINGS

By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

FIG1 is a block diagram of a digital operation and management system for a campus based on Internet of Things technology according to an embodiment of the present application.

FIG2 is a block diagram of a park vehicle information processing module in a park digital operation management system based on Internet of Things technology according to an embodiment of the present application.

FIG3 is a block diagram of a park vehicle monitoring video processing unit in a park digital operation management system based on Internet of Things technology according to an embodiment of the present application.

FIG4 is a block diagram of a park database extraction unit in a park digital operation and management system based on Internet of Things technology according to an embodiment of the present application.

FIG5 is a flowchart of a digital operation and management method for a park based on Internet of Things technology according to an embodiment of the present application.

DETAILED DESCRIPTION

Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

FIG1 illustrates a block diagram of a digital operation and management system for a campus based on Internet of Things technology according to an embodiment of the present application. As shown in FIG1 , a digital operation and management system 100 for a park based on the Internet of Things technology according to an embodiment of the present application includes: a park vehicle information acquisition module 110, which is used to obtain a target vehicle monitoring video and obtain all vehicle information in a database; a park vehicle information processing module 120, which is used to extract a license plate area enhancement feature vector from the target vehicle monitoring video, and encode all vehicle information in the database to obtain a license plate information feature vector of each vehicle information; a park vehicle information fusion module 130, which is used to fuse the license plate area enhancement feature vector and the license plate information feature vector of each vehicle information to obtain a classification feature vector based on each vehicle information in the database, and perform a probability-driven feature adjustment on the classification feature vector based on each vehicle information in the database to obtain an optimized classification feature vector based on each vehicle information in the database; a park vehicle information analysis module 140, which is used to pass the optimized classification feature vector based on each vehicle information in the database through a classifier to obtain a classification result corresponding to each vehicle information, and the classification result is used to indicate whether the license plate information of the target vehicle matches each vehicle in the database; a park vehicle information output module 150, which is used to output an alarm message based on the classification result when the license plate information of the target object does not match all vehicle information in the database.

In an embodiment of the present application, the park vehicle information acquisition module 110 is used to obtain the target vehicle monitoring video and obtain all vehicle information in the database. It should be understood that by obtaining the target vehicle monitoring video, the whereabouts of the target vehicle can be monitored and tracked in real time to ensure the safety and order of the vehicles in the park. The information of all vehicles in the database can be compared with the information of the target vehicle to verify the legality and authenticity of the target vehicle and prevent unauthorized vehicles from entering the park. Comprehensive acquisition of vehicle information can help park managers better understand the situation of vehicles in the park, detect abnormal situations in time and take necessary safety management measures. By acquiring all vehicle information in the database, the system can identify and monitor violations in the park, such as unauthorized parking or vehicles entering and leaving the park. Therefore, acquiring the target vehicle monitoring video and all vehicle information in the database is to improve the efficiency, safety and monitoring capabilities of park vehicle management.

In the embodiment of the present application, the park vehicle information processing module 120 is used to extract the enhanced feature vector of the license plate area from the target vehicle monitoring video, and to encode all the vehicle information in the database to obtain the license plate information feature vector of each vehicle information. It should be understood that extracting the enhanced feature vector of the license plate area can help the system accurately identify the license plate information of the vehicle and realize the automatic vehicle identification function. Encoding all the vehicle information in the database into the license plate information feature vector can easily and quickly match and compare the vehicle information, thereby realizing efficient management and query of the vehicle information. Through the extraction and encoding of the license plate information feature vector, real-time monitoring and tracking of vehicles in the park can be realized, helping to ensure the safety and order in the park. Using the feature vector technology, intelligent management of vehicle information can be realized, including functions such as vehicle entry and exit records and parking management, improving the efficiency and accuracy of park management. Therefore, extracting the enhanced feature vector of the license plate area from the target vehicle monitoring video and encoding all the vehicle information in the database into the license plate information feature vector can help realize the automation and intelligence of vehicle identification, management and monitoring, and improve the efficiency and safety of park vehicle management.

Specifically, in one embodiment of the present application, FIG2 illustrates a block diagram of a park vehicle information processing module in a park digital operation management system based on the Internet of Things technology according to an embodiment of the present application. As shown in FIG2, in the above-mentioned park digital operation management system 100 based on the Internet of Things technology, the park vehicle information processing module 120 includes: a park vehicle monitoring video processing unit 121, which is used to extract key frames from the target vehicle monitoring video and then encode them to obtain the license plate area enhanced feature vector; a park database extraction unit 122, which is used to generate a license plate information feature vector corresponding to each vehicle information based on all vehicle information in the database.

Accordingly, in a specific example of the present application, the park vehicle monitoring video processing unit 121 is used to extract key frames from the target vehicle monitoring video and then encode them to obtain the enhanced feature vector of the license plate area. It should be understood that extracting key frames and encoding them can improve the accuracy of the features of the license plate area. Key frames usually contain the main information of the vehicle. By encoding these key frames, the features of the license plate area can be extracted more accurately, which helps to accurately identify the license plate information. By extracting key frames and encoding them, the amount of data processed and the complexity of calculation can be reduced, and the processing efficiency can be improved. In this way, the required information can be extracted from the monitoring video more quickly, and real-time or near real-time vehicle identification and management can be achieved. Encoding can help extract key features of the license plate area, such as license plate number, vehicle color and other information. These feature vectors can be used for subsequent vehicle identification, comparison and management, providing the system with richer information. By automatically extracting key frames and encoding them, the system can realize automatic processing of vehicle information, reduce manual intervention, and improve the intelligence level of the system. Therefore, extracting key frames from the target vehicle monitoring video and encoding them to obtain the enhanced feature vector of the license plate area can improve the accuracy, efficiency and automation level of the vehicle recognition and management system, and provide better technical support for park vehicle management.

Further, FIG3 illustrates a block diagram of a park vehicle monitoring video processing unit in a park digital operation management system based on the Internet of Things technology according to an embodiment of the present application. As shown in FIG3, in the park vehicle information processing module 120 of the above-mentioned park digital operation management system 100 based on the Internet of Things technology, the park vehicle monitoring video processing unit 121 includes: a park vehicle key frame extraction subunit 1211, which is used to extract multiple vehicle monitoring key frames from the target vehicle monitoring video; a park vehicle target detection subunit 1212, which is used to pass the multiple vehicle monitoring key frames through the license plate target detection network to obtain multiple license plate information interest area maps; a park vehicle input tensor subunit 1213, which is used to arrange the multiple license plate information interest area maps into a license plate information input tensor and then pass convolution coding to obtain the license plate area enhanced feature vector.

Specifically, the park vehicle key frame extraction subunit 1211 is used to extract multiple vehicle monitoring key frames from the target vehicle monitoring video. It should be understood that extracting multiple key frames can capture different states and positions of the target vehicle at different time points. These key frames can provide more comprehensive and richer information, which is helpful for more accurate identification and tracking of the vehicle. Different key frames may show different angles, lighting conditions and motion states of the vehicle. By extracting multiple key frames, the risk of misidentification or missed identification caused by a single key frame can be reduced, and the accuracy of vehicle identification can be improved. There may be jitter or blur in the vehicle monitoring video, resulting in some frames not being clear or stable enough. By extracting multiple key frames, frames with high clarity and good stability can be selected as references to improve the stability of vehicle identification. The movement trajectory and behavior of the vehicle in the monitoring video may need to be fully displayed through multiple key frames. Extracting multiple key frames can help the system better understand the movement trajectory and behavior pattern of the vehicle and improve the analysis and prediction capabilities of the vehicle behavior. Different key frames may show different angles and features of the vehicle. By integrating information from multiple angles, the appearance characteristics of the target vehicle can be more comprehensively understood, which helps to improve the accuracy and robustness of vehicle identification. Therefore, extracting multiple vehicle monitoring key frames from the target vehicle monitoring video can improve the accuracy, stability and comprehensiveness of the vehicle identification system, and provide more reliable data support for subsequent vehicle identification, tracking and management. Accordingly, the park vehicle key frame extraction subunit is used to extract multiple vehicle monitoring key frames from the target vehicle monitoring video at a predetermined sampling frequency.

Specifically, the park vehicle target detection subunit 1212 is used to pass the multiple vehicle monitoring key frames through the license plate target detection network to obtain multiple license plate information interest area maps. It should be understood that the license plate area in each key frame can be accurately located through the license plate target detection network. This helps to accurately locate the position of the license plate when extracting the license plate information to avoid omission or misidentification. The license plate target detection network can identify the license plate area and extract the license plate information of interest. In this way, key recognition information can be extracted from the vehicle monitoring key frame, providing accurate input for subsequent license plate recognition and information extraction. By performing license plate target detection separately on each key frame, the entire license plate recognition process can be decomposed into multiple independent tasks, reducing complexity and computational complexity. This helps to improve the efficiency and accuracy of the system. Different key frames may have different lighting, angles and occlusion conditions. By processing each key frame separately, the license plate target detection network can better adapt to the license plate recognition requirements in different scenarios and improve the robustness of the system.

Specifically, the park vehicle input tensor subunit 1213 is used to arrange the multiple license plate information area of interest maps into a license plate information input tensor and then perform convolution coding to obtain the license plate area enhanced feature vector. It should be understood that convolution coding is an effective feature extraction method, and can learn distinguishing features from the license plate information area of interest map. This helps to capture the key features of the license plate area and improve the accuracy of license plate recognition. Convolutional neural networks can effectively retain the spatial structure information of input data, which is particularly important for processing image data. Through convolution coding, the spatial information in the license plate information area of interest map can be fully utilized to improve the robustness of license plate recognition. After arranging multiple license plate information area of interest maps as input tensors, the features between different license plate information can be fused and integrated through convolution coding. This helps to integrate the feature information of each license plate area and improve the comprehensive performance of license plate recognition. Through convolution coding, the high-dimensional data of the license plate information area of interest map can be converted into a more compact and abstract feature vector representation. This helps to reduce data dimensions and reduce computational complexity while retaining important feature information. The license plate area enhanced feature vector obtained through convolutional coding has better generalization ability, can better adapt to the recognition requirements of different license plate samples, and improve the performance of the system in practical applications.

Correspondingly, the park vehicle input tensor subunit includes: a three-dimensional convolution kernel secondary subunit, which is used to arrange the multiple license plate information area of interest maps into a license plate information input tensor and then obtain a license plate area feature map by using a first convolution neural network with a three-dimensional convolution kernel; a residual dual attention secondary subunit, which is used to obtain a license plate area enhanced feature map by using a residual dual attention mechanism model for the license plate area feature map; a feature map dimensionality reduction secondary subunit, which is used to perform global mean pooling on each feature matrix of the license plate area enhanced feature map along the channel dimension to obtain the license plate area enhanced feature vector.

Further, the three-dimensional convolution kernel secondary subunit is used to arrange the multiple license plate information interest area maps into a license plate information input tensor and then obtain a license plate area feature map by using a first convolution neural network with a three-dimensional convolution kernel. It should be understood that a convolution neural network (CNN) is a deep learning model that is good at extracting features from image data. By using a convolution kernel to perform a convolution operation, the network can learn local features in the image, thereby helping to identify key information in the license plate area. The three-dimensional convolution kernel can perform convolution operations in three dimensions, effectively retaining the spatial information of the input data. For two-dimensional image data such as the license plate information interest area map, the three-dimensional convolution kernel can better capture the spatial structural features of the image. The convolution operation has the characteristic of parameter sharing, which means that the same convolution kernel is used to extract features at different positions during the convolution process. This can reduce the number of model parameters and improve the generalization ability of the model. By stacking multiple convolution layers, the network can gradually learn more abstract and advanced feature representations. The first convolution layer is usually responsible for extracting low-level features, such as edges and textures, which helps subsequent layers learn more advanced features, such as shapes and patterns. The activation function in the convolutional neural network introduces nonlinear transformations, allowing the network to learn complex nonlinear relationships. This helps improve the network's representation ability, enabling it to better distinguish different types of license plate information. Therefore, the first convolutional neural network using a three-dimensional convolution kernel can effectively extract the features of the license plate area image and provide useful information for subsequent license plate recognition tasks.

Correspondingly, the three-dimensional convolution kernel secondary subunit is used to: use each layer of the three-dimensional convolution neural network to perform convolution processing, pooling processing and non-linear activation processing based on the three-dimensional convolution kernel on the input data in the forward pass of the layer so as to output the license plate area feature map by the last layer of the three-dimensional convolution kernel.

Furthermore, the residual dual attention secondary subunit is used to obtain a license plate area enhanced feature map by using the residual dual attention mechanism model. It should be understood that the residual connection can help solve the gradient vanishing and gradient explosion problems in the deep neural network training process, and help to better propagate the gradient, thereby improving the stability and effect of network training. Through the residual connection, the information in the license plate area feature map can be effectively enhanced. The attention mechanism can help the network focus on important feature parts and ignore unimportant parts. By introducing the attention mechanism, the network can learn the importance of different positions in the license plate area feature map, thereby improving the representation ability of the features. The dual attention mechanism can consider global information and local information at the same time, which helps the network to better understand the contextual information and local details of the entire feature map. This can make the representation of the license plate area feature map more comprehensive and accurate. Through the residual dual attention mechanism model, the useful information in the license plate area feature map can be effectively enhanced, and the representation ability and discrimination of the features can be improved. This helps to improve the accuracy and performance of subsequent license plate recognition or other related tasks. Therefore, by using the residual dual attention mechanism model to process the license plate area feature map, the feature information can be effectively enhanced and the feature representation ability can be improved, thereby obtaining a more discriminative and accurate license plate area enhanced feature map.

Correspondingly, the residual dual attention secondary sub-unit includes: a license plate area spatial attention tertiary sub-unit, which is used to pass the license plate area feature map through the spatial attention module of the residual dual attention mechanism model to obtain a license plate area spatial attention map; a license plate area channel attention tertiary sub-unit, which is used to pass the license plate area feature map through the channel attention module of the residual dual attention mechanism model to obtain a license plate area channel attention map; a license plate area fusion weighted tertiary sub-unit, which is used to calculate the position points between the license plate area spatial attention map and the license plate area channel attention map to obtain a license plate area weighted feature map; a license plate area fusion feature tertiary sub-unit, which is used to fuse the license plate area feature map and the license plate area weighted feature map to obtain an enhanced feature map of the license plate area.

Specifically, the license plate area spatial attention three-level subunit is used to pass the license plate area feature map through the spatial attention module of the residual dual attention mechanism model to obtain the license plate area spatial attention map. It should be understood that in image processing tasks, spatial information is crucial for understanding the shape, structure and position of objects. Through the spatial attention module, the network can learn the relationship between different spatial positions, so as to better capture the spatial information in the license plate area feature map. Features at different positions in the license plate area feature map may have different importance. Through the spatial attention module, the network can learn which spatial positions are most critical for identifying the license plate area, thereby strengthening the feature representation of these positions and improving the recognition accuracy. The spatial attention mechanism can help the network focus on specific spatial positions, so that the network pays more attention to important areas, thereby improving the representation ability of features. This helps the network to more effectively distinguish features at different positions and improve the quality of the license plate area feature map. The spatial attention module can help the network comprehensively consider the information at different positions in the entire license plate area feature map, rather than simply processing each position independently. This helps the network to more comprehensively understand the feature distribution of the entire license plate area and improve the accuracy and robustness of recognition.

Specifically, the license plate area channel attention tertiary subunit is used to pass the license plate area feature map through the channel attention module of the residual dual attention mechanism model to obtain the license plate area channel attention map. It should be understood that in a deep neural network, the feature representations of different channels may have different importance. Through the channel attention module, the network can learn which channels are more critical for identifying the target in the license plate area feature map, thereby enhancing the representation of these channels in a targeted manner. The channel attention module helps the network focus on the most informative channels, thereby reducing the impact of redundant information and improving the efficiency and representation of feature representation. Through the channel attention mechanism, the network can better utilize the correlation and complementarity between different channels, thereby generating a more representational and discriminative license plate area feature representation. Therefore, processing the license plate area feature map through the channel attention module can help the network learn and utilize the information between different channels more effectively, thereby improving the expression ability and recognition performance of the license plate area features.

Specifically, the license plate area fusion weighted three-level subunit is used to calculate the position points between the license plate area spatial attention map and the license plate area channel attention map to obtain the license plate area weighted feature map. It should be understood that by adding the spatial attention map and the channel attention map by position points, the spatial position information and the channel information can be effectively fused together. This helps to comprehensively consider the importance of different positions and channels, making the final feature map more representative and discriminative. The spatial attention map and the channel attention map capture the important relationship between the spatial position and the channel respectively, and the global information can be comprehensively considered through the addition operation. This helps the network to more comprehensively understand the spatial and channel distribution of the license plate area features and improve the accuracy and robustness of the feature representation. The addition operation is actually a weighted combination of different attention maps, where the weight of each position point is jointly determined by the spatial attention map and the channel attention map. This weighted feature representation can make the network pay more attention to important positions and channels and improve the representation ability of features. By adding the spatial attention map and the channel attention map, some irrelevant information can be eliminated and important information can be enhanced, thereby improving the quality of the license plate area feature map. This helps the network better understand and identify the license plate area and improves task performance. Therefore, adding the license plate area spatial attention map and the license plate area channel attention map by position point to obtain the license plate area weighted feature map can fully integrate the spatial position and channel information, improve the representation ability of feature representation, and enable the network to more effectively identify and understand the characteristics of the license plate area.

Specifically, the license plate area fusion feature three-level subunit is used to fuse the license plate area feature map and the license plate area weighted feature map to obtain the license plate area enhanced feature map. It should be understood that the fusion of the license plate area feature map and the weighted feature map can combine the original information in the original feature map with the information after weighted processing by the attention mechanism. Doing so helps to retain the detail information in the original feature map while emphasizing the key features processed by the attention mechanism. Fusion of two types of feature maps can improve the feature expression ability of the license plate area. The weighted feature map emphasizes the features processed by the attention mechanism, while the original feature map retains more comprehensive original information. The fusion of the two can make the final feature map more distinguishable and representative. The license plate area enhanced feature map combines the information of the original feature map and the weighted feature map, and can better reflect the key features of the license plate area. This helps to improve the performance of subsequent license plate recognition or other tasks, making the model more robust and generalized. Therefore, the fusion of the license plate area feature map and the license plate area weighted feature map can obtain the license plate area enhanced feature map, thereby improving the feature expression ability and recognition performance.

Correspondingly, the feature map dimension reduction secondary subunit is used to perform global mean pooling on each feature matrix along the channel dimension of the license plate area enhancement feature map to obtain the license plate area enhancement feature vector. It should be understood that global mean pooling can retain the spatial dimension information in the feature map, but reduce the dimension in the channel dimension. By performing mean pooling on the features on each channel, the information of each channel can be integrated into a single value, thereby reducing the dimension of the data and reducing the computational complexity. The global mean pooling operation can retain the global information of the entire feature map, not just the local information. This helps to capture the important features of the entire license plate area, rather than being limited to local details, thereby improving the representativeness and robustness of the features. Through global mean pooling, the overfitting of the model during the training process can be reduced. Because global mean pooling integrates the information in the feature map, reduces the dimension of the feature map, reduces the model complexity, and helps to avoid overfitting the training data. Global mean pooling can help extract the most important features in the entire license plate area image, because the pooling operation averages the entire feature map, thereby highlighting important features and helping to improve the representation ability of the features. Therefore, by performing global mean pooling on each feature matrix of the license plate area enhanced feature map along the channel dimension, we can effectively extract global information, reduce the dimension, reduce overfitting, and highlight important features, thereby obtaining a more representative and robust license plate area enhanced feature vector.

Accordingly, in a specific example of the present application, the park database extraction unit 122 is used to generate a license plate information feature vector corresponding to each vehicle information based on all vehicle information in the database. It should be understood that by generating a license plate information feature vector of the vehicle information, the license plate information can be converted into a feature vector with a numerical representation, thereby facilitating computer processing and analysis. This helps to extract key features in the license plate information. Generating a license plate information feature vector can map the license plate information of different vehicles into a feature space, so that the similarity or difference between different vehicle information can be compared by calculating the similarity between the feature vectors. Generating a license plate information feature vector based on vehicle information can be used for pattern recognition and classification tasks. By extracting and vectorizing the features of the license plate information, a machine learning model can be trained or other algorithms can be applied to identify the pattern of a specific vehicle information. Representing the license plate information as a feature vector can be used for retrieval and matching of vehicle information. By comparing the feature vectors of different vehicle information, rapid retrieval and matching of vehicle information can be achieved, for example, in a vehicle management system or a security monitoring system. Therefore, generating a license plate information feature vector corresponding to each vehicle information based on all vehicle information in the database can help extract key features, perform similarity comparisons, perform pattern recognition, and achieve functions such as retrieval and matching.

Further, FIG4 illustrates a block diagram of a park database extraction unit in a park digital operation and management system based on the Internet of Things technology according to an embodiment of the present application. As shown in FIG4, in the park vehicle information processing module 120 of the above-mentioned park digital operation and management system 100 based on the Internet of Things technology, the park database extraction unit 122 includes: a park database segmentation subunit 1221, which is used to segment all vehicle information in the database to obtain a segment sequence corresponding to each vehicle information; a park database segment semantic encoding subunit 1222, which is used to segment each segment in the segment sequence corresponding to each vehicle information and then pass it through the context encoder containing the embedding layer to obtain a segment semantic feature vector corresponding to each segment; a park database cascade subunit 1223, which is used to cascade the segment semantic feature vectors of each segment to obtain a license plate information feature vector of each vehicle information.

Accordingly, the park database segmentation subunit 1221 is used to segment all vehicle information in the database to obtain a segment sequence corresponding to each vehicle information. It should be understood that vehicle information usually contains multiple parts, such as license plate number, vehicle type, owner information, etc. By segmenting the vehicle information, it can be structured into smaller units (segments), which helps to extract and process the information of each part. Segmentation can help extract the key features of each part. Different parts of information may have different importance for subsequent tasks, and segmentation helps to accurately capture these key features. Segmenting the vehicle information into a segment sequence can better understand the context of the information. Each segment can represent a specific information unit, and by analyzing the segment sequence, the overall meaning of the vehicle information can be better understood. Segmenting the vehicle information into a segment sequence can make the information easier to process. When performing feature extraction, semantic encoding or other processing, it is more efficient to operate on each segment than to directly process the entire vehicle information. Some vehicle information may have a complex structure and contain multiple sub-information units. Through segmentation, this complex structure can be better dealt with to ensure that each part is properly processed and analyzed.

Correspondingly, the park database segment semantic encoding subunit 1222 is used to perform word segmentation processing on each segment in the segment sequence corresponding to each vehicle information and then pass it through the context encoder containing the embedding layer to obtain the segment semantic feature vector corresponding to each segment. It should be understood that word segmentation processing helps to convert text information into smaller semantic units, so as to better understand the meaning of each segment. By decomposing each segment into smaller words or phrases, the semantic information therein can be captured more accurately. After word segmentation, each word or phrase can be regarded as a feature, and through the context encoder of the embedding layer, these features can be mapped to a vector representation in a high-dimensional space, so as to better represent the semantic features of each word or phrase. The context encoder can consider the position and relationship of each word or phrase in its context, so as to better grasp the semantic information of the entire segment. This helps to avoid ambiguity and improve the accuracy of semantic expression. By combining the semantic feature vectors of each word or phrase, the semantic feature vector of the entire segment can be obtained. This cascade processing helps to comprehensively consider the information of each part of the segment and improve the understanding and representation ability of the entire segment. By using the context encoder of the embedding layer to generate segment semantic feature vectors, richer and more efficient input data can be provided for subsequent model training, which helps to improve the performance and accuracy of the model.

Specifically, the park database cascade subunit 1223 is used to cascade the segment semantic feature vectors of each segment to obtain the license plate information feature vector of each vehicle information. It should be understood that cascading the semantic feature vectors of each segment can integrate the information of each part into one vector. This helps to combine the features of different parts of the vehicle information to form a more comprehensive representation. Through the cascading operation, it can be ensured that the semantic feature vectors of each segment have the same dimension. Doing so helps to ensure that the information of each part can be effectively combined to form a complete license plate information feature vector. Cascading the feature vectors of each segment can retain the information of each part and avoid information loss. This ensures that the license plate information feature vector contains the semantic information of all segments and maintains the integrity of the information. The cascading operation can combine the feature vectors of each segment into a richer feature representation. This helps to improve the representation ability of the license plate information feature vector, making it more suitable for subsequent tasks such as classification, retrieval, etc.

In the embodiment of the present application, the park vehicle information fusion module 130 is used to fuse the license plate area enhancement feature vector and the license plate information feature vector of each vehicle information to obtain a classification feature vector based on each vehicle information in the database, and perform probability-driven feature adjustment on the classification feature vector based on each vehicle information in the database to obtain an optimized classification feature vector based on each vehicle information in the database. It should be understood that the fusion of the license plate area enhancement feature vector and the license plate information feature vector of the vehicle information can comprehensively consider the characteristics of the license plate area and the vehicle information, so that the final classification feature vector is more representative and comprehensive. The license plate area enhancement feature vector and the license plate information feature vector of the vehicle information may capture different levels and types of information respectively. By fusing these two feature vectors, the representation ability of the features can be enhanced and the accuracy of classification can be improved. Comprehensive consideration of the characteristics of the license plate area and vehicle information helps to improve the performance of the classification model. By fusing features from different sources, the vehicle information can be described more comprehensively, thereby improving the accuracy and robustness of the classification. Fusing and merging the features from different sources can reduce the loss and loss of information. This can ensure that the final classification feature vector contains as much useful information as possible and improve the classification effect. By fusing different types of feature vectors, personalized classification of different vehicle information can be achieved. This helps to better distinguish different types of vehicles and improve the level of personalization of classification.

In particular, in the technical solution of the present application, it is considered that if the classification feature vectors based on the vehicle information of the database have a high degree of consistency in expression, similar vehicle information (such as similar license plate numbers or patterns) will generate similar feature vectors, thereby reducing the possibility of misidentification. In an actual monitoring environment, the quality of the captured images of vehicles may be different due to various factors (such as different lighting conditions, vehicle occlusion, camera viewing angle changes, etc.). Improving the consistency of the classification feature vectors based on the vehicle information of the database can help the system better adapt to these changes and enhance the robustness of the system. Consistent classification feature vectors based on the vehicle information of the database can simplify the design of the classifier, because the classifier can more easily learn to distinguish different categories of patterns from highly consistent features. This helps to improve the generalization ability and accuracy of the classifier. In a security monitoring system, both false positives (misidentifying non-target vehicles as target vehicles) and false negatives (failure to identify the true target vehicle) need to be avoided. Improving the consistency of feature vectors helps to reduce the occurrence of these two situations. Based on this, in order to improve the expression consistency of the classification feature vector based on each vehicle information in the database in its distribution direction, in the technical solution of the present application, the classification feature vector based on each vehicle information in the database is subjected to probability-driven feature adjustment.

Specifically, in one embodiment of the present application, the classification feature vector based on each vehicle information in the database is subjected to a probability-driven feature adjustment to obtain an optimized classification feature vector based on each vehicle information in the database, including: performing a probability-driven feature adjustment on the classification feature vector based on each vehicle information in the database to obtain a modulated feature vector; calculating the position point multiplication between the modulated feature vector and the classification feature vector based on each vehicle information in the database to obtain the optimized classification feature vector based on each vehicle information in the database. Furthermore, the classification feature vector based on each vehicle information in the database is subjected to probability-driven feature adjustment to obtain a modulated feature vector, including: calculating the mean and variance of a feature set composed of all feature values of the classification feature vector based on each vehicle information in the database; using the inverse of the variance of the feature set as the exponent of the natural constant to calculate an exponential function value with the natural constant as the base to obtain a first exponential function value; subtracting one from the first exponential function value to obtain a first difference; using the mean of the feature set as the exponent of the natural constant to calculate an exponential function value with the natural constant as the base to obtain a second exponential function value; dividing the feature value at a predetermined position of the classification feature vector based on each vehicle information in the database by the first difference and then subtracting the second exponential function value, and then passing through a ReLU function to obtain a first activation value; multiplying the first activation value by the feature value at a predetermined position of the classification feature vector based on each vehicle information in the database, taking the absolute value, and then passing through a softmax function to obtain the feature value at a predetermined position of the modulated feature vector.

Specifically, the classification feature vector based on each vehicle information in the database is subjected to probability-driven feature adjustment to obtain a modulation feature vector, including:

The classification feature vector based on each vehicle information in the database is subjected to probability-driven feature adjustment to obtain a modulation feature vector using the following formula, wherein the formula is:

Among them, _vi represents the eigenvalue of the i-th position of the classification feature vector based on each vehicle information in the database, μ and σ are the mean and variance of the feature set composed of all eigenvalues of the classification feature vector based on each vehicle information in the database, ReLU represents the linear rectification function, softmax represents the normalized exponential function, and _vi ' represents the eigenvalue of the i-th position of the modulation feature vector.

Specifically, in order to improve the expression consistency of the classification feature vectors based on the vehicle information in the database in their distribution direction, in the technical solution of the present application, the classification feature vectors based on the vehicle information in the database are subjected to probability-driven feature adjustment, which is performed by probabilistically responsively adjusting the classification feature vectors based on the vehicle information in the database, and activating directional recursion using the statistical characteristics of the feature distribution, thereby accurately inferring the directional distribution of the features at each sampling position. Furthermore, a directional squeeze-incentive mechanism supported by Bayesian inference composed of a ReLU-softmax function is used to obtain the sampling position confidence value with enhanced attention, so as to improve the expression consistency of each classification feature vector based on the vehicle information in the database in its distribution direction.

In the embodiment of the present application, the park vehicle information analysis module 140 is used to pass the optimized classification feature vector based on each vehicle information in the database through the classifier to obtain the classification result corresponding to each vehicle information, and the classification result is used to indicate whether the license plate information of the target vehicle matches each vehicle in the database. It should be understood that by classifying the vehicle information through the classifier, the feature vector of the target vehicle can be compared and matched with the vehicle information in the database. The classification result can indicate which vehicle in the database is most similar or matched to the target vehicle. The classifier can help determine whether the license plate information of the target vehicle matches the vehicle information in the database. Through the classification result, it can be known whether there is a match between the target vehicle and the vehicle in the database, thereby improving the recognition accuracy of the vehicle information. Automatically classifying and matching the vehicle information through the classifier can reduce manual intervention and improve matching efficiency. The classification result can intuitively indicate the matching situation of the license plate information of the target vehicle and each vehicle in the database. Using the classification result to indicate the matching situation of the license plate information of the target vehicle and the vehicle in the database, real-time matching and identification can be achieved, providing timely and effective information for fields such as security monitoring and vehicle management.

In an embodiment of the present application, the park vehicle information output module 150 is used to output an alarm message based on the classification result when the license plate information of the target object does not match all the vehicle information in the database. It should be understood that if the license plate information of the target vehicle does not match all the vehicle information in the database, this may mean that the target vehicle is not in the database, and there may be security risks or abnormal situations. Outputting an alarm message can help attract attention and take necessary safety measures. The mismatch may indicate that the license plate information of the target vehicle does not match any known vehicle information, which may be due to abnormal situations such as incorrect vehicle information entry, car theft, forged license plates, etc. Outputting an alarm message can help detect abnormal situations in a timely manner. Outputting an alarm message helps ensure that the vehicle information in the database is consistent with the actual situation. When the license plate information of the target vehicle does not match all the vehicle information in the database, data update or verification may be required to ensure the accuracy and completeness of the database.

In summary, the digital operation management system and method of the park based on the Internet of Things technology in the embodiment of this application realizes intelligent management and safety monitoring of vehicles in the park by identifying the target objects and obtaining the license plate information of the vehicles in the park monitoring video and matching them with the vehicle information in the database. Through feature vector extraction, fusion and classifier analysis, the system can timely identify abnormal vehicles and output alarm information, effectively improving the digital operation management level of the park.

As described above, the digital operation and management system 100 for a park based on the Internet of Things technology according to the embodiment of the present application can be implemented in various terminal devices, such as a server of the digital operation and management system for a park based on the Internet of Things technology. In one example, the digital operation and management system 100 for a park based on the Internet of Things technology can be integrated into a terminal device as a software module and/or a hardware module. For example, the digital operation and management system 100 for a park based on the Internet of Things technology can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the digital operation and management system 100 for a park based on the Internet of Things technology can also be one of the many hardware modules of the terminal device.

Alternatively, in another example, the campus digital operation and management system 100 based on the Internet of Things technology and the terminal device may also be separate devices, and the campus digital operation and management system 100 based on the Internet of Things technology may be connected to the terminal device via a wired and/or wireless network and transmit interactive information in accordance with an agreed data format.

FIG5 is a flow chart of a digital operation and management method for a park based on the Internet of Things technology according to an embodiment of the present application. As shown in FIG5, the digital operation and management method for a park based on the Internet of Things technology according to an embodiment of the present application includes the following steps: S110, obtaining a target vehicle monitoring video, and obtaining all vehicle information in a database; S120, extracting a license plate area enhancement feature vector from the target vehicle monitoring video, and encoding all vehicle information in the database to obtain a license plate information feature vector for each vehicle information; S130, respectively fusing the license plate area enhancement feature vector and the license plate information feature vector of each vehicle information to obtain a classification feature vector based on each vehicle information in the database, and performing a probability-driven feature adjustment on the classification feature vector based on each vehicle information in the database to obtain an optimized classification feature vector based on each vehicle information in the database; S140, passing the optimized classification feature vector based on each vehicle information in the database through a classifier to obtain a classification result corresponding to each vehicle information, and the classification result is used to indicate whether the license plate information of the target vehicle matches each vehicle in the database; S150, based on the classification result, when the license plate information of the target object does not match all vehicle information in the database, outputting an alarm message.

Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned campus digital operation and management method based on Internet of Things technology have been introduced in detail in the description of the campus digital operation and management system based on Internet of Things technology with reference to Figures 1 to 4 above, and therefore, its repeated description will be omitted.

The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and/or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and/or recombined. Such decomposition and/or recombination should be regarded as equivalent solutions of the present disclosure.

Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims (10)

1. The utility model provides a garden digital operation management system based on internet of things which characterized in that includes:

The park vehicle information acquisition module is used for acquiring a target vehicle monitoring video and acquiring all vehicle information in the database;

The park vehicle information processing module is used for extracting license plate region enhancement feature vectors from the target vehicle monitoring video and encoding all vehicle information in the database to obtain license plate information feature vectors of each vehicle information;

The park vehicle information fusion module is used for respectively fusing the license plate region enhancement feature vector and the license plate information feature vector of each vehicle information to obtain a classification feature vector based on each vehicle information of the database, and carrying out feature adjustment based on probability driving on the classification feature vector based on each vehicle information of the database to obtain an optimized classification feature vector based on each vehicle information of the database;

The park vehicle information analysis module is used for respectively passing the optimized classification feature vectors based on the vehicle information of the database through a classifier to obtain classification results corresponding to the vehicle information, wherein the classification results are used for indicating whether license plate information of the target vehicle is matched with the vehicles of the database;

and the park vehicle information output module is used for outputting alarm information when the license plate information of the target object is not matched with all the vehicle information of the database based on the classification result.

2. The system for managing digital operations of a campus based on technology of internet of things according to claim 1, wherein the information processing module of the campus vehicle comprises:

The park vehicle monitoring video processing unit is used for extracting key frames from the target vehicle monitoring video and then obtaining the license plate region enhancement feature vector through coding;

And the park database extraction unit is used for generating license plate information feature vectors corresponding to all the vehicle information in the database.

3. The system for managing digital operations of a campus based on the internet of things technology according to claim 2, wherein the video processing unit for monitoring the vehicles of the campus comprises:

A park vehicle key frame extraction subunit, configured to extract a plurality of vehicle monitoring key frames from the target vehicle monitoring video;

the park vehicle target detection subunit is used for respectively passing the plurality of vehicle monitoring key frames through a license plate target detection network to obtain a plurality of license plate information interested region diagrams;

And the park vehicle input tensor subunit is used for arranging the license plate information interesting area graphs into license plate information input tensors and obtaining the license plate area enhancement feature vectors through convolution coding.

4. The system for managing digital operations of a campus based on technology of internet of things according to claim 3, wherein the campus vehicle key frame extraction subunit is configured to: a plurality of vehicle monitoring key frames are extracted from the target vehicle monitoring video at a predetermined sampling frequency.

5. The system for managing digital operations of a campus based on internet of things according to claim 4, wherein the campus vehicle input tensor subunit comprises:

The three-dimensional convolution kernel secondary subunit is used for arranging the license plate information interesting area graphs into license plate information input tensors and then obtaining a license plate area feature graph through a first convolution neural network using a three-dimensional convolution kernel;

the residual double-attention secondary subunit is used for obtaining a license plate region enhancement feature map by using a residual double-attention mechanism model from the license plate region feature map;

And the feature map dimension reduction secondary subunit is used for carrying out global mean value pooling on each feature matrix of the license plate region enhancement feature map along the channel dimension so as to obtain the license plate region enhancement feature vector.

6. The system for managing digital operations in a campus based on internet of things according to claim 5, wherein the three-dimensional convolution kernel two-level subunit is configured to: and respectively carrying out convolution processing, pooling processing and nonlinear activation processing based on a three-dimensional convolution kernel on input data in forward transmission of layers by using each layer of the three-dimensional convolution neural network so as to output the license plate region feature map by the last layer of the three-dimensional convolution kernel.

7. The system for managing digital operations of a campus based on the internet of things according to claim 6, wherein the campus database extraction unit comprises:

The park database segmentation subunit is used for carrying out segmentation processing on all the vehicle information in the database to obtain segment sequences corresponding to each vehicle information;

the park database segment semantic coding subunit is used for obtaining segment semantic feature vectors corresponding to each segment through the context encoder containing the embedded layer after word segmentation processing is carried out on each segment in the segment sequence corresponding to each vehicle information;

And the park database cascading subunit is used for cascading the segment semantic feature vectors of each segment to obtain license plate information feature vectors of each piece of vehicle information.

8. The system for managing digital operations in a campus based on the internet of things according to claim 7, wherein performing feature adjustment based on probability driving on the classification feature vector based on each vehicle information in the database to obtain an optimized classification feature vector based on each vehicle information in the database, comprises:

performing probability-driven feature adjustment on the classification feature vector based on the vehicle information of the database to obtain a modulation feature vector;

And calculating the position-based point multiplication between the modulation characteristic vector and the classification characteristic vector based on the vehicle information of the database to obtain the optimized classification characteristic vector based on the vehicle information of the database.

9. The system for managing digital operations in a campus based on technology of internet of things according to claim 8, wherein performing feature adjustment based on probability driving on the classification feature vector based on each piece of vehicle information in the database to obtain a modulation feature vector, comprises:

calculating the mean and variance of a feature set consisting of all feature values of the classification feature vector based on the vehicle information of the database;

calculating an exponential function value based on a natural constant by taking the inverse number of the variance of the feature set as an index of the natural constant to obtain a first exponential function value;

subtracting one from the first exponential function value to obtain a first difference value;

calculating an index function value based on the natural constant by taking the average value of the feature set as an index of the natural constant to obtain a second index function value;

dividing the characteristic value of the preset position of the classification characteristic vector based on the vehicle information of the database by the first difference value and subtracting the second index function value, and then obtaining a first activation value through a ReLU function;

Multiplying the first activation value by the characteristic value of the preset position of the classification characteristic vector based on the vehicle information of the database, taking the absolute value again, and obtaining the characteristic value of the preset position of the modulation characteristic vector through a softmax function.

10. The park digital operation management method based on the Internet of things technology is characterized by comprising the following steps of:

acquiring a target vehicle monitoring video and acquiring all vehicle information in a database;

Extracting license plate region enhancement feature vectors from the target vehicle monitoring video, and coding all vehicle information in the database to obtain license plate information feature vectors of each vehicle information;

Respectively fusing the license plate region enhancement feature vector and the license plate information feature vector of each piece of vehicle information to obtain a classification feature vector based on each piece of vehicle information of the database, and carrying out feature adjustment based on probability driving on the classification feature vector based on each piece of vehicle information of the database to obtain an optimized classification feature vector based on each piece of vehicle information of the database;

The optimized classification feature vector based on the information of each vehicle in the database is respectively passed through a classifier to obtain a classification result corresponding to the information of each vehicle, wherein the classification result is used for indicating whether license plate information of a target vehicle is matched with each vehicle in the database;

and outputting alarm information when the license plate information of the target object is not matched with all the vehicle information of the database based on the classification result.