CN118379664A - Video identification method and system based on artificial intelligence - Google Patents

Abstract

The present invention proposes a video recognition method and system based on artificial intelligence, the method includes: constructing an adaptive framework based on video content analysis, dynamically adjusting the model structure and parameters according to the complexity of real-time video frames; applying efficient image preprocessing technology and deep learning algorithms to extract features from video frames according to model parameters; using generative adversarial networks to process the features extracted from video frames, stylizing key frames to obtain stylized video frames, and strengthening visual behavioral features; designing hybrid neural networks to perform behavior recognition and behavior prediction based on historical data according to stylized video frames. The present invention not only significantly improves the accuracy and efficiency of video recognition, but also enhances the real-time processing capability of the system and the foresight of future behavior prediction, effectively solving the core problems existing in the prior art.

Description

A video recognition method and system based on artificial intelligence

Technical Field

The present invention belongs to the technical field of artificial intelligence, and in particular relates to a video recognition method and system based on artificial intelligence.

Background technique

In the current technical environment, video recognition systems have been widely used in many fields such as security monitoring, content recommendation, and traffic management. Their core function is to automatically parse and identify video content through computer vision technology and deep learning algorithms. Traditional video recognition systems usually rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to process video frames and identify objects, actions, or events in them. These systems can achieve high accuracy in standardized environments, but they still have many shortcomings when facing complex and changing practical application scenarios. For example, the recognition accuracy of traditional models will drop significantly when dealing with behaviors in very complex or dynamically changing backgrounds. In addition, most existing systems need to rely on a large amount of labeled data for training, which is not only costly, but also further affects the generalization ability and practicality of the model when the quality of data annotation is inconsistent. In addition, existing video recognition technology still faces technical challenges in real-time performance processing, adaptive adjustment of models, and prediction of future behaviors.

Therefore, the adaptability of existing technologies to complex scenarios, processing efficiency, and learning ability of unlabeled data need to be improved.

Summary of the invention

The purpose of the present invention is to design a video recognition method and system based on artificial intelligence, which integrates multiple new technologies for intelligent video recognition and prediction to solve the above problems.

In order to achieve the above object, a first aspect of the present invention provides a video recognition method based on artificial intelligence, the method comprising the following steps:

S1. Build an adaptive framework based on video content analysis to dynamically adjust the model structure and parameters according to the complexity of real-time video frames;

S2. According to the model parameters, efficient image preprocessing technology and deep learning algorithm are applied to extract features from the video frame; wherein the image preprocessing technology includes illumination correction and noise removal; the feature extraction from the video frame is specifically to use the improved Sobel operator to extract the edge information of the video frame, and then use the local binary pattern operator to analyze the image texture features, specifically:

Among them, G _x and G _y represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

Where _gc represents the central pixel value, _gp represents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y) is the local binary pattern value calculated at the image coordinate (x,y);

S3. Use the generative adversarial network to process the features extracted from the video frames, stylize the key frames to obtain stylized video frames, and strengthen the visual behavioral features, including:

S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

Where _Fk represents the feature extraction function for the key feature k, K represents the predefined associated feature set, _λk represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, I represents the original frame, and G(z) represents the output of the generator G when the input is the hidden variable z;

The total loss function is expressed as follows:

Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network;

S303, training the generative adversarial network using a progressive training method;

S4. Designing a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network;

The method further comprises at least the following steps:

A. Integrate unsupervised learning algorithms to dynamically adjust the models and parameters of S1-S4 based on real-time feedback;

B. Establish a feedback mechanism to dynamically adjust strategies based on user and performance feedback and continuously optimize strategy execution.

Furthermore, the S1 specifically includes:

S101, input a video frame sequence, each frame of the video is first processed by a specific feature extraction sub-network;

S102, using the feature vector difference between two consecutive frames to evaluate the degree of scene change, including:

D _t = || V _t - _{V t-1} || ₂

Where _Dt represents the content change amplitude of the frame between time t and t-1, _Vt represents the feature vector of each frame of video at time t, and Vt _-1 represents the feature vector of each frame of video at time t-1;

The scene complexity C _t is obtained by averaging these differences through a sliding window W of fixed size:

S103, automatically adjusting the configuration of the network according to the comparison result between C _t and the preset threshold θ;

S104. In combination with real-time performance feedback, the back propagation algorithm is used to adjust network weights and structural parameters.

Furthermore, the illumination correction is to apply automatic white balance and exposure compensation to the input video frame, which is expressed as follows:

Where I(x,y) is the original pixel value, I _corr (x,y) is the corrected pixel value, I _min and I _max are the minimum and maximum pixel values in the frame respectively;

The noise removal is to apply a bilateral filter to remove potential image noise and maintain edge clarity, which is expressed as follows:

Where _Wp is the normalization coefficient, _fr and _fs are Gaussian functions based on intensity and spatial approximation, respectively, to ensure that only adjacent pixels with similar intensities affect the current pixel value, (x,y) represents the coordinates of the pixel, (x',y') represents the coordinates of the adjacent pixels involved in the calculation when applying the filter, and I _filtered (x,y) represents the pixel value at the coordinate (x,y) of the image after bilateral filtering.

Furthermore, the input of the generative network is the feature vector of each frame of video at time t, and then multiple convolutional layers are used, each layer is followed by batch normalization and ReLU activation function, and the last layer uses tanh activation function to output a stylized image, which is expressed as follows:

I _styled =tanh(Conv(BN(ReLU(Conv(…V _t …)))))

The input of the discriminant network includes I _styled and the original frame I. Multiple convolutional layers are used for feature extraction. After each convolutional layer, normalization and LeakyReLU activation functions are connected. The last layer uses the sigmoid function to output the probability, which is expressed as follows:

P _real =σ(Conv(LReLU(BN(Conv(…I _styled ,I…)))))

Among them, σ represents the sigmoid activation function.

Furthermore, in S303, the progressive training method means first training the network at a low resolution and gradually transitioning to a high resolution.

Furthermore, the hybrid neural network includes a feature fusion layer, a temporal fusion layer and a behavior recognition and prediction output layer; the feature fusion layer uses a deep convolutional network to extract the spatial features of each frame of the image, which is expressed as follows:

F _spatial = CNN(I _styled )

Among them, F _spatial represents the spatial feature vector extracted from the stylized image;

The temporal fusion layer takes the feature vector F _spatial of consecutive frames as input and processes the temporal dependency and dynamic changes through the recurrent neural network module, which is expressed as follows:

_St =RNN(F _spatial ,S _t-1 )

Among them, _St represents the hidden state at time t, which carries the accumulated information of past video frames; St _-1 represents the hidden state at time t-1;

The behavior recognition and prediction output layer combines F _spatial and _St to generate the final behavior recognition and behavior prediction output, which is expressed as follows:

Y _acti on＝Softmax(Dense(S _t ))

Y _predict =Softmax(Dense(S _t ,F _spatial ))

Among them, Y _action is the action recognition result of the current frame, and Y _predict is the result of predicting future actions based on current and past information.

Furthermore, the cross entropy loss function is used to simultaneously optimize the accuracy of behavior recognition and prediction, which is expressed as follows:

Among them, y _c is the one-hot encoding of the true behavior label, is the probability distribution predicted by the model, and C is the number of behavior categories.

Furthermore, the step A specifically comprises the following steps:

A feature transformation network F' is constructed using a reversible feature transformation technique to extract and transform features from the original video frames;

Design reconstruction and anomaly scoring using autoencoder-based anomaly detection techniques to detect abnormal behaviors or salient events in video frames;

Define the total loss function Reconstruction error and behavior recognition error The weighted sum of is calculated by introducing additional regularization terms to enhance the generalization and adaptability of the model, and a dynamic weight adjustment mechanism based on feedback is introduced to adjust the weights;

Integrate dynamic elements to adapt to changing video content and optimize feature transformation networks;

Introduce a model adaptive adjustment strategy to automatically adjust the network architecture or parameters based on the currently detected behavior patterns and historical data.

Furthermore, the step B specifically comprises the following steps:

Regularly collect feedback data and process the collected data using statistical analysis and machine learning methods;

Based on feedback data, define parameter adjustment strategies and dynamically adjust learning rates according to changes in model performance;

Perform comprehensive evaluations periodically and update the model's operating parameters based on the evaluation results, which are then fed back to the operating parameter adjustment function and the learning rate update strategy.

In a second aspect of the present invention, a video recognition system based on artificial intelligence is provided, the system comprising:

An adaptive framework building module is used to build an adaptive framework based on video content analysis and dynamically adjust the model structure and parameters according to the complexity of real-time video frames;

The algorithm design module is used to extract features from video frames by applying efficient image preprocessing technology and deep learning algorithm according to model parameters; wherein the image preprocessing technology includes illumination correction and noise removal; the feature extraction from the video frame is specifically to extract edge information of the video frame using an improved Sobel operator, and then analyze image texture features using a local binary pattern operator, specifically:

Among them, G _x and G _y represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

Where _gc represents the central pixel value, _gp represents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y0) is the local binary pattern value calculated at the image coordinate (x,y);

The enhanced feature module is used to process the features extracted from the video frames using a generative adversarial network, stylize the key frames to obtain stylized video frames, and enhance the visual behavioral features, including:

S301, respectively designing a generative adversarial network, including a generative network and a discriminative network;

S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

Where _Fk represents the feature extraction function for the key feature k, K represents the predefined associated feature set, _λk represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, I represents the original frame, and G(z) represents the output of the generator G when the input is the hidden variable z;

The total loss function is expressed as follows:

Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network;

S303, training the generative adversarial network using a progressive training method;

A behavior prediction module, used to design a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network;

The method further comprises at least 0 of the following modules:

Unsupervised learning algorithm module, used to integrate unsupervised learning algorithms and dynamically adjust the models and parameters of S1-S4 based on real-time feedback;

The feedback mechanism optimization module is used to establish a feedback mechanism, dynamically adjust strategies based on user and performance feedback, and continuously optimize strategy execution.

The beneficial technical effects of the present invention are at least as follows:

The present invention proposes an intelligent video recognition and prediction system integrating multiple new technologies in a video recognition method and system based on artificial intelligence. First, an adaptive learning model dynamic adjustment technology is introduced, which can automatically adjust the structure and parameters of the deep learning model according to the complexity of the input video, which is crucial to improving the adaptability and efficiency of the system in a changing environment. Secondly, through the stylized processing of video frames based on generative adversarial networks (GANs), the system can improve the accuracy and robustness of action recognition through visual information without adding additional sensors, especially in visually complex or low-quality video data. In addition, combined with time series analysis and video-to-video translation technology, the system can not only identify the current behavior, but also predict the next possible behavior sequence, which is particularly valuable in security monitoring and emergency response systems. Finally, the use of unsupervised learning technology enables the system to continuously learn and optimize the model without labeled data, solving the problem of relying on a large amount of labeled data in the prior art. The combination of these technologies not only significantly improves the accuracy and efficiency of video recognition, but also enhances the real-time processing capability of the system and the foresight of future behavior prediction, effectively solving the core problems existing in the prior art.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

FIG1 is a flow chart of a video recognition method based on artificial intelligence according to an embodiment of the present invention.

FIG2 is a framework diagram of a video recognition system based on artificial intelligence according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

In one or more embodiments, as shown in FIG1 , a video recognition method based on artificial intelligence is disclosed. The method includes steps 1 to 6, including:

S1. Build an adaptive framework based on video content analysis to dynamically adjust the model structure and parameters according to the complexity of real-time video frames, including:

S101, input a video frame sequence, each frame of the video is first processed by a specific feature extraction sub-network;

Specifically, a sequence of video frames is input, and each frame is first processed by a specific feature extraction subnetwork. A simplified ResNet model is used, which is lightweight and designed for real-time data streams. Through this network, each frame of video is converted into a feature vector V _t , where V _t is a vector of fixed dimension, containing the key visual information of the frame, such as edges, textures, color distribution, etc.

S102, using the feature vector difference between two consecutive frames to evaluate the degree of scene change, including:

D _t = || V _t - _{V t-1} || ₂

Where _Dt represents the content change amplitude of the frame between time t and t-1, _Vt represents the feature vector of each frame of video at time t, and Vt _-1 represents the feature vector of each frame of video at time t-1;

The scene complexity C _t is obtained by averaging these differences through a sliding window W of fixed size:

Where i represents the i-th moment, and _Di represents the frame content change at the i-th time point;

S103, automatically adjusting the configuration of the network according to the comparison result between C _t and the preset threshold θ;

Specifically, according to the comparison result between C _t and the preset threshold θ, the system automatically adjusts the configuration of the network. If C _t exceeds the threshold θ, it indicates that the scene changes greatly and a more complex model is needed to capture the details. At this time, the system may increase the depth of the convolution layer or the size of the convolution kernel; conversely, if C _t is lower than θ, the network structure is simplified, such as reducing the convolution layer or using a smaller convolution kernel, to increase the processing speed and reduce the consumption of computing resources.

S104, combining real-time performance feedback, using back propagation algorithm to adjust network weights and structural parameters;

In the embodiment of the present invention, combined with real-time performance feedback, the system uses the back propagation algorithm to adjust the network weights and structural parameters to meet the needs of the current video content. This includes adjusting the learning rate, optimizing the gradient descent strategy, etc., ensuring that each adjustment is based on the latest performance evaluation data, thereby minimizing errors and optimizing overall performance.

S2. According to the model parameters, applying efficient image preprocessing technology and deep learning algorithm to extract features from the video frame; wherein the image preprocessing technology includes illumination correction and noise removal;

Here, lighting correction means applying automatic white balance and exposure compensation to the input video frames to standardize the lighting conditions. Use the formula:

Among them, I(x,y) is the original pixel value, I _corr (x,y) is the corrected pixel value, I _min and I _max are the minimum and maximum pixel values in the frame respectively.

Noise removal is to apply a bilateral filter to remove potential image noise, maintain edge clarity, and calculate the new pixel value using the following formula:

Where _Wp is the normalization coefficient, _fr and _fs are Gaussian functions based on intensity and spatial approximation, respectively, to ensure that only adjacent pixels with similar intensities affect the current pixel value, (x,y) represents the coordinates of the pixel, (x',y') represents the coordinates of the adjacent pixels involved in the calculation when applying the filter, and I _filtered (x,y0) represents the pixel value at the coordinate (x,y) of the image after bilateral filtering.

The feature extraction from the video frame is specifically to use the improved Sobel operator to extract the edge information of the video frame, and then use the local binary pattern operator to analyze the image texture features, specifically:

Among them, G _x and G _y represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

Where _gc represents the central pixel value, _gp represents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y0) is the local binary pattern value calculated at the image coordinate (x,y).

As an embodiment of the present invention, through the above steps, video frame preprocessing will standardize the input data and reduce the interference of environmental variables, while feature extraction focuses on obtaining visual information that is critical to subsequent behavior analysis. In addition, the physical basis methods of illumination correction and noise filtering ensure the universal applicability and efficiency of the processing process, while edge detection and texture analysis provide the necessary detailed visual description for video content analysis, laying a solid foundation for subsequent in-depth analysis. Such a processing flow ensures that the video surveillance system can maintain efficient and accurate operation under various environmental conditions and meet the needs of real-time monitoring.

S3. Use the generative adversarial network to process the features extracted from the video frames, stylize the key frames to obtain stylized video frames, and strengthen the visual behavioral features, including:

S301, respectively designing a generative adversarial network, including a generative network and a discriminative network;

Specifically, the design _input of the generative network (G) is the feature vector of each frame of the video obtained in step 2. The network structure uses multiple convolutional layers, each layer is followed by batch normalization and ReLU activation function, and the last layer uses tanh activation function to output stylized images. The goal of the generative network is to generate visually striking images based on the input feature vector, highlighting the key content, as shown below:

I _styled =tanh(Conv(BN(ReLU(Conv(…V _t …))))) (5)

The discriminant network (D) is designed with inputs of I _styled and the original frame I. It uses multiple convolutional layers for feature extraction. After each convolution layer, it is connected to the normalization and LeakyReLU activation function. The last layer uses the sigmoid function to output the probability of whether the image is real or generated by the generative network, which is expressed as follows:

P _real =σ(Conv(LReLU(BN(Conv(…I _styled ,I…)))))) (6)

Among them, σ represents the sigmoid activation function, which is used to convert the output into a probability value.

S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

Where _Fk represents the feature extraction function for key feature k, K represents the predefined key feature set, _λk represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, I represents the original frame, and G(z) represents the output of the generator G when the input is the hidden variable z;

The total loss function is expressed as follows:

Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term, and D represents the discriminator network;

S303, training the generative adversarial network using a progressive training method;

Specifically, a progressive training method is adopted to first train the network at a low resolution and gradually transition to a high resolution. This method helps the network capture large-scale behavior patterns at an early stage and then gradually refine to important details at a small scale, thereby more effectively processing complex or blurred video scenes.

S4. According to the stylized video frames, a hybrid neural network is designed to perform behavior recognition and behavior prediction based on historical data; wherein the hybrid neural network includes a CNN network and an RNN network, which are specifically expressed as follows:

The hybrid neural network includes a feature fusion layer, a temporal fusion layer and a behavior recognition and prediction output layer;

The feature fusion layer uses a deep convolutional network to extract the spatial features of each frame of image, which is expressed as follows:

F _spatial =CNN(I _styled ) (9)

Among them, F _spatial represents the spatial feature vector extracted from the stylized image;

The temporal fusion layer takes the feature vector F _spatial of consecutive frames as input and processes the temporal dependency and dynamic changes through the recurrent neural network module, which is expressed as follows:

S _t =RNN (F _spatial ,S _t-1 ) (10)

Among them, _St represents the hidden state at time t, which carries the accumulated information of past video frames; St _-1 represents the hidden state at time t-1;

The behavior recognition and prediction output layer combines F _spatial and _St to generate the final behavior recognition and behavior prediction output, which is expressed as follows:

Among them, Y _action is the action recognition result of the current frame, and Y _predict is the result of predicting future actions based on current and past information.

The cross entropy loss function is used to simultaneously optimize the accuracy of behavior recognition and prediction, which is expressed as follows:

Among them, y _c is the one-hot encoding of the true behavior label, is the probability distribution predicted by the model, and C is the number of behavior categories.

S5, integrates unsupervised learning algorithms to dynamically adjust the models and parameters of S1-S4 based on real-time feedback, including:

A feature transformation network F' is constructed using a reversible feature transformation technique to extract and transform features from the original video frames.

In the embodiment of the present invention, a small convolutional neural network is constructed, whose goal is to extract and transform features from the original video frames to better meet the model recognition requirements. The network uses reversible feature transformation technology to not only extract features, but also enhance adaptability to key variables such as illumination changes and motion simulation, as shown below:

F′＝FTN(I _frame ) (13)

Among them, F' is the transformed feature vector, and I _frame is the input video frame.

Design reconstruction and anomaly scoring using autoencoder-based anomaly detection techniques to detect unusual behaviors or salient events in video frames.

Specifically, abnormal behaviors or prominent events in video frames are detected using an anomaly detection technique based on an autoencoder, which attempts to reconstruct input features, and the reconstruction error is used to judge anomalies. The reconstruction and anomaly scores are expressed as follows:

Among them, a high anomaly score indicates that the current frame may contain abnormal or unlearned behavior features.

Define the total loss function Reconstruction error and behavior recognition error At the same time, an additional regularization term is introduced to enhance the generalization ability and adaptability of the model, and a dynamic weight adjustment mechanism based on feedback is introduced to adjust the weights.

Specifically, define the total loss function Reconstruction error and behavior recognition error The weighted sum of , but an additional regularization term is introduced to enhance the generalization ability and adaptability of the model, which is expressed as follows:

in, Measures the accuracy of reconstruction after feature transformation; is the regularization term of the model, such as L2 regularization, which is used to avoid overfitting; α, β, γ are weight parameters, which are adjusted according to the importance of different tasks.

Then, a dynamic weight adjustment mechanism based on feedback is introduced. The values of α, β, and γ are automatically adjusted according to the detection frequency of abnormal behaviors in real-time monitoring data to optimize the performance of the model in a changing environment, as shown below:

Where f, g, h are functions that adjust weights based on historical performance feedback.

Integrate dynamic elements to optimize feature transformation network to adapt to changing video content.

In the embodiment of the present invention, the feature transformation network not only performs feature encoding and decoding, but also integrates dynamic elements to adapt to the ever-changing video content. For example, time-dependent components such as small RNNs or attention mechanisms are introduced to enhance the understanding of time series and feature extraction capabilities:

F″＝Attention(F,context) (17)

Among them, context is the contextual information extracted from the video sequence, which enhances the feature vector's ability to describe the current scene.

In the embodiment of the present invention, a model adaptive adjustment strategy is introduced to automatically adjust the network architecture or parameters based on the currently detected behavior pattern and historical data. For example, the model pruning technology or neural network search (NAS) is used to automatically adjust the complexity and structure of the network to adapt to different behavior recognition requirements.

This step not only improves the accuracy of behavior recognition, but also enhances the system's ability to adapt to new environments and unknown behavior patterns by comprehensively considering the reconstruction error, behavior recognition accuracy and dynamic optimization of model complexity. It is suitable for complex and dynamically changing monitoring scenarios.

S6. Establish a feedback mechanism to dynamically adjust strategies based on user and performance feedback and continuously optimize strategy execution, including:

Feedback data is collected regularly and processed using statistical analysis and machine learning methods.

Specifically, the system regularly collects data including user feedback (error reports, annotation corrections), performance indicators automatically generated by the system (recognition accuracy, processing delay), etc. Statistical analysis and machine learning methods are used to process the collected data, identify common problems and performance bottlenecks, and output performance improvement reports and adjustment suggestions.

Based on feedback data, define parameter adjustment strategies and dynamically adjust learning rates according to model performance changes, including:

Automatically adjust the learning rate to increase or decrease the learning rate based on the performance evaluation results;

When new behavior patterns are detected to occur frequently, it may be necessary to adjust the network structure (such as increasing the number of layers, adjusting the filter size) to better learn these features;

Adjust the weights of each item in the loss function according to the needs of the specific task, and make the adjustment in a formula:

Among them, θ represents the model parameters, _ηt is the dynamically adjusted learning rate, is a composite loss function that takes feedback into account, _Rt is the adjustment suggestion from feedback, and data includes performance data and user feedback;

Then dynamically adjust the learning rate according to the performance changes of the model. When the model performance improves, gradually reduce the learning rate to refine the learning; when the model performance decreases, increase the learning rate to quickly adapt to new situations. The update strategy formula is:

η _t+1 =η _t ·(1+δ·sign(ΔPerf _t )) (19)

Among them, δ is the adjustment factor, ΔPerf _t is the performance change rate, and the sign function gives the corresponding sign according to whether the performance is improved or deteriorated.

Perform comprehensive evaluations periodically and update the model's operating parameters based on the evaluation results, which are then fed back to the operating parameter adjustment function and the learning rate update strategy.

In the embodiment of the present invention, the system is periodically comprehensively evaluated, including precision, recall rate and F1 score, and compared with previous performance data. The operating parameters of the system are updated according to the evaluation results and fed back to the parameter adjustment function and the learning rate update strategy.

In one or more embodiments, as shown in FIG2 , the present invention discloses a video recognition system based on artificial intelligence, comprising:

An adaptive framework building module 101 is used to build an adaptive framework based on video content analysis, and dynamically adjust the model structure and parameters according to the complexity of real-time video frames;

The algorithm design module 102 is used to extract features from the video frame by applying efficient image preprocessing technology and deep learning algorithm according to the model parameters; wherein the image preprocessing technology includes illumination correction and noise removal; the feature extraction from the video frame is specifically to extract the edge information of the video frame by using the improved Sobel operator, and then analyze the image texture features by using the local binary pattern operator, specifically:

Among them, G _x and G _y represent the horizontal and vertical gradients respectively, and G represents the final edge strength;

Where _gc represents the central pixel value, _gp represents the neighborhood pixel value, and P represents the number of pixels in the neighborhood; LBP(x,y) represents the LBP feature, which reflects the relative strength of the current pixel and its surrounding neighborhood; (x,y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP(x,y) is the local binary pattern value calculated at the image coordinate (x,y);

The feature enhancement module 103 is used to process the features extracted from the video frames using a generative adversarial network, stylize the key frames to obtain stylized video frames, and enhance the visual behavioral features, including:

S301, respectively designing a generative adversarial network, including a generative network and a discriminative network;

S302, designing a feature-emphasized loss term to generate a total loss function to train the generative network, and adding a perception layer to the discriminative network; wherein the feature-emphasized loss term It is expressed as follows:

Where _Fk represents the feature extraction function for key feature k, K represents the predefined key feature set, _λk represents the weight associated with feature k, which is used to adjust the contribution of each feature in the total loss, and I represents the original frame;

The total loss function is expressed as follows:

Among them, β represents the hyperparameter that controls the influence of the feature emphasis loss term;

S303, training the generative adversarial network using a progressive training method;

The behavior prediction module 104 is used to design a hybrid neural network for behavior recognition and behavior prediction based on historical data according to the stylized video frames; wherein the hybrid neural network includes a CNN network and an RNN network;

The method further comprises at least 0 of the following modules:

An unsupervised learning algorithm module 105, for integrating an unsupervised learning algorithm and dynamically adjusting the models and parameters of S1-S4 based on real-time feedback;

The feedback mechanism optimization module 106 is used to establish a feedback mechanism, dynamically adjust the strategy according to user and performance feedback, and continuously optimize the strategy execution.

In summary, the present invention initializes an adaptive learning model framework to set the foundation of the system, which can automatically adjust its network structure and parameters according to the complexity of the real-time video content. This adaptive ability is the key to solving the application of video surveillance in a changing environment. It ensures that no matter how complex the environment is, the system can adjust itself to achieve the best processing performance and recognition accuracy. Based on this framework, real-time video frames are preprocessed and feature extracted to ensure that the most critical information is obtained from each frame of video. This step is efficiently executed because the adaptive framework has provided it with model parameters and structures that are most suitable for the current video content, thereby optimizing the feature extraction process and improving the speed and accuracy of data processing. The extracted features are stylized using generative adversarial networks (GANs), which enhances the visual representation of key behaviors in the video frames and makes subsequent behavior recognition more accurate. Stylization is performed based on optimized features, so it can more effectively highlight behavioral features, especially in scenes with complex or low-quality visual information. A hybrid neural network is used to perform behavior recognition and future behavior prediction on the stylized video frames. This step is effective because the stylized video frames have strengthened the necessary behavioral features, allowing the neural network to learn and predict more accurately. Unsupervised online learning mechanisms are integrated into the system, allowing the model to self-optimize in actual operation, and continuously adjust parameters and strategies through real-time feedback. The introduction of this mechanism enables the system to adapt to long-term changes in the operating environment, reduces dependence on manual intervention, and improves the autonomy and reliability of the system. By establishing a system feedback mechanism, user and performance data are continuously collected to dynamically optimize the entire system. This closed-loop feedback mechanism ensures that the system can continue to improve and adjust its strategies in real time to meet new challenges and needs.

The present invention solves the problems of insufficient recognition accuracy, poor adaptability to complex environments, low data processing efficiency, and reliance on a large amount of labeled data in the background technology. Through the introduction of adaptive learning models, the system can automatically optimize processing strategies for different video contents; through the application of stylized processing and hybrid neural networks, the accuracy and predictive ability of behavior recognition are greatly improved; and the combination of unsupervised learning and feedback mechanisms ensures the long-term effectiveness and autonomous optimization capabilities of the system. This design is not only technologically innovative, but also provides significant performance improvements in practical applications, which fully meets the needs of modern intelligent video surveillance systems.

These are just some preferred embodiments of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims (10)

1. A video recognition method based on artificial intelligence, the method comprising the steps of:

s1, constructing an adaptive framework based on video content analysis, and dynamically adjusting a model structure and parameters according to the complexity of a real-time video frame;

S2, according to model parameters, an efficient image preprocessing technology and a deep learning algorithm are applied, and features are extracted from the video frames; wherein the image preprocessing technique includes illumination correction and noise division; the extracting features from the video frame specifically comprises extracting edge information of the video frame by using an improved Sobel operator, and analyzing image texture features by using a local binary pattern operator, wherein the extracting features specifically comprise the following steps:

Wherein G _x and G _y represent horizontal and vertical gradients, respectively, and G represents final edge intensity;

Where g _c denotes a center pixel value, g _p denotes a neighborhood pixel value, and P denotes the number of pixels in the neighborhood; LBP (x, y) represents LBP characteristics, reflecting the relative intensity of the current pixel and its surrounding neighborhood; (x, y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP (x, y) is a local binary pattern value calculated at the image coordinates (x, y);

S3, using the characteristics extracted from the generated countermeasure network processing video frames to stylize the key frames to obtain stylized video frames, strengthening the visual behavior characteristics, and comprising the following steps:

s301, respectively designing and generating an countermeasure network, wherein the countermeasure network comprises a generation network and a discrimination network;

S302, generating a total loss function by using a loss item with emphasized design characteristics, training a generation network, and adding a perception layer in a discrimination network; wherein the characteristic emphasizes the loss term The expression is as follows:

Wherein F _k denotes a feature extraction function for a key feature K, K denotes a predefined set of associated features, λ _k denotes weights related to feature K for adjusting the contribution of each feature in the total loss, I denotes the original frame, G (z) denotes the output of generator G at input as hidden variable z;

the total loss function is expressed as follows:

wherein, beta represents a hyper-parameter of the control characteristic to emphasize the influence of the loss term, and D represents a discriminator network;

S303, training the generated countermeasure network by adopting a progressive training method;

s4, designing a hybrid neural network to conduct behavior recognition and behavior prediction based on historical data according to the stylized video frame; wherein the hybrid neural network comprises a CNN network and an RNN network;

The method further comprises at least 0 steps of:

A. integrating an unsupervised learning algorithm, and dynamically adjusting the models and parameters of the S1-S4 based on real-time feedback;

B. and setting up a feedback mechanism, dynamically adjusting the strategy according to the user and performance feedback, and continuously optimizing the strategy execution.

2. The video recognition method based on artificial intelligence according to claim 1, wherein S1 specifically comprises:

s101, inputting a video frame sequence, wherein each frame of video is processed through a specific feature extraction sub-network;

s102, evaluating the change degree of the scene by utilizing the feature vector difference between two continuous frames, comprising:

D_t＝||V_t-V_t-1||₂

Wherein D _t represents the content variation amplitude of the frame between time t and t-1, V _t represents each frame of video feature vector at time t, and V _t-1 represents each frame of video feature vector at time t-1;

The average of these differences is calculated by a sliding window W of fixed size, resulting in scene complexity C _t:

S103, automatically adjusting the configuration of the network according to the comparison result of the C _t and the preset threshold value theta;

s104, combining real-time performance feedback, and adjusting network weight and structural parameters by using a back propagation algorithm.

3. The artificial intelligence based video recognition method of claim 1, wherein the illumination correction is to apply automatic white balance and exposure compensation to the input video frames, as follows:

Where I (x, y) is the original pixel value, I _corr (x, y) is the corrected pixel value, and I _min and I _max are the minimum and maximum pixel values, respectively, in the frame;

The noise is divided into two parts by applying a bilateral filter to remove potential image noise, and the edge definition is maintained, and is expressed as follows:

Where W _p is a normalized coefficient, f _r and f _s are gaussian functions based on intensity and spatial approximation, respectively, ensuring that only adjacent pixels with similar intensity affect the current pixel value, (x, y) represents coordinates of the pixel, (x ', y') represents coordinates of adjacent pixels involved in calculation when filtering is applied, and I _filtered (x, y) represents pixel values of the image after bilateral filtering processing at coordinates (x, y).

4. The method of claim 1, wherein the generating network inputs each frame of video feature vector at time t, then uses multiple convolution layers, each layer is followed by batch normalization and ReLU activation functions, and the last layer outputs a stylized image using a tanh activation function, expressed as follows:

I_styled＝tanh(Conv(BN(ReLU(Conv(...V_t...)))))

the input of the discrimination network comprises I _styled and an original frame I, the characteristic extraction is carried out by using a plurality of convolution layers, the normalization function and LeakyReLU activation function are accessed after each layer of convolution, and the output probability of the last layer by using a sigmoid function is expressed as follows:

P_real＝σ(Conv(LReLU(BN(Conv(...I_styled，I...)))))

where σ represents the sigmoid activation function.

5. The artificial intelligence based video recognition method according to claim 4, wherein the progressive training method in S303 is to train the network at a low resolution first, and gradually transition to a high resolution.

6. The video recognition method based on artificial intelligence according to claim 1, wherein the hybrid neural network comprises a feature fusion layer, a time sequence fusion layer and a behavior recognition and prediction output layer;

The feature fusion layer adopts a depth convolution network to extract the spatial features of each frame of image, and the spatial features are expressed as follows:

F_spatial＝CNN(I_styled)

Wherein F _spatial denotes a spatial feature vector extracted from the stylized image;

The time sequence fusion layer takes the characteristic vector F _spatial of continuous frames as input, and processes time dependence and dynamic change through a cyclic neural network module, and the time sequence fusion layer is expressed as follows:

S_t＝RNN(F_spatial,S_t-1)

S _t represents the hidden state at time t, and carries the accumulated information of the past video frames; s _t-1 represents the hidden state at time t-1;

The behavior recognition and prediction output layer in combination with F _spatial and S _t generates the final behavior recognition and behavior prediction output, expressed as follows:

Y_action＝Softmax(Dense(S_t))

Y_predict＝Softmax(Dense(S_t,F_spatial))

Where Y _action is the behavior recognition result of the current frame and Y _predict is the result of predicting future behavior based on current and past information.

7. The artificial intelligence based video recognition method of claim 6, wherein the cross entropy loss function is used to simultaneously optimize accuracy of behavior recognition and prediction, as follows:

Wherein y _c is the one-time thermal encoding of the real behavior tag, Is the probability distribution of model predictions, and C is the number of behavior categories.

8. The method for identifying video based on artificial intelligence according to claim 1, wherein the step a specifically comprises the following steps:

constructing a feature transformation network F' by using a reversible feature transformation technology, and extracting and transforming features from an original video frame;

reconstructing and anomaly scoring using a self-encoder based anomaly detection technique design to detect anomalous behavior or salient events in the video frames;

definition of the Total loss function Reconstruction errorsAnd behavior recognition errorsSimultaneously introducing additional regularization terms to enhance model generalization capability and adaptability, and simultaneously introducing a dynamic weight adjustment mechanism adjustment weight based on feedback;

integrating a video content optimization feature transformation network with dynamic element adaptation change;

a model self-adaptive adjustment strategy is introduced, and the network architecture or parameters are automatically adjusted based on the currently detected behavior pattern and historical data.

9. The method for identifying video based on artificial intelligence according to claim 1, wherein the step B specifically comprises the steps of:

Periodically collecting feedback data and processing the collected data using statistical analysis and machine learning methods;

defining a parameter adjustment strategy and dynamically adjusting the learning rate according to the performance change of the model based on the feedback data;

And periodically performing overall evaluation, updating the operation parameters of the model according to the evaluation result, and feeding back to the operation parameter adjustment function and the learning rate updating strategy.

10. An artificial intelligence based video recognition system, the system comprising:

The self-adaptive frame construction module is used for constructing a self-adaptive frame based on video content analysis and dynamically adjusting the model structure and parameters according to the complexity of the real-time video frame;

The algorithm design module is used for extracting features from the video frames by applying an efficient image preprocessing technology and a deep learning algorithm according to the model parameters; wherein the image preprocessing technique includes illumination correction and noise division; the extracting features from the video frame specifically comprises extracting edge information of the video frame by using an improved Sobel operator, and analyzing image texture features by using a local binary pattern operator, wherein the extracting features specifically comprise the following steps:

Wherein G _x and G _y represent horizontal and vertical gradients, respectively, and G represents final edge intensity;

Where g _c denotes a center pixel value, g _p denotes a neighborhood pixel value, and P denotes the number of pixels in the neighborhood; LBP (x, y) represents LBP characteristics, reflecting the relative intensity of the current pixel and its surrounding neighborhood; (x, y) represents the horizontal and vertical coordinates of the current pixel, and P represents the total number of pixels in the neighborhood; LBP (x, y) is a local binary pattern value calculated at the image coordinates (x, y);

the strengthening feature module is used for generating features extracted from the video frames processed by the countermeasure network, and carrying out stylization on the key frames to obtain stylized video frames, so as to strengthen visual behavior features, and comprises the following steps:

s301, respectively designing and generating an countermeasure network, wherein the countermeasure network comprises a generation network and a discrimination network;

S302, generating a total loss function by using a loss item with emphasized design characteristics, training a generation network, and adding a perception layer in a discrimination network; wherein the characteristic emphasizes the loss term The expression is as follows:

Wherein F _k denotes a feature extraction function for a key feature K, K denotes a predefined set of associated features, λ _k denotes weights related to feature K for adjusting the contribution of each feature in the total loss, I denotes the original frame, G (z) denotes the output of generator G at input as hidden variable z;

the total loss function is expressed as follows:

wherein, beta represents a hyper-parameter of the control characteristic to emphasize the influence of the loss term, and D represents a discriminator network;

S303, training the generated countermeasure network by adopting a progressive training method;

The behavior prediction module is used for designing a hybrid neural network to perform behavior recognition and behavior prediction based on historical data according to the stylized video frame; wherein the hybrid neural network comprises a CNN network and an RNN network;

the method further comprises at least 0 modules of:

the non-supervision learning algorithm module is used for integrating the non-supervision learning algorithm and dynamically adjusting the models and parameters of the S1-S4 based on real-time feedback;

and the feedback mechanism optimization module is used for setting up a feedback mechanism, dynamically adjusting the strategy according to the user and performance feedback, and continuously optimizing the strategy execution.