CN115805865B

CN115805865B - Heavy-duty vehicle blind spot warning method, device, equipment and storage medium

Info

Publication number: CN115805865B
Application number: CN202211562774.4A
Authority: CN
Inventors: 赵明业; 郭鹏; 赵德赢; 田磊; 赵玉超
Original assignee: China National Heavy Duty Truck Group Jinan Power Co Ltd
Current assignee: China National Heavy Duty Truck Group Jinan Power Co Ltd
Priority date: 2022-12-07
Filing date: 2022-12-07
Publication date: 2024-08-13
Anticipated expiration: 2042-12-07
Also published as: CN115805865A

Abstract

The application provides a dead zone early warning method, device and equipment for a heavy truck and a storage medium, and relates to the technical field of vehicles. The method comprises the steps of acquiring surrounding target information of a heavy truck through an ultrasonic radar and a millimeter wave radar arranged on the heavy truck, acquiring a characteristic value combination of the early warning target according to the surrounding target information and image data of the early warning target, acquiring a target characteristic value combination of optimal fitness through a genetic characteristic data fusion algorithm, inputting the target characteristic value combination into a classification model, and acquiring the type of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, the weight value of each neural network in the multi-BP neural network model is obtained according to a Monte Carlo simulation method, and determining and early warning modes according to the type of the early warning target. The application solves the problems that the coverage of blind areas is not wide enough and accurate and timely early warning cannot be ensured in the prior art.

Description

Heavy truck blind area early warning method, device, equipment and storage medium

Technical Field

The application relates to the technical field of vehicles, in particular to a method, a device, equipment and a storage medium for pre-warning dead zones of a heavy truck.

Background

The heavy-duty car has a large number of visual field blind areas in the running process, and particularly has a large number of traffic accidents caused by the visual field blind areas in complex environments such as cities or construction sites.

The earliest blind area early warning system of development belongs to the collision early warning of backing a car, through arranging ultrasonic radar at the rear of a vehicle, through ultrasonic echo detection distance rear collision thing distance, through the sound early warning to the driver when near the collision. At present, the method for using the blind area early warning system is single, mainly focuses on how to cope with special scene requirements, and provides blind area images for drivers by using a sensor to detect or using a blind-supplement camera, so that the blind area coverage is not wide enough; due to the weakness of each sensor and the defects of a system arrangement scheme, the blind area early warning system cannot ensure that collision risks in all the peripheral blind areas of the vehicle are accurately and timely reported actively, so that the blind area early warning system is still deficient in timely and accurate early warning.

Aiming at the problems of how to improve the coverage of dead zones and how to timely and accurately early warn the heavy vehicles, no relevant effective solution is proposed at present.

Disclosure of Invention

The application provides a method, a device, equipment and a storage medium for pre-warning dead zones of a heavy truck, which are used for solving the problems that the existing dead zone pre-warning system is not wide enough in coverage of dead zones and cannot guarantee accurate and timely pre-warning.

In a first aspect, the present application provides a method for early warning a dead zone of a heavy truck, including:

surrounding target information of the heavy truck is obtained through the ultrasonic radar and the millimeter wave radar, and after an early warning target is determined according to the surrounding target information, image data of the early warning target is obtained through the camera;

Acquiring a characteristic value combination of the early warning target according to the surrounding target information and the image data of the early warning target;

performing cross mutation processing on the characteristic value combination according to a genetic characteristic data fusion algorithm to obtain a target characteristic value combination with optimal fitness;

Inputting the target characteristic value combination into a classification model, and obtaining the type of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, and the weight value of each neural network in the multi-BP neural network model is obtained according to a Monte Carlo simulation method;

and determining an early warning mode according to the type of the early warning target and carrying out early warning.

In one possible design, the millimeter wave radar continuously detects a long-distance target around the heavy truck, the ultrasonic radar continuously detects a short-distance target around the heavy truck, and the acquiring the information of the target around the heavy truck by the ultrasonic radar and the millimeter wave radar includes:

Filtering invalid targets in surrounding long-distance targets according to detection information of the millimeter wave radar to obtain an early warning target and first information of the early warning target, wherein the invalid targets are targets with the size or the height smaller than a preset value, and the first information comprises the speed, the distance, the azimuth angle and the reflection energy value of the early warning target;

according to the detection information of the ultrasonic radar, determining an early warning target in the near-distance targets around the heavy truck and second information of the early warning target, wherein the second information comprises the position of the early warning target;

and obtaining the surrounding target information according to the first information and/or the second information.

In one possible design, the cross mutation processing is performed on the feature value combination according to a genetic feature data fusion algorithm to obtain a target feature value combination with optimal fitness, which includes:

Expressing the target characteristic value combination as a binary string, and randomly generating M random individuals as an initial group based on the binary string;

Wherein the fitness S _i (i=1, 2, … … M) of each individual in the initial population is achieved by the following formula:

Wherein, T _bj and T _bk respectively represent the b feature mean values of the jth and kth class targets, and sigma _bj ² and sigma _bk ² respectively represent the b feature variances of the jth and kth class targets;

The probability of each individual being selected is calculated according to the fitness S _i (i=1, 2, … … M), and the probability of the target selected individual P _i,max being selected is obtained by the following formula:

Wherein S _i,max refers to the fitness of the current target individual;

According to the crossover operation in the genetic algorithm, exchanging partial characteristic data of two different individuals in the initial population at random;

According to variation operation in a genetic algorithm, randomly performing inverse operation on characteristic data of individuals in the initial population;

and based on the individuals subjected to the crossover operation and the mutation operation, calculating the fitness S _i and the probability P _i,max of the selected target selection individual circularly until the probability P _i,max of the selected target selection individual is not changed any more, and combining the characteristics of the individual with the greatest fitness as the target characteristic value.

In one possible design, before the target feature values are combined and input into the classification model, the method further includes:

Acquiring a data set D containing m sample feature combinations; the m is an integer greater than 1, and the sample feature combination is a feature combination obtained through a genetic feature data fusion algorithm and a sample label corresponding to the feature combination;

The ith bagging (i=1, 2, … …, q), extracting data sets D _i with sample capacity of n from the data sets D in a put-back way according to uniform probability distribution, wherein each data set D _i is used for deriving a BP neural network to obtain q BP neural networks, n and q are integers, and n is smaller than m;

And obtaining the classification model according to the q BP neural networks and the weight value of each BP neural network.

In one possible design, the weight value of each BP neural network is implemented according to a monte carlo fusion algorithm by the following formula:

Wherein BP _i (j) is the input characteristic data of the ith BP neural network to the jth target, z _i is the weight value of the ith BP neural network, and multi_BP_out (j) is the identification result of the BP neural network i to the jth target;

the identification error δ _j for the jth target is set as follows, the fusion input error is misrecorded as 1, and is correctly recorded as 0, and is expressed by the following formula:

Wherein, multi_bp_out (j) +.labe (j) is used to indicate that the fusion output label is inconsistent with the sample label, and multi_bp_out (j) = labe (j) is used to indicate that the fusion output label is consistent with the sample label;

The method is converted into the following formula to solve:

The Monte Carlo method can be used for randomly generating a preset group of weight values, the recognition errors delta _total are calculated in sequence, and a group of corresponding weight values with the minimum delta _total are selected to be used as the weight values of the multi-BP neural network.

In one possible design, the determining the early warning mode according to the type of the early warning target and performing early warning includes:

determining an early warning risk level according to the type of the early warning target, the speed of the heavy truck and the vehicle corner;

Determining an early warning mode according to the early warning risk level and carrying out early warning; the early warning risk level comprises a low risk early warning level and a high risk early warning level, and the early warning mode suitable for the low risk early warning level is to flash and remind through a warning lamp; the early warning mode suitable for the high-risk early warning level is reminding through an acousto-optic combination mode.

In one possible design, the method further includes the camera entering a sleep mode when the camera detects that the current driving environment is night;

generating a characteristic value combination of the early warning target according to surrounding target information of the heavy truck, which is acquired by the ultrasonic radar and the millimeter wave radar; wherein, the quantity of the ultrasonic radar and the millimeter wave radar corresponding to night is more than that of the ultrasonic radar and the millimeter wave radar corresponding to daytime.

In a second aspect, the present application provides a heavy-duty car blind zone early warning device, including:

The acquisition module is used for acquiring surrounding target information of the heavy truck through the ultrasonic radar and the millimeter wave radar, and acquiring image data of an early warning target through the camera after determining the early warning target according to the surrounding target information;

The first processing module is used for acquiring a characteristic value combination of the early warning target according to the surrounding target information and the image data of the early warning target;

The second processing module is used for carrying out cross mutation processing on the characteristic value combination according to a genetic characteristic data fusion algorithm to obtain a target characteristic value combination with optimal fitness;

The classification module is used for inputting the target characteristic value combination into a classification model, and obtaining the type of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, and the weight value of each neural network in the multi-BP neural network model is obtained according to a Monte Carlo simulation method;

and the execution module is used for determining an early warning mode and carrying out early warning according to the type of the early warning target.

In a third aspect, the present application provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

and the processor executes the computer execution instructions stored in the memory to realize the dead zone early warning method of the heavy truck.

In a fourth aspect, the present application provides a computer readable storage medium, where computer executable instructions are stored, where the computer executable instructions are used to implement a heavy truck blind area warning method when executed by a processor.

The application provides a method, a device, equipment and a storage medium for warning a dead zone of a heavy truck, which are characterized in that surrounding target information of the heavy truck is acquired through an ultrasonic radar and a millimeter wave radar arranged on the heavy truck, after the warning target is determined, image data of the warning target is acquired through a camera, characteristic value combinations of the warning target are acquired according to the surrounding target information and the image data of the warning target, cross mutation processing is carried out on the characteristic value combinations through a genetic characteristic data fusion algorithm to obtain target characteristic value combinations with optimal fitness, the target characteristic value combinations are input into a classification model, the type of the warning target output by the classification model is acquired, wherein the classification model is a multi-back propagation BP neural network model, weight values of all neural networks in the multi-BP neural network model are acquired according to a Monte Carlo simulation method, and the warning mode is determined and the warning is carried out according to the type of the warning target.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a system composition of a dead zone early warning of a heavy truck according to an embodiment of the present application;

Fig. 2 is a schematic flow chart of a dead zone early warning method for a heavy truck according to an embodiment of the present application;

fig. 3 is a schematic flow chart II of a dead zone early warning method for a heavy truck according to an embodiment of the present application;

FIG. 4 is a layout diagram of a blind area early warning sensor provided by an embodiment of the application;

fig. 5 is a schematic structural diagram of a dead zone early warning device for a heavy truck according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a heavy truck blind area early warning electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the application, as detailed in the accompanying claims, rather than all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

First, the related concepts or nouns related to the present application are explained:

ultrasonic radar: the distance is measured by utilizing the ultrasonic wave transmitting device to transmit ultrasonic waves outwards and utilizing the time difference when the reflected ultrasonic waves are received by the receiver, and the distance measuring device is generally used for detecting a short-distance target ranging from 0.1 meter to 2 meters from a heavy vehicle.

Millimeter wave radar: the millimeter wave radar is a radar working in millimeter wave band detection, the wavelength of the millimeter wave is between the microwave and the centimeter wave, so the millimeter wave radar has the advantages of both the microwave radar and the photoelectric radar, and is generally used for detecting and tracking a static target and a dynamic target which are in the range of 0.5 meter to 50 meters away from a heavy vehicle.

BP (Back Propagation), neural network: the multi-layer feedforward network is trained according to an error counter-propagation algorithm, the learning rule is that a gradient descent method is used, the weight and the threshold value of the network are continuously adjusted through counter-propagation, the square sum of errors of the network is minimized, the BP neural network model topological structure comprises an input layer, a hidden layer and an output layer, and the learning process of the BP network consists of two processes of forward propagation of information and counter-propagation of errors.

Genetic algorithm (Genetic Algorithm, GA): the method is a calculation model of a biological evolution process simulating natural selection and genetic mechanism of Darwin biological evolution theory, is a method for searching an optimal solution by simulating the natural evolution process, takes all individuals in a group as objects, and guides a coded parameter space to perform efficient search by utilizing a randomization technology.

The blind area of the visual field of the heavy truck in the driving process is more, and when the heavy truck is driven in the environment such as a city or a construction site, the blind area of the visual field of the driver of the heavy truck is more complex, so that the blind area early warning system plays a very important role in early warning and reminding when assisting the driver to drive the vehicle.

The current blind area early warning system has a single use method, mainly focuses on the special scene requirement, provides blind area images for a driver only by adding an external hardware sensor, and has certain defects in an arrangement scheme of the sensor arranged on the heavy vehicle, so that the current blind area early warning system has defects in the problems of how to improve the coverage of the blind area and how to timely and accurately early warn, and therefore, the blind area early warning method capable of improving the coverage of the blind area, timely and accurately identifying dangerous obstacles and reminding the driver is needed.

The application provides a heavy-duty car blind area early warning method, which is characterized in that surrounding target information of a heavy-duty car is acquired through an ultrasonic radar and a millimeter wave radar arranged on the heavy-duty car, after an early warning target is determined, image data of the early warning target is acquired through a camera, characteristic value combinations of the early warning target are acquired according to the surrounding target information and the image data of the early warning target, cross mutation processing is carried out on the characteristic value combinations through a genetic characteristic data fusion algorithm, target characteristic value combinations with optimal fitness are obtained, the target characteristic value combinations are input into a classification model, the type of the early warning target output by the classification model is acquired, the classification model is a multi-back propagation BP neural network model, weight values of all neural networks in the multi-BP neural network model are acquired according to a Monte Carlo simulation method, and early warning modes are determined and early warning is carried out according to the type of the early warning target.

Fig. 1 is a schematic diagram of a system composition of a dead zone early warning of a heavy truck according to an embodiment of the present application. As shown in fig. 1, the system includes a millimeter wave radar 101, an ultrasonic radar 102, a camera 103, a blind zone early warning controller 104, and an early warning device 105.

The millimeter wave radar 101 feeds detected remote target information back to the blind area early warning controller 104, and meanwhile, the millimeter wave radar 101 can also accept a sleep command or the requirements of detection distance and sensitivity sent by the blind area early warning controller 104;

The ultrasonic radar 102 feeds detected short-distance target information back to the blind zone early-warning controller 104, and the ultrasonic radar 102 can also accept the sleep command or the requirements of detection distance and sensitivity sent by the blind zone early-warning controller;

The camera 103 is connected with the blind area early warning controller 104, the acquired image information is transmitted to an image processing unit in the blind area early warning controller 104 for processing, the camera 103 also receives a sleep command from the blind area early warning controller 104, the blind area early warning controller 104 analyzes and determines the collision risk level of a target according to target information fed back by the millimeter wave radar 101 and the ultrasonic radar 102 and the image information acquired by the camera 103, and the blind area early warning controller 104 reminds a driver of the collision risk of the corresponding level by controlling the early warning device 105 through signals such as sound and light and the like based on the determined collision risk level.

The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems by adopting specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Example 1

Fig. 2 is a schematic flow chart of a dead zone early warning method for a heavy truck according to an embodiment of the present application. As shown in fig. 2, the execution body of the embodiment may be, for example, the blind area early warning controller 104 in fig. 1, and the method includes:

S201, acquiring surrounding target information of the heavy truck through the ultrasonic radar and the millimeter wave radar, and acquiring image data of an early warning target through the camera after determining the early warning target according to the surrounding target information;

Specifically, millimeter wave radar is used for continuously detecting distant surrounding targets, and after preliminary ineffective targets such as the targets outside the isolation fence, cobbles, shrubs and the like are screened, information such as speed, distance, azimuth angle, reflected energy value and the like of effective targets is obtained, wherein the effective targets are early warning targets;

Similarly, the ultrasonic radar is used for continuously detecting a near-distance surrounding target, and after an obstacle is detected and the reflected energy exceeds a set value, the obstacle is identified as an early warning target, and information such as the position, the distance and the like of the early warning target is acquired;

When the ultrasonic radar and the millimeter wave radar do not determine the early warning target, the blind area of the heavy truck is considered to be in a safe state, the image processing unit is not started at the moment, the consumption of system resources can be reduced, the camera is in a running state at the moment, the image data transmitted by the camera is also acquired, but the image data is not processed by the image processing unit, and only when the ultrasonic radar and the millimeter wave radar determine the early warning target, the image data acquired by the camera is processed by the image processing unit;

When the camera detects that the current running environment is at night, the camera enters a sleep mode; and carrying out risk level early warning based on surrounding target information of the heavy truck obtained by the ultrasonic radar and/or the millimeter wave radar.

S202, acquiring a characteristic value combination of the early warning target according to the surrounding target information and the image data of the early warning target;

Specifically, after early warning target information fed back by the millimeter wave radar and/or early warning target information detected by the ultrasonic radar is received, an image processing unit is started, and the image processing unit maps the area where the early warning target is located into an image coordinate system according to the early warning target information such as the speed, the position and the azimuth angle of the target fed back by the millimeter wave radar and/or the ultrasonic radar, so that target identification is carried out on a certain area of an image, and the characteristic value of the early warning target is extracted;

after image data are processed through an image processing module, data of identified target feature values such as image gray scale, outline, pixel points and the like are obtained, and then information of target information fed back by a millimeter wave radar and/or an ultrasonic radar such as speed, distance, azimuth angle, reflected energy and the like is fused with the obtained target feature values after image identification, so that a feature value combination which is an early warning target is obtained.

S203, performing cross mutation processing on the characteristic value combination according to a genetic characteristic data fusion algorithm to obtain a target characteristic value combination with optimal fitness;

Specifically, the genetic characteristic data fusion algorithm is that in the selected fitness function, the characteristic value combination is screened through duplication, intersection and variation in inheritance, so that the characteristic value combination with high fitness is reserved to form a new group, the new group inherits the information of the previous generation and is superior to the previous generation, the fitness of the characteristic value combination in the group is continuously improved until a certain condition is met, and the global optimal solution, namely the target characteristic value combination with optimal fitness, is obtained;

The method comprises the steps of expressing a plurality of characteristic value combinations of an obtained early warning target into binary strings containing each characteristic value, randomly generating a plurality of characteristic value combinations as an initial group, calculating fitness of each characteristic value combination, calculating probability of each characteristic value combination being selected through fitness in selection operation, obtaining an optimal selection individual based on the selected probability, and obtaining a target characteristic value combination of optimal fitness through intersection, variation and repeated operation in a genetic algorithm, wherein when 10 continuous optimal selection individuals are not changed, the genetic characteristic data fusion algorithm tends to converge.

S204, inputting the target characteristic value combination into a classification model, and obtaining the type of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, and the weight value of each neural network in the multi-BP neural network model is obtained according to a Monte Carlo simulation method;

Specifically, after the target characteristic value combination of the optimal fitness is obtained, the next target classification recognition operation is needed, for the set design of the base classifier in the target classification recognition operation, the difference between targets in the base classifier is ensured to be large enough as much as possible, and the final classification effect is better and more accurate, so that a plurality of BP neural networks are selected to be used as the base classifier, recognition errors are reduced through fusion among the neural networks, and the stability and the accuracy of target recognition are improved;

The multi-neural network fusion algorithm needs to set the weight value of each BP neural network so as to ensure that an optimal fusion classification result is obtained, and in order to determine the assigned weight value of each BP neural network, a Monte Carlo fusion algorithm is used for searching an optimal weight value combination;

The Monte Carlo fusion algorithm is a numerical calculation method based on a theory and a method of probability and statistics, the solved problem is associated with a certain probability model, statistical simulation or sampling is realized by a computer to obtain an approximate solution of the problem, so the Monte Carlo fusion algorithm is also called a random sampling method or a statistical test method, a preset group such as 100000 groups or more weight values are randomly generated by the Monte Carlo fusion algorithm, then identification errors in the BP neural network are calculated, and a group of corresponding weight values with the minimum identification errors is selected as the weight values of BP neural network classification.

S205, determining an early warning mode according to the type of the early warning target and carrying out early warning;

Specifically, after the type of the early warning target is determined, an early warning judging algorithm is called to determine the early warning risk level, firstly, the risk warning of the early warning target is determined according to the information of the detected distance, speed and the like of the target object, and meanwhile, the signals of the speed, the rotation angle and the like of the vehicle body are combined, and secondly, the adaptive warning risk level is determined according to the finally identified type of the early warning target;

After the type of the early warning target is determined, the evaluation level of risks is changed according to different types of the early warning target, the blind zone safety early warning system gives the targets which are extremely easy to be dangerous to human bodies, such as pedestrians, bicycles, electric vehicles and the like, the blind zone safety early warning system gives the targets the highest priority and weight ratio, and the blind zone safety early warning system gives the targets with lower priority and weight ratio;

Specifically, when the millimeter wave radar or the ultrasonic radar identifies that an early warning target of a heavy vehicle is very close to the vehicle, meanwhile, the type of an early warning target is identified as a pedestrian, and the vehicle speed and the vehicle corner condition at the moment are combined, if the vehicle speed is smaller than a set value and the vehicle corner is very small, the low risk early warning level is judged and the light warning is sent out through the early warning device to slightly remind a driver; if the speed of the vehicle is greater than a set value or the vehicle has a corner, judging that the heavy vehicle collides with a pedestrian with high probability, and at the moment, reinforcing the danger early warning level is required, and sending an acousto-optic combined warning signal through the early warning device to remind a driver of paying attention to collision risks.

The embodiment provides a heavy truck blind area early warning method, which comprises the steps of acquiring surrounding target information of a heavy truck through an ultrasonic radar and a millimeter wave radar arranged on the heavy truck, acquiring image data of the early warning target through a camera after the early warning target is determined based on the surrounding target information, acquiring characteristic value combinations of the early warning target according to the surrounding target information and the image data of the early warning target, carrying out cross mutation processing on the characteristic value combinations through a genetic characteristic data fusion algorithm to obtain target characteristic value combinations with optimal fitness, inputting the target characteristic value combinations into a classification model, and acquiring the types of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, the weight values of all the neural networks in the multi-BP neural network model are obtained according to a Monte Carlo method, and determining an early warning mode and carrying out early warning according to the types of the early warning target.

The following describes in detail a method of controlling the distribution of tire forces according to the present application, using a specific example.

Example two

Fig. 3 is a schematic flow chart II of a dead zone early warning method for a heavy truck according to an embodiment of the present application. Fig. 4 is a layout diagram of a blind area early warning sensor provided by an embodiment of the application. The method is described in detail with reference to fig. 3 and 4, and includes:

s301, acquiring surrounding target information of a heavy truck through an ultrasonic radar and a millimeter wave radar;

Specifically, a millimeter wave radar continuously detects a long-distance target around the heavy-duty car, an ultrasonic radar continuously detects a short-distance target around the heavy-duty car, and the information of the target around the heavy-duty car is obtained through the ultrasonic radar and the millimeter wave radar;

According to the detection information of the ultrasonic radar, determining an early warning target in a close range target around the heavy truck and second information of the early warning target, wherein the second information comprises the position of the early warning target; obtaining surrounding target information according to the first information and/or the second information;

Specifically, as shown in fig. 4, a millimeter wave radar is arranged near the front center of the vehicle, which has good tracking and identifying ability to the forward or transverse running target, and ultrasonic radars are respectively arranged at the left and right corners of the front of the vehicle to compensate the detection blind area left by the millimeter wave radar, when the target is in the millimeter wave radar detection blind area, i.e. the position of the target is very close to the heavy vehicle, the millimeter wave radar is limited by the self ability and can not detect the target well; the ultrasonic radar has better detection performance in a close range, can detect a close target, and aims at the detection range of a dead zone at the rear of the vehicle, which is the same as the arrangement scheme in front of the vehicle, so that the dead zone caused by using only a single type of sensor is effectively covered by the complementary advantages of the ultrasonic radar and the millimeter wave radar;

The millimeter wave radar is also arranged at the side front and the side rear of the vehicle to monitor the target conditions of the side front and the side rear areas of the vehicle respectively, and is particularly important for the monitoring of the side front and the side rear of the vehicle especially when the vehicle turns; meanwhile, in order to ensure target detection in a close range and complement blind areas monitored by the sensor, ultrasonic radars are respectively arranged at the front side and the rear side of the vehicle, and are respectively arranged at two sides of the millimeter wave radar, so that the range of the sensor covering the detection blind areas is greatly improved;

With reference to fig. 4, the whole coverage of the blind area is considered, and meanwhile, the redundancy of sensor arrangement is properly increased, so that the stability and the robustness of the blind area early warning function are ensured; meanwhile, considering the cost of the sensor, ultrasonic radars with relatively low price are mostly used to supplement the shortages of millimeter wave radar detection.

S302, after an early warning target is determined according to surrounding target information, acquiring image data of the early warning target through a camera;

Specifically, after the early warning target is determined according to surrounding target information detected by the millimeter wave radar or the ultrasonic radar, the image processing unit maps the area where the early warning target is located into an image coordinate system according to early warning target information such as the speed, the position and the azimuth angle of the target fed back by the millimeter wave radar or the ultrasonic radar, so that target recognition is mainly carried out on a certain area of an image, and image data such as image gray scale, contour, pixel points and the like of the early warning target are extracted;

As shown in fig. 4, a wide-angle camera is arranged in the front center of the vehicle, so that an optimal view can be ensured, the perceived image quality of the camera can be improved, and because the vehicle length of the heavy vehicle is longer, cameras are respectively arranged at the left side and the right side of the vehicle to ensure that the dead zones at the left side and the right side of the vehicle are completely covered, surrounding target information is perceived by comprehensively utilizing sensors such as an ultrasonic radar, a millimeter wave radar, the camera and the like, and meanwhile, more accurate target recognition results are obtained through the complementary advantages among different kinds of perception information; and a stable blind area monitoring range.

S303, acquiring a characteristic value combination of the early warning target according to surrounding target information and image data of the early warning target;

Specifically, after receiving target information transmitted by a millimeter wave radar or an ultrasonic radar, an image recognition module is started to recognize characteristic values of image data such as image gray scale, outline, pixel points and the like of an early warning target, and the characteristic values such as target distance, target speed and target energy reflection value transmitted by the millimeter wave radar or the ultrasonic radar form a characteristic value combination of the early warning target;

When the camera detects that the current running environment is at night, the camera enters a sleep mode, the image processing module is closed, and a characteristic value combination of an early warning target is generated according to surrounding target information of the heavy vehicle, which is acquired by the ultrasonic radar and the millimeter wave radar; wherein, the number of the ultrasonic radars and the millimeter wave radars corresponding to night is more than that of the ultrasonic radars and the millimeter wave radars corresponding to daytime; because the image data transmitted by the camera cannot be used during night driving, at the moment, the blind area early warning controller can open target information fed back by more millimeter wave radars, and simultaneously fuse target characteristic information detected by the millimeter wave radars and the ultrasonic radars, and the early warning target can be tracked in advance by adjusting configuration parameters of the sensor, so that the monitoring area for triggering the early warning signal is properly enlarged;

After an early warning target is detected, the image processing unit performs the next operations of image data feature recognition, target recognition and the like, and in this way, the calculated amount and the occupation of the storage space can be greatly reduced, so that the consumption of software resources is reduced, and the data transmitted by the millimeter wave radar or the ultrasonic radar can be more rapidly and effectively monitored;

If only the ultrasonic radar is used for detecting the early warning target, the image processing unit is used for processing the image data transmitted by the camera closest to the ultrasonic radar for determining the early warning target, identifying the image data in the image, and combining the image data with the early warning target information fed back by the ultrasonic radar to form a characteristic value combination.

S304, performing cross mutation treatment on the characteristic value combination according to a genetic characteristic data fusion algorithm to obtain a target characteristic value combination with optimal fitness;

Specifically, the target characteristic value combination is expressed as a binary string, and M random individuals are randomly generated as an initial group based on the binary string;

The probability of each individual being selected is calculated from fitness S _i (i=1, 2, … … M), and the probability of the target selected individual P _i,max being selected is obtained by the following formula:

Wherein S _i,max refers to the fitness of the current target individual;

according to variation operation in a genetic algorithm, randomly performing inverse operation on characteristic data of individuals in an initial population;

based on the individuals after the crossover operation and the mutation operation, the fitness S _i and the probability P _i,max that the target selected individual is selected are circularly recalculated until the probability P _i,max that the target selected individual is selected is no longer changed, and the characteristics of the individual with the greatest fitness are used as the target characteristic value combination.

S305, combining the target characteristic values and inputting the target characteristic values into a classification model, and obtaining the type of the early warning target output by the classification model;

wherein obtaining the classification model comprises:

acquiring a data set D containing m sample feature combinations, wherein m is an integer greater than 1, and the sample feature combinations are feature combinations obtained through a genetic feature data fusion algorithm and sample tags corresponding to the feature combinations;

The ith bagging (i=1, 2, … …, q), extracting a data set D _i with a sample capacity of n from the data set D in a put-back way according to a uniform probability distribution, wherein each data set D _i is used for deriving a BP neural network to obtain q BP neural networks, n and q are integers, and n is smaller than m;

obtaining a classification model according to q BP neural networks and the weight value of each BP neural network;

The weight value of each BP neural network is realized according to a Monte Carlo fusion algorithm by the following formula:

The method is converted into the following formula to solve:

The Monte Carlo method can be used for randomly generating a preset group of weight values, the recognition errors delta _total are calculated in sequence, and a group of corresponding weight values with the minimum delta _total are selected to be used as the weight values of the multi-BP neural network;

Specifically, after the target characteristic value combination is obtained, the target characteristic value combination is classified through the multi-BP neural network, the most conforming early warning target type is determined, the optimal weight value combination is obtained for the weight value of each neural network in the multi-BP neural network through a Monte Carlo simulation method, the obtained optimal weight value combination corresponds to the set target type respectively, when the target characteristic value combination is input, the result obtained by multiplying the weight value of each neural network falls into a result interval conforming to the target type, the type of the early warning target is determined, the recognition error of the target can be reduced based on a classification model formed by the multi-BP neural network, and the stability and the accuracy of early warning target recognition are improved.

S306, determining an early warning mode according to the type of the early warning target and carrying out early warning;

The early warning risk level is determined according to the type of the early warning target, the speed of the heavy truck and the vehicle corner;

Determining an early warning mode according to the early warning risk level and carrying out early warning; the early warning risk level comprises a low risk early warning level and a high risk early warning level, the early warning mode corresponding to the low risk early warning level is a flashing prompt through an alarm lamp, and the early warning mode corresponding to the high risk early warning level is a prompt through an acousto-optic combination mode;

the yaw sensor can detect a weak corner state of the vehicle, acquire the corner angle of the vehicle at the moment, expand the detection range when the vehicle turns and increase the sensitivity for risk judgment by arranging a yaw (Yawrate) sensor in the millimeter wave radar;

Specifically, according to the information of the distance, speed and the like of the target objects detected by the radar, and by combining signals of the vehicle body speed, the rotation angle and the like, determining risk alarming of the early warning targets, after determining the types of the early warning targets, determining early warning risk levels according to the types of the finally identified target objects, changing the early warning risk levels according to the different types of the early warning targets, giving the highest priority and weight ratio to the targets which are extremely easy to be dangerous to human bodies, such as pedestrians, bicycles, electric vehicles and the like, and giving the lower priority and weight ratio to the early warning targets to the stationary targets such as guardrails, walls, stones and the like;

the early warning device comprises an alarm lamp and a buzzer, the early warning device is arranged on left and right side posts in a cab, and an audible and visual alarm signal is sent out based on the early warning signal sent out by the recognized early warning target type and the state information of the heavy vehicle so as to remind a driver of which side has blind area collision risk.

According to the heavy-duty car blind area early warning method provided by the application, the heavy-duty car is obtained through aiming at the ultrasonic radar, the millimeter wave radar and the camera which are reasonably arranged on the heavy-duty car, so that all sensors form advantage complementation, the full coverage of the heavy-duty car blind area is realized, after the early warning target is determined based on surrounding target information fed back by the ultrasonic radar and the millimeter wave radar, the image data of the early warning target is obtained through the camera, the characteristic value combination of the early warning target is obtained according to the surrounding target information and the image data of the early warning target, the characteristic value combination is subjected to cross mutation processing through the genetic characteristic data fusion algorithm, the target characteristic value combination with optimal fitness is obtained, the target characteristic value combination is input into the classification model, the type of the early warning target output by the classification model is obtained, the weight value of each neural network in the multi-BP neural network model is obtained according to the Monte Carlo simulation method, and the early warning mode is determined according to the type of the early warning target, and the early warning stability and accuracy of early warning target recognition are improved, and therefore the blind area early warning method provided by the application can also recognize obstacles and prompt drivers in time.

The embodiment of the invention can divide the functional modules of the electronic device or the main control device according to the method example, for example, each functional module can be divided corresponding to each function, and two or more functions can be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present invention, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

Fig. 5 is a schematic structural diagram of a dead zone early warning device for a heavy truck according to an embodiment of the present application.

As shown in fig. 5, the apparatus 500 includes:

The acquisition module 501 is configured to acquire surrounding target information of the heavy vehicle through the ultrasonic radar and the millimeter wave radar, and acquire image data of an early warning target through the camera after determining the early warning target according to the surrounding target information;

the first processing module 502 is configured to obtain a feature value combination of the early warning target according to the surrounding target information and the image data of the early warning target;

the second processing module 503 is configured to perform cross mutation processing on the feature value combination according to a genetic feature data fusion algorithm, so as to obtain a target feature value combination with optimal fitness;

The classification module 504 is configured to input the target feature value combination into a classification model, and obtain a type of the early warning target output by the classification model, where the classification model is a multi-back propagation BP neural network model, and weight values of each neural network in the multi-BP neural network model are obtained according to a monte carlo simulation method;

and the execution module 505 is used for determining an early warning mode and carrying out early warning according to the type of the early warning target.

Further, the obtaining module 501 is specifically configured to: the millimeter wave radar continuously detects long-distance targets around the heavy-duty car, the ultrasonic radar continuously detects short-distance targets around the heavy-duty car, the information of the targets around the heavy-duty car is obtained through the ultrasonic radar and the millimeter wave radar, and the method comprises the following steps:

Further, the second processing module 503 is specifically configured to: the cross mutation processing is carried out on the characteristic value combination according to the genetic characteristic data fusion algorithm to obtain a target characteristic value combination with optimal fitness, and the method comprises the following steps:

Wherein S _i,max refers to the fitness of the current target individual;

Further, the classification module 504 is specifically configured to: before the target characteristic values are combined and input into a classification model and the type of the early warning target output by the classification model is acquired, the method further comprises the steps of:

Further, the classification module 504 is specifically configured to: the weight value of each BP neural network is realized according to a Monte Carlo fusion algorithm by the following formula:

The method is converted into the following formula to solve:

Further, the execution module 505 is further configured to: according to the type of the early warning target, determining an early warning mode and carrying out early warning, including:

Further, the first processing module 502 is specifically configured to: when the camera detects that the current running environment is at night, the camera enters a sleep mode;

The dead zone early warning device for the heavy truck provided by the embodiment can execute the dead zone early warning method for the heavy truck of the above embodiment, and the implementation principle and the technical effect are similar, and the embodiment is not repeated here.

In the specific implementation of the heavy-duty car dead zone early warning method, each module may be implemented as a processor, and the processor may execute computer execution instructions stored in the memory, so that the processor executes the heavy-duty car dead zone early warning method.

Fig. 6 is a schematic structural diagram of a heavy-duty car blind zone early warning device according to an embodiment of the present application. As shown in fig. 6, the apparatus 600 includes: at least one processor 601 and a memory 602. The device 600 further comprises a communication component 603. The processor 601, the memory 602, and the communication section 603 are connected via a bus 604.

In a specific implementation process, at least one processor 601 executes the computer-executed instructions stored in the memory 602, so that the at least one processor 601 executes the heavy truck blind area early warning method executed on the equipment side as above.

The specific implementation process of the processor 601 may refer to the above-mentioned method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.

In the above embodiment, it should be understood that the Processor may be a central processing unit (english: central Processing Unit, abbreviated as CPU), or may be other general purpose processors, digital signal processors (english: DIGITAL SIGNAL Processor, abbreviated as DSP), application specific integrated circuits (english: application SPECIFIC INTEGRATED Circuit, abbreviated as ASIC), or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.

The memory may comprise high speed RAM memory or may further comprise non-volatile storage NVM, such as at least one disk memory.

The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (PERIPHERAL COMPONENT, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the buses in the drawings of the present application are not limited to only one bus or to one type of bus.

The scheme provided by the embodiment of the invention is introduced aiming at the functions realized by the electronic equipment and the main control equipment. It will be appreciated that the electronic device or the master device, in order to implement the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. The present embodiments can be implemented in hardware or a combination of hardware and computer software in combination with the various exemplary elements and algorithm steps described in connection with the embodiments disclosed in the embodiments of the present invention. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not to be considered as beyond the scope of the embodiments of the present invention.

The application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions, the dead zone early warning method of the heavy truck is realized.

The computer readable storage medium described above may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. A readable storage medium can be any available medium that can be accessed by a general purpose or special purpose computer.

An exemplary readable storage medium is coupled to the processor such the processor can read information from, and write information to, the readable storage medium. In the alternative, the readable storage medium may be integral to the processor. The processor and the readable storage medium may reside in an Application SPECIFIC INTEGRATED Circuits (ASIC). The processor and the readable storage medium may reside as discrete components in an electronic device or a master device.

The present application also provides a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any one of the embodiments described above.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims

1. The utility model provides a heavy car blind area early warning method which characterized in that, heavy car is provided with ultrasonic radar, millimeter wave radar and camera, the method includes:

Inputting the target characteristic value combination into a classification model, and obtaining the type of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, and the weight value of each neural network in the multi-back propagation BP neural network model is obtained according to a Monte Carlo simulation method;

2. The method according to claim 1, wherein the millimeter wave radar continuously detects a long-distance object around the heavy-duty car, the ultrasonic radar continuously detects a short-distance object around the heavy-duty car, the acquiring the information of the object around the heavy-duty car by the ultrasonic radar and the millimeter wave radar includes:

3. The method according to claim 1, wherein the cross-mutation processing is performed on the feature value combination according to a genetic feature data fusion algorithm to obtain a target feature value combination with optimal fitness, including:

Wherein S _i,max refers to the fitness of the current target individual;

4. The method of claim 1, wherein the combining the target feature values into a classification model, prior to obtaining the type of the pre-warning target output by the classification model, further comprises:

5. The method of claim 4, wherein the weight value of each BP neural network is implemented according to a monte carlo fusion algorithm by:

The method is converted into the following formula to solve:

6. The method according to any one of claims 1-5, wherein determining an early warning mode and performing early warning according to the type of the early warning target comprises:

7. The method according to claim 1, wherein the method further comprises:

when the camera detects that the current running environment is at night, the camera enters a sleep mode;

8. The utility model provides a heavy car blind area early warning device which characterized in that includes:

the acquisition module is used for acquiring surrounding target information of the heavy truck through an ultrasonic radar and a millimeter wave radar, and acquiring image data of an early warning target through a camera after determining the early warning target according to the surrounding target information;

the classification module is used for inputting the target characteristic value combination into a classification model, and obtaining the type of the early warning target output by the classification model, wherein the classification model is a multi-back propagation BP neural network model, and the weight value of each neural network in the multi-back propagation BP neural network model is obtained according to a Monte Carlo simulation method;

9. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 7.

10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 7.