CN111797657A - Vehicle surrounding obstacle detection method, device, storage medium and electronic device - Google Patents

Abstract

The embodiments of the present application disclose a method, a device, a storage medium, and an electronic device for detecting obstacles around a vehicle. Among them, the embodiment of the present application acquires regional images of at least two regions around the vehicle; extracts the region of interest in the region image; determines the obstacle image block in the region image by detecting the region of interest; The obstacle image block is corrected to output the correct obstacle location. In this embodiment of the present application, the location of the obstacle is determined and corrected by detecting the region of interest, and by acquiring multiple area images, the location of the obstacle around the vehicle can be effectively focused, and the detection speed and accuracy of the obstacle can be accelerated, which is beneficial to the subsequent control of the vehicle and ensures driving. Safety.

Description

Vehicle surrounding obstacle detection method, device, storage medium and electronic device

technical field

The present application relates to the field of safe driving, and in particular, to a method, device, storage medium and electronic device for detecting obstacles around a vehicle.

Background technique

In the field of safe driving, obstacle detection is very important. The detection of obstacles around the vehicle is an important measure to improve driving safety and improve the traffic environment.

In the existing systems and methods for obstacle detection, especially pedestrian detection, most of the methods and devices use deep learning to detect images. This solution relies on a powerful GPU onboard, which requires a large amount of computing resources, which leads to vehicle-mounted terminals. The cost is high and the detection efficiency is low.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, device, storage medium, and electronic device for detecting obstacles around a vehicle, which can reduce the cost of detecting obstacles in a vehicle terminal, speed up detection, and improve detection accuracy.

An embodiment of the present application provides a method for detecting obstacles around a vehicle, wherein the method for detecting obstacles around a vehicle includes:

Obtain area images of at least two areas around the vehicle;

extracting a region of interest in the region image;

By detecting the region of interest, an obstacle image block is determined in the region image;

The obstacle image blocks are corrected to output the correct obstacle location.

Embodiments of the present application also provide a device for detecting obstacles around a vehicle, including:

an acquisition module for acquiring regional images of at least two areas around the vehicle;

an extraction module for extracting the region of interest in the region image;

a detection module, configured to determine an obstacle image block in the region image by detecting the region of interest;

The correction module is used for correcting the obstacle image block to output the correct obstacle position.

The embodiment of the present application also provides a storage medium, wherein a computer program is stored in the storage medium, and when the computer program runs on the computer, the computer is caused to perform the following steps:

Obtain area images of at least two areas around the vehicle;

extracting a region of interest in the region image;

By detecting the region of interest, an obstacle image block is determined in the region image;

The obstacle image blocks are corrected to output the correct obstacle location.

The embodiment of the present application also provides an electronic device, wherein the electronic device includes a processor and a memory, and a computer program is stored in the memory, and the processor is used to perform the following steps by calling the computer program stored in the memory:

Obtain area images of at least two areas around the vehicle;

extracting a region of interest in the region image;

By detecting the region of interest, an obstacle image block is determined in the region image;

The obstacle image blocks are corrected to output the correct obstacle location.

In this embodiment of the present application, the location of the obstacle is determined and corrected by detecting the region of interest, and by acquiring multiple area images, the location of the obstacle around the vehicle can be effectively focused, and the detection speed and accuracy of the obstacle can be accelerated, which is beneficial to the subsequent control of the vehicle and ensures driving. Safety.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a schematic diagram of an application scenario of a method for detecting obstacles around a vehicle provided by an embodiment of the present application.

FIG. 2 is a first schematic flowchart of a method for detecting obstacles around a vehicle provided by an embodiment of the present application.

FIG. 3 is a schematic diagram of another application scenario of the method for detecting obstacles around a vehicle provided by an embodiment of the present application.

FIG. 4 is a second schematic flowchart of the method for detecting obstacles around a vehicle provided by an embodiment of the present application.

FIG. 5 is a schematic structural diagram of a first structure of a device for detecting obstacles around a vehicle provided by an embodiment of the present application.

FIG. 6 is a schematic diagram of a second structure of the device for detecting obstacles around a vehicle according to an embodiment of the present application.

FIG. 7 is a schematic diagram of a third structure of the device for detecting obstacles around a vehicle according to an embodiment of the present application.

FIG. 8 is a schematic diagram of a fourth structure of the device for detecting obstacles around a vehicle according to an embodiment of the present application.

FIG. 9 is a schematic diagram of a fifth structure of the device for detecting obstacles around a vehicle provided by an embodiment of the present application.

FIG. 10 is a schematic diagram of a first structure of an electronic device provided by an embodiment of the present application.

FIG. 11 is a schematic diagram of a second structure of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of this application.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of an application scenario of a method for detecting obstacles around a vehicle provided by an embodiment of the present application. The obstacle detection method around the vehicle can be applied to electronic equipment. A panoramic perception architecture is provided in the electronic device. Panoramic perception architecture is the integration of hardware and software in electronic devices for implementing obstacle detection methods around vehicles.

Among them, the panoramic perception architecture includes an information perception layer, a data processing layer, a feature extraction layer, a scenario modeling layer, and an intelligent service layer.

The information perception layer is used to acquire the information of the electronic device itself and/or the information in the external environment. The information perception layer may include multiple sensors. For example, the information perception layer includes a distance sensor, a magnetic field sensor, a light sensor, an acceleration sensor, a fingerprint sensor, a Hall sensor, a position sensor, a gyroscope, an inertial sensor, an attitude sensor, a barometer, a heart rate sensor, and other sensors.

Among them, the distance sensor can be used to detect the distance between the electronic device and the external object. The magnetic field sensor can be used to detect the magnetic field information of the environment in which the electronic device is located. The light sensor can be used to detect the light information of the environment where the electronic device is located. Acceleration sensors can be used to detect acceleration data of electronic devices. The fingerprint sensor can be used to collect the user's fingerprint information. Hall sensor is a magnetic field sensor made according to the Hall effect, which can be used to realize automatic control of electronic equipment. The location sensor can be used to detect the current geographic location of the electronic device. Gyroscopes can be used to detect the angular velocity of electronic devices in various directions. Inertial sensors can be used to detect motion data of electronic devices. The attitude sensor can be used to sense the attitude information of the electronic device. A barometer can be used to detect the air pressure in the environment in which the electronic device is located. The heart rate sensor may be used to detect the user's heart rate information.

The data processing layer is used to process the data obtained by the information perception layer. For example, the data processing layer can perform data cleaning, data integration, data transformation, data reduction and other processing on the data obtained by the information perception layer.

Among them, data cleaning refers to cleaning a large amount of data obtained by the information perception layer to eliminate invalid data and duplicate data. Data integration refers to integrating multiple single-dimensional data obtained by the information perception layer into a higher or more abstract dimension to comprehensively process multiple single-dimensional data. Data transformation refers to converting the data type or format of the data obtained by the information perception layer, so that the transformed data can meet the processing requirements. Data reduction refers to reducing the amount of data to the greatest extent possible on the premise of keeping the original data as much as possible.

The feature extraction layer is used to perform feature extraction on the data processed by the data processing layer to extract features included in the data. The extracted features may reflect the state of the electronic device itself, the state of the user, or the environmental state of the environment in which the electronic device is located.

Among them, the feature extraction layer can extract features or process the extracted features by filtering method, packaging method, integration method and other methods.

The filtering method refers to filtering the extracted features to remove redundant feature data. The packing method is used to filter the extracted features. The integration method refers to the integration of multiple feature extraction methods to construct a more efficient and accurate feature extraction method for feature extraction.

The scenario modeling layer is used to construct a model according to the features extracted by the feature extraction layer, and the obtained model can be used to represent the state of the electronic device, the state of the user, or the environment state, etc. For example, the scenario modeling layer can construct a key value model, a pattern identification model, a graph model, an entity relationship model, an object-oriented model, etc. according to the features extracted by the feature extraction layer.

The intelligent service layer is used to provide users with intelligent services according to the model constructed by the scenario modeling layer. For example, the intelligent service layer can provide users with basic application services, can perform system intelligent optimization for electronic devices, and can also provide users with personalized intelligent services.

In addition, the panoramic perception architecture can also include multiple algorithms, each of which can be used to analyze and process data, and multiple algorithms can form an algorithm library. For example, the algorithm library may include Markov algorithm, latent Dirichlet distribution algorithm, Bayesian classification algorithm, support vector machine, K-means clustering algorithm, K-nearest neighbor algorithm, conditional random field, residual network, long Algorithms such as short-term memory networks, convolutional neural networks, and recurrent neural networks.

An embodiment of the present application provides a method for detecting obstacles around a vehicle, and the method for detecting obstacles around a vehicle can be applied to electronic devices. The electronic device can be a smartphone, a tablet computer, a gaming device, an AR (Augmented Reality) device, a car, a vehicle surrounding obstacle detection device, an audio playback device, a video playback device, a notebook, a desktop computing device, a wearable device such as a watch , glasses, helmets, electronic bracelets, electronic necklaces, electronic clothing and other equipment.

Referring to FIG. 2 , FIG. 2 is a first schematic flowchart of a method for detecting obstacles around a vehicle provided by an embodiment of the present application. The method for detecting obstacles around the vehicle includes the following steps:

110. Obtain regional images of at least two regions around the vehicle.

Acquiring area images of at least two areas around the vehicle includes: acquiring area images of at least two areas around the vehicle under different focal lengths. The focal length refers to the variation range of the focal length of the zoom lens, and different focal lengths can include wide-angle, telephoto, and ordinary focal lengths.

The periphery of the vehicle may include the front, rear, and side of the vehicle, as well as distant and distant areas related to the driving of the vehicle. The specific range of the periphery of the vehicle involved in the embodiment of the present application may be set by the user.

Regional images can be obtained through lenses, cameras or camera modules of different focal lengths. For example, use the wide-angle lens to obtain the area image on the side of the vehicle to detect pedestrians at the edge of the vehicle; use the clear segment of the wide-angle lens or ordinary camera to obtain the image of the area in front of or behind the vehicle to detect pedestrians in front of the vehicle; use the telephoto lens to obtain Distant area images to detect pedestrians on the far side.

The wide-angle lens is a photographic lens with a focal length shorter than a standard lens, a viewing angle greater than that of the standard lens, a focal length longer than a fisheye lens, and a viewing angle smaller than that of the fisheye lens. Wide-angle lenses are divided into ordinary wide-angle lenses and ultra-wide-angle lenses. A telephoto lens is a photographic lens with a longer focal length than a standard lens. Telephoto lenses are divided into two categories: ordinary telephoto lenses and super telephoto lenses. A normal telephoto lens has a focal length close to that of a standard lens, while a super-telephoto lens has a focal length much larger than that of a standard lens. Different lenses are responsible for image acquisition in different areas. Through multiple lenses with different focal lengths, all areas around the car can be included as much as possible to obtain a complete area image of all areas around the car.

In some embodiments, the front end of the vehicle is provided with a mobile terminal access device, and the mobile terminal can be fixed in a certain position, for example, the mobile terminal is inserted into the mobile terminal access device and fixed in front of the windshield of the front end of the car. In addition, the mobile terminal device is connected to the vehicle control system as an intermediate bridge connecting the mobile terminal and the vehicle control system. For example, the mobile terminal device can communicate with the central control system of the car through the built-in vehicle control communication protocol, so that the user can obtain data such as the current actual speed of the car. Specifically, the vehicle control communication protocol may be provided by the manufacturer, which is not limited here.

In some embodiments, regional images of at least two regions around the vehicle are acquired through lenses at different fixed positions, so as to locate and track obstacles. Obstacles include living bodies and non-living bodies, where living bodies include pedestrians, animals, etc., and non-living bodies include trees and railings around vehicles. Specifically, it can include relevant data applied to the intelligent service layer in the panoramic perception architecture. Specifically, the area image on the side of the vehicle can be obtained through the wide-angle camera, and pedestrians can be detected at the edge of the vehicle; the area image in front of or behind the vehicle can be obtained through the clear segment of the wide-angle camera or the ordinary camera, and the pedestrian in front of the vehicle can be detected; through the telephoto camera model The group obtains images of distant areas and detects pedestrians at the far end.

In some embodiments, regional images of at least two regions around the vehicle are acquired through lenses at different fixed positions, and the same obstacles contained in the at least two regions are located and tracked, which can effectively focus on the positions in the images. For example, the image of the area in front of the vehicle is obtained through the clear segment of the wide-angle camera or the ordinary camera, the image of the area on the right side of the vehicle is obtained through the wide-angle camera, and the same pedestrian in the right front of the vehicle contained in the two area images is subsequently located. Effectively focus locations in pedestrian images to speed up pedestrian detection and detection accuracy.

120. Extract the region of interest in the region image.

After acquiring the region images of at least two regions around the vehicle, regions of interest in the region images of these regions are extracted, which are also called ROI regions (region of interest, region of interest). The basic processing methods of the region of interest include: outline the region to be processed in various shapes from the processed image, such as box, circle, ellipse, irregular polygon, etc., take the region as the focus of image analysis, and delineate the region. for further processing. For example, in some embodiments, a rectangle containing a target object (such as an obstacle) is delineated in the area image as an analysis focus, and the rectangular area is further reduced and optimized by an algorithm to find the position containing the target object, for example, in some machines Various operators and functions are commonly used in vision software to obtain the ROI of the region of interest, such as Halcon, OpenCV, Matlab, etc. Using ROI to delineate the target can reduce processing time and increase accuracy. Multiple region images correspond to multiple ROI regions.

In some embodiments, after the region of interest in the region image is extracted, the step of continuously optimizing the region of interest may include: inputting the position of the obstacle as a feedback signal into the learning algorithm model to adjust the area of the region of interest.

For example, at the beginning of the algorithm, the whole image is used as the ROI area. After a certain period of learning, the ROI effective areas (smaller and more accurate ROI areas) of different shots are obtained after optimization as the input of the subsequent steps. In other words, at the beginning of the algorithm, the ROI areas of different focal segments are all full images. After a period of learning, the ROI areas of different focal segments are gradually distinguished. Starting from the full image range, the detection of obstacles can be performed to the greatest extent possible. to avoid omissions.

The location information of obstacles in different focal lengths is different. Taking the obstacle as a pedestrian as an example, the angle of the ordinary color camera is relatively normal, and the area where the pedestrian appears can actually only appear on the road (actually mapped to the lower half of the image). ) rather than appearing in the sky. In some embodiments, after the region image is obtained, the evaluation is performed based on the position information of the obstacle feature in the obtained region image, and some images in which the obstacle is unlikely to appear and the obstacle is not in the dangerous area are not included in the detection, for example, The area of interest is obtained by removing the sky and ground scenes of the upper and lower pixels of the area image, and the scenes on both sides of the road of the left and right pixels of the image.

ROI area detection methods include motion detection methods, shape-based detection methods, and regional thresholding methods. For example, when the obstacle is a living body, the motion detection method can segment the region of interest by detecting the motion information in the regional image, combined with the motion detection of optical flow analysis, that is, based on the coherent motion of small areas of the same color pixels, each pixel point are given a possible probability of belonging to a given small area, and the movement of each small area is classified according to a probabilistic model of motion in preparation for the next stage of biometric identification, through a unique coherent consistent motion region, and then vote for a straight path based on the inter-frame analysis. Only when a regular trajectory is detected in a set of frames, an organism is considered to be detected, and the region of interest is obtained from this, or the region of interest is optimized. . Furthermore, a zero-crossing detection algorithm can be used to obtain the region of interest by using the spatial-temporal Gaussian convolution of historical pixel values in the past several frames, and then processing the occlusion by using an extended second-order Kalman filter.

130. Determine an obstacle image block in the area image by detecting the region of interest.

Obstacle image blocks are image blocks containing obstacles in the regional image. Obstacles include living bodies and non-living bodies. Among them, objects include pedestrians, animals, etc., and non-living bodies include trees, railings, etc. around the vehicle. Specifically, it can include panoramic views. Relevant data applied to the intelligent service layer in the perception architecture. After a certain period of learning, regions of interest of different shots are extracted from step 120 for detection. Detection of regions of interest, including obstacle detection. By detecting obstacles in the region of interest, the obstacle image block is determined in the regional image, which specifically includes: inputting the region of interest in the convolutional neural network model, and judging the probability that the region of interest contains obstacles; when the probability is greater than or equal to the predicted When setting the probability threshold, the part of the region image corresponding to the region of interest is determined as the obstacle image block. For example, through the Faster R-CNN (Towards Real-Time Object Detection with Region Proposal Networks) network model, the specific convolutional neural network can use a lightweight network structure to detect bounding boxes that may be pedestrians. Among them, the bounding box can be understood as a rectangular box containing the target, and the detection of the bounding box refers to determining a position (such as x, y, w, h) for the target area (such as a pedestrian) in the image, corresponding to the upper left corner of the target respectively. The coordinates (x,y), and the length (h) and width (w) of the box. Using a lightweight network structure, the operation rate can be increased on the terminal and the operation accuracy can be maintained. For example, mobile-net/sequzee-net can be used instead of CNN layers.

In some embodiments, in addition to determining the obstacle image block through obstacle detection, other methods can also be used to determine the obstacle image block, assign weights to the obstacle image blocks determined by two or more methods, calculate the expected value, and further improve the detection precision. For example, a clustering regression algorithm is used to detect the regional image, and the expected value is calculated together with the obstacle detection result, which can include:

(1) By detecting the region of interest, before determining the obstacle image block in the regional image, the method further includes: determining the first obstacle image block by performing cluster regression on the regional image.

The step of determining the first obstacle image block by performing cluster regression on the regional image may include: dividing and compressing the regional image to obtain several image blocks; extracting feature values from the image blocks; Whether the obstacle feature is included; when it is determined that the image block contains the obstacle feature, the part corresponding to the image block in the area image is determined as the first obstacle image block.

For example, bounding box (bounding box regression, which can be understood as the smallest rectangle containing the target object) divides pedestrians in the front area of the vehicle, divides 100 possible range areas for pedestrians, and divides the regional images corresponding to different range areas. Extract the image of the detected bounding box position, and then use the PCA algorithm (principal Component Analysis, that is, principal component analysis method, which is the most widely used data compression algorithm) to compress the image to obtain a lower dimensional feature value, Then use a classifier to determine whether it is an obstacle, such as SVM (Support Vector Machines) classifier or random forest classifier.

Among them, the algorithm idea of PCA is mainly: data is converted from the original coordinate system to the new coordinate system. For example, in the embodiment of the present application, the area image data is converted from the original large coordinate system to the new small coordinate through the PCA algorithm system to achieve image compression. When converting the coordinate system, the direction with the largest variance is used as the direction of the coordinate axis, because the largest variance of the data gives the most important information of the data. The first new axis selects the direction with the largest variance in the original data, and the second new axis selects the direction that is orthogonal to the first new axis and has the next largest variance. This process is repeated, and the number of repetitions is the feature dimension of the original data.

In the new coordinate system obtained in this way, most of the variance is contained in the first few coordinate axes, and the variance contained in the latter coordinate axis is almost 0. Therefore, the remaining coordinate axes can be ignored, and only the first few coordinate axes containing most of the variance are retained, and therefore, they can be used for image compression in the embodiments of the present application. In fact, this is equivalent to retaining only the dimension features containing most of the variance, and ignoring the feature dimensions containing almost 0 variance, which also realizes the dimensionality reduction processing of the data features.

A classifier is a classification function or a classification model constructed on the basis of existing data. The function or model can map the data records in the database to one of the given categories, that is, "classification". The classification can be done by a classifier. SVM (support vector machine, support vector machine) is a two-class classification model. Its basic model is defined as a linear classifier with the largest interval on the feature space. Its learning strategy is to maximize the interval, which can be finally transformed into a convex two Solving the secondary programming problem. Random forest refers to a classifier that uses multiple trees to train and predict samples. It is a classifier containing multiple decision trees, and the output categories are determined by the mode of the output categories of individual trees.

Specifically, obstacle features can be preset in the classifier, and when the image block contains these obstacle features, it is considered that the image block contains obstacles, and the part of the area image corresponding to the image block is determined as the first obstacle image block. For each input frame of regional image, first process an image block in the regional image, and process each image block in each frame of regional image cyclically.

For example, since the obstacle includes a living body, the obstacle feature can be set to "existing motion track". When each frame of image is detected in turn, it is found that the image block corresponds to a motion track, and it can be considered that the image block contains obstacle features, that is, contains obstacle.

(2) Determining the obstacle image block in the area image by detecting the region of interest includes: determining the second obstacle image block by performing obstacle detection on the region of interest.

For example, the obstacle image block determined by the aforementioned neural network model is used as the second obstacle image block.

(3) Set different weights for the first obstacle image block and the second obstacle image block, and calculate the expected value of the obstacle image block according to the weight.

Specifically, a preset weight of one time can be set for the estimated bounding box in the first obstacle image block, five times the preset weight can be set for the bounding box estimated in the second obstacle image block, and the expected value of the obstacle image block is calculated according to the weight. . Preset weights can be preset.

(4) Set the threshold to cut off the expected value of the obstacle image block, so as to determine the obstacle image block in the area image.

For example, when the expected value of the bounding box of the obstacle image block reaches a certain preset threshold, the position corresponding to the bounding box in the area image is determined as the obstacle image block.

140. Correct the obstacle image block to output the correct obstacle position.

In actual detection, detection errors may be caused by various factors. Therefore, correcting the detected obstacle image block can effectively reduce the error and output the correct obstacle position.

Specifically, according to the detected obstacle image block, the sensor can be used to detect whether there is an obstacle in the direction and distance corresponding to the surrounding of the vehicle.

It should be noted that obstacles include living organisms. When the obstacle is a living body, the step of correcting the obstacle image block may include: determining a core tracking point in the obstacle image block, and using a filtering method to predict the movement trajectory of the living body; If the error between the currently detected movement trajectories of the organism is within the preset error range, the core tracking point will be used as the correct obstacle position; If the error is outside the preset error range, the core tracking point is used as the wrong obstacle position; the correct obstacle position is displayed, and the wrong obstacle position is deleted.

For example, the center position of the obstacle image block is determined as the core tracking point, the Kalman filter method is used to track the trajectory of the organism, and the detection results of S4.2 and S4.3 are confirmed through consecutive multi-frame images (t frames). Specifically, the trajectory tracking is continued for the organisms detected in the historical frame data, where the gradient series [t-15,t-13,t-11,t-9,…,t-3,t] is used as the history In the detection frame, if the error between the predicted biological trajectory and the currently detected biological trajectory is within the preset error range, that is, the biological trajectory can be effectively modeled in the image of the above frame, Then, it is considered that the current biological detection result is valid, the core tracking point is used as the correct obstacle position, and the correct obstacle position is affine to the coordinate system where the vehicle is located, so as to display the correct obstacle position in the coordinate system. If the error between the predicted trajectory of the organism and the currently detected trajectory of the organism is outside the preset error range, that is, the difference between the current value and the predicted value calculated by Kalman aluminum foil is large, it is considered wrong. Detect, discard and delete the wrong obstacle position.

Through the correction, the detection accuracy can be effectively improved, and the wrong results can be avoided to be displayed to the user as much as possible. For the correct results, affine them into the coordinate system and display them to the user, which is beneficial to the intelligent driving system in the subsequent control of the vehicle. For example, when an organism is detected in front of the car, it will avoid or remind the driver, and predict the walking trajectory and direction of the organism detected in the distance to avoid subsequent dangers. It can effectively assist or participate in the intelligent driving system or the assisted driving system by sending out the situation in front of the car.

Please refer to FIG. 3 , which is a schematic diagram of another application scenario of the method for detecting obstacles around a vehicle provided by an embodiment of the present application. The ROI area has different scenarios. Taking pedestrian detection as an example, the lenses of different focal lengths of the terminal device are respectively connected to the ROI strategy module. The ROI strategy module of each lens is responsible for collecting an image of an area and extracting the area of interest. For example, the wide-angle camera corresponds to The far left in the figure is responsible for detecting the areas of interest on both sides, the middle is the ordinary camera module, which is responsible for detecting the more important areas of interest in the center, and the far right is the telephoto camera module, which is responsible for detecting the distance, that is, the center of the image area of interest. Finally, the three shots extract their respective regions of interest and input them into the corresponding pedestrian detection module. On the one hand, the results detected by the pedestrian detection module are input to the pedestrian behavior recognition module for correction, and on the other hand, they are fed back to the ROI strategy module. The ROI area is optimized by a learning algorithm.

Please continue to refer to FIG. 4 . FIG. 4 is a schematic flowchart of a second method for detecting obstacles around a vehicle provided by an embodiment of the present application. The method for detecting obstacles around the vehicle includes the following steps:

210. Obtain regional images of at least two regions around the vehicle;

Acquiring area images of at least two areas around the vehicle includes acquiring area images of multiple areas around the vehicle. The periphery of the vehicle may include the front, rear, and side of the vehicle, as well as distant and distant areas related to the driving of the vehicle. The specific range of the periphery of the vehicle involved in the embodiment of the present application may be set by the user.

Acquiring area images of at least two areas around the vehicle includes: acquiring area images of at least two areas around the vehicle under different focal lengths. The regional image can be obtained through lenses, cameras or camera modules of different focal lengths, and the different focal lengths can include wide-angle, telephoto, and common focal lengths. For example, the wide-angle camera is used to obtain the area image of the side of the vehicle to detect pedestrians at the edge of the vehicle; the clear segment of the wide-angle camera or the ordinary camera is used to obtain the image of the area in front of or behind the vehicle to detect pedestrians in front of the vehicle; the telephoto camera is used to model The group acquires images of distant regions to detect pedestrians at the far end. Different cameras are responsible for image acquisition in different areas. Through multiple cameras with different focal lengths, all areas around the car can be included as much as possible to obtain complete regional images of all areas around the car.

In some embodiments, the front end of the vehicle is provided with a mobile terminal access device, which can fix the mobile terminal at a certain position, perform high-precision distance calculation, and determine the position of obstacles to control or assist in controlling the vehicle. For example, insert the mobile terminal into the mobile terminal access device and fix it in front of the windshield of the front end of the car. In addition, the mobile terminal device is connected to the vehicle control system as an intermediate bridge connecting the mobile terminal and the vehicle control system. For example, the mobile terminal device can communicate with the central control system of the car through the built-in vehicle control communication protocol, so that the user can obtain data such as the current actual speed of the car. Specifically, the vehicle control communication protocol may be provided by the manufacturer, which is not limited here.

In some embodiments, regional images of at least two regions around the vehicle are acquired through lenses at different fixed positions, so as to locate and track obstacles. Obstacles include living bodies and non-living bodies, where living bodies include pedestrians, animals, etc., and non-living bodies include trees and railings around vehicles. Specifically, it can include relevant data applied to the intelligent service layer in the panoramic perception architecture. Specifically, the area image on the side of the vehicle can be obtained through the wide-angle camera, and pedestrians can be detected at the edge of the vehicle; the area image in front of or behind the vehicle can be obtained through the clear segment of the wide-angle camera or the ordinary camera, and the pedestrian in front of the vehicle can be detected; through the telephoto camera model The group obtains images of distant areas and detects pedestrians at the far end.

In some embodiments, regional images of at least two regions around the vehicle are acquired through lenses at different fixed positions, and the same obstacles contained in the at least two regions are located and tracked, which can effectively focus on the positions in the images. For example, the image of the area in front of the vehicle is obtained through the clear segment of the wide-angle camera or the ordinary camera, the image of the area on the right side of the vehicle is obtained through the wide-angle camera, and the same pedestrian in the right front of the vehicle contained in the two area images is subsequently located. Effectively focus locations in pedestrian images to speed up pedestrian detection and detection accuracy.

220, extract the region of interest in the region image;

After acquiring regional images of at least two regions around the vehicle, regions of interest, also called ROI regions, in the regional images of these regions are extracted. Multiple region images correspond to multiple ROI regions.

The location information of obstacles in different focal lengths is different. Taking the obstacle as a pedestrian as an example, the angle of the ordinary color camera is relatively normal, and the area where the pedestrian appears can actually only appear on the road (actually mapped to the lower half of the image). ) rather than appearing in the sky. In some embodiments, after the region image is obtained, the evaluation is performed based on the position information of the obstacle feature in the obtained region image, and some images in which the obstacle is unlikely to appear and the obstacle is not in the dangerous area are not included in the detection, for example, The area of interest is obtained by removing the sky and ground scenes of the upper and lower pixels of the area image, and the scenes on both sides of the road of the left and right pixels of the image. Regions of Interest (ROI) detection methods include motion detection methods, shape-based detection methods, and region thresholding methods. For example, when the obstacle is a living body, the motion detection method can segment the region of interest by detecting the motion information in the regional image, combined with the motion detection of optical flow analysis, that is, based on the coherent motion of small areas of the same color pixels, each pixel point are given a possible probability of belonging to a given small area, and the movement of each small area is classified according to a probabilistic model of motion in preparation for the next stage of biometric identification, through a unique coherent consistent motion region, and then vote for a straight path based on the inter-frame analysis. Only when a regular trajectory is detected in a set of frames, an organism is considered to be detected, and the region of interest is obtained from this, or the region of interest is optimized. . Furthermore, a zero-crossing detection algorithm can be used to obtain the region of interest by using the spatial-temporal Gaussian convolution of historical pixel values in the past several frames, and then processing the occlusion by using an extended second-order Kalman filter.

Please refer to Figure 3. The ROI area has different scenes. Taking pedestrian detection as an example, the lenses of different focal lengths of the terminal device are respectively connected to the ROI strategy module. The ROI strategy module of each lens is responsible for collecting an image of an area and extracting the region of interest. For example, the wide-angle camera corresponds to the far left in the figure and is responsible for detecting the regions of interest on both sides, the middle is the ordinary camera module, which is responsible for detecting the more important regions of interest in the center, and the far right is the telephoto camera module, which is responsible for detecting Distant is the region of interest in the center of the image. Finally, the three shots extract their respective regions of interest and input them into the corresponding pedestrian detection module. The results detected by the pedestrian detection module are input to the pedestrian behavior recognition module for correction on the one hand, and fed back to the ROI strategy module on the other hand. , the ROI area is optimized by the learning algorithm.

231. Divide and compress the regional image to obtain several image blocks;

For example, bounding box (bounding box regression, which can be understood as the smallest rectangle containing the target object) divides pedestrians in the front area of the vehicle, divides 100 possible range areas for pedestrians, and divides the regional images corresponding to different range areas. Several obtained image blocks correspond to several areas around the vehicle respectively.

232, extract feature values from the image block;

Extract the image of the detected bounding box position, and then use the PCA algorithm to compress the image to obtain lower-dimensional eigenvalues. For example, the matrix in the bounding box box is [50,50], and the box after PCA compression The matrix of is [20, 20], here is not just reducing the size of the matrix, but mathematically calculating it, the co-square matrix extracts common features, leaving the most important eigenvalues. That is, the redundancy of data is reduced and the display degree of features is increased.

233. Determine whether the image block contains obstacle features according to the feature value;

234. When it is determined that the image block contains the obstacle feature, determine the part of the area image corresponding to the image block as the first obstacle image block.

Then use effective classifiers such as SVM classifier or random deep forest to determine whether it is an obstacle. It can be judged according to the preset obstacle characteristics in the classifier. When these obstacle features are included in the image block, it is considered that the image block contains obstacles, and the part corresponding to the image block in the area image is determined as the first obstacle image block. For each input frame of regional image, first process an image block in the regional image, and process each image block in each frame of regional image cyclically.

For example, since the obstacle includes a living body, the obstacle feature can be set to "existing motion track". After each frame of image is detected in turn, it is found that the image block corresponds to a motion track, and it can be considered that the image block contains obstacle features, that is, contains obstacle.

235. Continuously optimize the region of interest, and determine a second obstacle image block by performing obstacle detection on the region of interest;

In some embodiments, after the region of interest in the region image is extracted, the step of continuously optimizing the region of interest may include: inputting the position of the obstacle as a feedback signal into the learning algorithm model to adjust the area of the region of interest. The optimization of the ROI area, on the one hand, is to reduce the area, and on the other hand, to optimize the area itself.

For example, at the beginning of the algorithm, the whole image is used as the ROI area. After a certain period of learning, the ROI effective areas (smaller and more accurate ROI areas) of different shots are obtained after optimization as the input of the subsequent steps. In other words, at the beginning of the algorithm, the ROI areas of different focal segments are all full images. After a period of learning, the ROI areas of different focal segments are gradually distinguished. Starting from the full image range, the detection of obstacles can be performed to the greatest extent possible. to avoid omissions.

By detecting obstacles in the area of interest, the obstacle image block is determined in the area image, including: inputting the area of interest in the convolutional neural network model, and judging the probability that the area of interest contains obstacles; when the probability is greater than or equal to the preset When the probability threshold is set, the part of the region image corresponding to the region of interest is determined as the second obstacle image block.

240. Set different weights for the first obstacle image block and the second obstacle image block, and calculate the expected value of the obstacle image block according to the weight;

Specifically, a preset weight of one time can be set for the estimated bounding box in the first obstacle image block, five times the preset weight can be set for the bounding box estimated in the second obstacle image block, and the expected value of the obstacle image block is calculated according to the weight. . Preset weights can be preset.

The weights of the first obstacle image block and the second obstacle image block may be the same or different. Specifically, it depends on the method for determining the first obstacle image block and the second obstacle image block and the needs of the user. For example, the weight of the first obstacle image block and the second obstacle image block may also be Five times the preset weight is set for the estimated bounding box in the obstacle image block, one preset weight is set for the estimated bounding box in the second obstacle image block, and so on.

It can be seen that steps 231-234 and step 235 are respectively two methods for obtaining obstacle image blocks, namely cluster regression and obstacle detection. In some embodiments, only one of these methods may be used; in some embodiments, any method may be used. One or two other methods are used to replace these two methods; in some embodiments, a new obstacle image block determination method may be added, and multiple methods are used to detect obstacle image blocks respectively, and set weights on the detected multiple obstacle image blocks , calculate the expected value. Two or more of them are used to calculate the expected value, which can increase the detection accuracy of the obstacle image block to a certain extent.

250. Set a threshold to truncate the expected value of the obstacle image block, so as to determine the obstacle image block in the area image.

When the expected value of the obstacle image block reaches a certain preset threshold, the position in the area image corresponding to the expected value of the obstacle image block is determined as the obstacle image block. Specifically, when the expected value of the bounding box of the obstacle image block reaches a certain preset threshold, the position corresponding to the bounding box in the area image is determined as the obstacle image block.

260. Use the filtering method to predict the movement trajectory of the organism, and determine whether the core tracking point is the correct obstacle position according to the error size of the prediction result;

In actual detection, detection errors may be caused by various factors. Therefore, correcting the detected obstacle image block can effectively reduce the error and output the correct obstacle position.

Specifically, according to the detected obstacle image block, the sensor can be used to detect whether there is an obstacle in the direction and distance corresponding to the surrounding of the vehicle.

It should be noted that obstacles include living organisms. When the obstacle is a living body, the step of correcting the obstacle image block may include: determining a core tracking point in the obstacle image block, and using a filtering method to predict the movement trajectory of the living body; If the error between the currently detected movement trajectories of the organism is within the preset error range, the core tracking point will be used as the correct obstacle position; If the error is outside the preset error range, the core tracking point is used as the wrong obstacle position; the correct obstacle position is displayed, and the wrong obstacle position is deleted.

For example, the center position of the obstacle image block is determined as the core tracking point, the Kalman filter method is used to track the trajectory of the organism, and the detection results of S4.2 and S4.3 are confirmed through consecutive multi-frame images (t frames). Specifically, the trajectory tracking is continued for the organisms detected in the historical frame data, where the gradient series [t-15,t-13,t-11,t-9,…,t-3,t] is used as the history In the detection frame, if the error between the predicted biological trajectory and the currently detected biological trajectory is within the preset error range, that is, the biological trajectory can be effectively modeled in the image of the above frame, Then, it is considered that the current biological detection result is valid, the core tracking point is used as the correct obstacle position, and the correct obstacle position is affine to the coordinate system where the vehicle is located, so as to display the correct obstacle position in the coordinate system. If the error between the predicted trajectory of the organism and the currently detected trajectory of the organism is outside the preset error range, that is, the difference between the current value and the predicted value calculated by Kalman aluminum foil is large, it is considered wrong. Detect, discard and delete the wrong obstacle position. Through correction, the detection accuracy can be effectively improved, and false results can be avoided as much as possible from being displayed to the user.

270. Affine the correct obstacle position into the coordinate system where the vehicle is located, so as to display the correct obstacle position in the coordinate system.

For the correct result, affine it into the coordinate system and show it to the user, which is beneficial to the intelligent driving system to control the vehicle later. For example, when an organism is detected in front of the car, it will avoid or remind the driver, and predict the walking trajectory and direction of the organism detected in the distance to avoid subsequent dangers. It can effectively assist or participate in the intelligent driving system or the assisted driving system by sending out the situation in front of the car.

Specifically, the perspective transformation method can be used to affine the position of the obstacle in the regional image, and affine the position of the obstacle to the coordinate system where the vehicle is located to determine the position of the obstacle. For example, you can affine into a coordinate system with the car as the origin.

In some embodiments, the model processing method may specifically include: firstly acquiring information of the user's electronic device through the information perception layer (specifically including electronic device operation information, user behavior information, information acquired by various sensors, electronic device status information, electronic device Display content information, download information on electronic devices, etc.), and then process the information of electronic devices through the data processing layer (such as extracting regional images, etc.), and then extract the required information from the information processed by the data processing layer through the feature extraction layer. The obstacle image block and the correct obstacle position (for the acquisition of the obstacle image block and the correct obstacle position, please refer to the description of the above embodiment), and then the correct obstacle position is input into the scene modeling layer, and the scene modeling layer includes a preset The stored prediction model, the prediction model of the scenario modeling layer is trained according to the correct obstacle position, and the prediction model is continuously optimized. In addition, the intelligent service layer can use the prediction model to make predictions, and judge whether the obstacle position is correct according to the prediction results.

It should be understood that, in the embodiments of the present application, terms such as "first" and "second" are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. exchange.

During specific implementation, the present application is not limited by the execution order of the described steps, and certain steps may also be performed in other sequences or simultaneously under the condition of no conflict.

As can be seen from the above, the method for detecting obstacles around a vehicle provided by the embodiments of the present application first obtains regional images of at least two regions around the vehicle; extracts the regions of interest in the regional images; and determines obstacles in the regional images by detecting the regions of interest Image patch; the obstacle image patch is corrected to output the correct obstacle location. In this embodiment of the present application, the location of the obstacle is determined and corrected by detecting the region of interest, and by acquiring multiple area images, the location of the obstacle around the vehicle can be effectively focused, and the detection speed and accuracy of the obstacle can be accelerated, which is beneficial to the subsequent control of the vehicle and ensures driving. Safety.

Referring to FIG. 5 , FIG. 5 is a schematic structural diagram of a first structure of a device for detecting obstacles around a vehicle provided by an embodiment of the present application. The device 300 for detecting obstacles around a vehicle may be integrated into an electronic device, and the device 300 for detecting obstacles around a vehicle includes an acquisition module 301 , an extraction module 302 , a detection module 303 and a correction module 304 .

an acquisition module 301, configured to acquire regional images of at least two areas around the vehicle;

an extraction module 302, configured to extract the region of interest in the region image;

The detection module 303 is used to determine the obstacle image block in the regional image by detecting the region of interest;

A correction module 304, configured to correct the obstacle image block to output the correct obstacle position;

The optimization module 305 is used to continuously optimize the region of interest, and is specifically configured to: input the obstacle position as a feedback signal into the learning algorithm model to adjust the area of the region of interest.

Please also refer to FIG. 6 . FIG. 6 is a schematic diagram of a second structure of the device for detecting obstacles around a vehicle according to an embodiment of the present application. In some embodiments, the detection module 303 detects the region of interest extracted by the extraction module 302, and the detection module 303 can be used to detect obstacles in the region of interest, and determine obstacle image blocks in the region image, specifically including:

The first judging unit 3031 is used to input the region of interest in the convolutional neural network model, and determine the probability that the region of interest contains obstacles;

The first determining unit 3032 is configured to determine, when the probability is greater than or equal to a preset probability threshold, a part of the region image corresponding to the region of interest as an obstacle image block.

Please refer to FIG. 7 together. FIG. 7 is a third structural schematic diagram of the device for detecting obstacles around a vehicle according to an embodiment of the present application. In some embodiments, the first obstacle image block may be determined by performing cluster regression on the regional image, and the obstacle image block determined by the detection module 303 is used as the second obstacle image block. At this time, the peripheral obstacle detection device may further A determination module 306 is included, which is configured to perform cluster regression on the regional images to determine the first obstacle image block. In addition, the detection module 303 may also include:

The calculation unit 3033 is used to set different weights for the first obstacle image block and the second obstacle image block, and calculate the expected value of the obstacle image block according to the weight;

The truncation unit 3034 is configured to set the threshold value to truncate the expected value of the obstacle image block, so as to determine the obstacle image block in the area image.

At this time, the detection module 303 determines the obstacle image block through further calculation to increase the detection accuracy.

Please continue to refer to FIG. 8 . FIG. 8 is a schematic diagram of a fourth structure of the device for detecting obstacles around a vehicle according to an embodiment of the present application. In some embodiments, the determination module 306 performs cluster regression on the regional images to determine the first obstacle image block, which may specifically include:

The processing unit 3061 is used to divide and compress the regional image to obtain several image blocks;

For each input frame of regional image, the processing unit 3061 first processes an image block in the regional image, and then processes each image block in each frame of regional image in a loop.

Extraction unit 3062, for extracting feature values from the image block;

The second judging unit 3063 is used for judging whether the image block contains obstacle features according to the feature value;

The second determining unit 3064 is configured to, when it is determined that the image block contains obstacle features, determine the part of the area image corresponding to the image block as the first obstacle image block.

Please continue to refer to FIG. 9 , FIG. 9 is a schematic diagram of a fifth structure of the apparatus for detecting obstacles around a vehicle provided by an embodiment of the present application. In some embodiments, the correction module 304 corrects the obstacle image block determined by the detection module 303 to output the correct obstacle position. Correction module 304 may include:

The prediction unit 3041 is configured to determine the core tracking point in the obstacle image block, and use the filtering method to predict the movement trajectory of the living body.

If the error between the predicted movement trajectory of the organism and the currently detected movement trajectory of the organism is within the preset error range, the core tracking point is used as the correct obstacle position;

If the error between the predicted movement trajectory of the organism and the currently detected movement trajectory of the organism is outside the preset error range, the core tracking point is used as the wrong obstacle position.

The display unit 3042 is used to display the correct obstacle position and delete the wrong obstacle position, and is specifically used to affine the correct obstacle position into the coordinate system where the vehicle is located, so as to display the correct obstacle in the coordinate system Location.

In some embodiments, the acquisition module 301 acquires regional images of at least two regions around the vehicle through lenses at different fixed positions, so as to locate and track obstacles. Obstacles include living bodies and non-living bodies, where living bodies include pedestrians, animals, etc., and non-living bodies include trees, railings, etc. around vehicles. Specifically, they can be obtained through the information perception layer in the panoramic perception architecture. When the acquired obstacle is a living body or a pedestrian in particular, the correction module 304 predicts the behavioral trajectory of the living body, so as to control or assist the vehicle to avoid possible dangers.

As can be seen from the above, an embodiment of the present application provides a device for detecting obstacles around a vehicle. First, the acquisition module 301 acquires regional images of at least two regions around the vehicle; the extraction module 302 extracts the regions of interest in the regional images; the detection module 303 detects For the region of interest, an obstacle image block is determined in the area image; the correction module 304 corrects the obstacle image block to output the correct obstacle position. In this embodiment of the present application, the location of the obstacle is determined and corrected by detecting the region of interest, and by acquiring multiple area images, the location of the obstacle around the vehicle can be effectively focused, and the detection speed and accuracy of the obstacle can be accelerated, which is beneficial to the subsequent control of the vehicle and ensures driving. Safety. In addition, the optimization module 305 continuously optimizes the region of interest to further improve the detection accuracy.

The embodiments of the present application also provide an electronic device. The electronic device can be a smartphone, a tablet computer, a gaming device, an AR (Augmented Reality) device, a car, a vehicle surrounding obstacle detection device, an audio playback device, a video playback device, a notebook, a desktop computing device, a wearable device such as a watch , glasses, helmets, electronic bracelets, electronic necklaces, electronic clothing and other equipment.

Referring to FIG. 10 , FIG. 10 is a first structural schematic diagram of an electronic device 400 provided by an embodiment of the present application. The electronic device 400 includes a processor 401 and a memory 402 . The processor 401 is electrically connected to the memory 402 .

The processor 401 is the control center of the electronic device 400, uses various interfaces and lines to connect various parts of the entire electronic device, executes the electronic device by running or calling the computer program stored in the memory 402, and calling the data stored in the memory 402. Various functions of the device and processing data, so as to carry out the overall monitoring of the electronic device.

In this embodiment, the processor 401 in the electronic device 400 loads the instructions corresponding to the processes of one or more computer programs into the memory 402 according to the following steps, and is executed by the processor 401 and stored in the memory 402 A computer program in , which implements various functions:

Obtain area images of at least two areas around the vehicle;

Extract the region of interest in the region image;

By detecting the region of interest, the obstacle image block is determined in the regional image;

Correction of the obstacle image patch to output the correct obstacle location.

In some embodiments, after extracting the region of interest in the region image, the processor 401 performs the following steps:

Continuous optimization of the region of interest; includes:

The obstacle location is input into the learning algorithm model as a feedback signal to adjust the area of interest.

In some embodiments, when the obstacle image block is determined in the area image by detecting the region of interest, the processor 401 performs the following steps:

Obstacle image blocks are determined in the area image by detecting obstacles in the region of interest; specifically, it includes:

Input the region of interest in the convolutional neural network model, and determine the probability that the region of interest contains obstacles;

When the probability is greater than or equal to the preset probability threshold, the part of the region image corresponding to the region of interest is determined as an obstacle image block.

In some embodiments, before determining the obstacle image block in the area image by detecting the region of interest, the processor 401 performs the following steps:

Determine the first obstacle image block by performing cluster regression on the regional image;

When an obstacle image block is determined in the area image by detecting the region of interest, the processor 401 performs the following steps:

Determine the second obstacle image block by performing obstacle detection on the region of interest;

Set different weights for the first obstacle image block and the second obstacle image block, and calculate the expected value of the obstacle image block according to the weight;

Set the threshold to truncate the expected value of the obstacle image block to identify the obstacle image block in the area image.

In some embodiments, when the first obstacle image block is determined by performing cluster regression on the regional images, the processor 401 performs the following steps:

Divide and compress the regional image to obtain several image blocks;

Extract feature values from image blocks;

Determine whether the image block contains obstacle features according to the feature value;

When it is determined that an obstacle feature is contained in the image block, a part of the area image corresponding to the image block is determined as the first obstacle image block.

In some embodiments, when the first obstacle image block is determined by performing cluster regression on the regional image, the processor 401 first processes an image block in the regional image for each frame of the input regional image, and cyclically processes each image block in the regional image. Each image block in the frame area image.

In some embodiments, when correcting the obstacle image block, the processor 401 performs the following steps:

Determine the core tracking point in the obstacle image block, and use the filtering method to predict the trajectory of the organism;

If the error between the predicted movement trajectory of the organism and the currently detected movement trajectory of the organism is within the preset error range, the core tracking point is used as the correct obstacle position;

If the error between the predicted movement trajectory of the organism and the currently detected movement trajectory of the organism is outside the preset error range, the core tracking point is used as the wrong obstacle position;

Display the correct obstacle position and delete the wrong obstacle position.

In some embodiments, when displaying the correct obstacle position, the processor 401 performs the following steps:

Affine the correct obstacle location into the vehicle's coordinate system to show the correct obstacle location in the coordinate system.

Please continue to refer to FIG. 11 , which is a schematic diagram of a second structure of an electronic device 400 provided by an embodiment of the present application. The electronic device 400 further includes: a display screen 403 , a control circuit 404 , an input unit 405 , a sensor 406 and a power supply 407 . The processor 401 is electrically connected to the display screen 403 , the control circuit 404 , the input unit 405 , the sensor 406 and the power supply 407 , respectively.

The display screen 403 may be used to display information input by or provided to the user and various graphical user interfaces of the electronic device, which may be composed of images, text, icons, videos, and any combination thereof.

The control circuit 404 is electrically connected to the display screen 403 for controlling the display screen 403 to display information.

The input unit 405 may be used to receive input numbers, character information or user characteristic information (eg fingerprints), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Wherein, the input unit 405 may include a fingerprint identification module.

The sensor 406 is used to collect the information of the electronic device itself or the user's information or the external environment information. For example, sensors 406 may include distance sensors, magnetic field sensors, light sensors, acceleration sensors, fingerprint sensors, Hall sensors, position sensors, gyroscopes, inertial sensors, attitude sensors, barometers, heart rate sensors, and the like.

Power supply 407 is used to power various components of electronic device 400 . In some embodiments, the power supply 407 may be logically connected to the processor 401 through a power management system, so as to implement functions such as managing charging, discharging, and power consumption through the power management system.

Although not shown in FIG. 10 and FIG. 11 , the electronic device 400 may further include a camera, a Bluetooth module, and the like, which will not be repeated here.

As can be seen from the above, the embodiment of the present application provides an electronic device, and the processor in the electronic device performs the following steps: firstly acquiring regional images of at least two regions around the vehicle; extracting regions of interest in the regional images; The obstacle image block is determined in the area image; the obstacle image block is corrected to output the correct obstacle position. In this embodiment of the present application, the location of the obstacle is determined and corrected by detecting the region of interest, and by acquiring multiple area images, the location of the obstacle around the vehicle can be effectively focused, and the detection speed and accuracy of the obstacle can be accelerated, which is beneficial to the subsequent control of the vehicle and ensures driving. Safety.

An embodiment of the present application further provides a storage medium, where a computer program is stored in the storage medium, and when the computer program runs on the computer, the computer executes the method for detecting obstacles around a vehicle in any of the foregoing embodiments.

For example, in some embodiments, when a computer program is run on a computer, the computer performs the following steps:

Obtain area images of at least two areas around the vehicle;

Extract the region of interest in the region image;

By detecting the region of interest, the obstacle image block is determined in the regional image;

Correction of the obstacle image patch to output the correct obstacle location.

It should be noted that those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, The storage medium may include, but is not limited to, a read only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and the like.

The method, device, storage medium, and electronic device for detecting obstacles around a vehicle provided by the embodiments of the present application have been described in detail above. The principles and implementations of the present application are described herein using specific examples, and the descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; meanwhile, for those skilled in the art, according to the Thoughts, there will be changes in the specific implementation and application scope. In conclusion, the content of this specification should not be construed as a limitation on the application.

Claims (11)

1. A method for detecting obstacles around a vehicle, wherein the method comprises: Obtain area images of at least two areas around the vehicle; extracting a region of interest in the region image; By detecting the region of interest, an obstacle image block is determined in the region image; The obstacle image blocks are corrected to output the correct obstacle location. 2. The method for detecting obstacles around a vehicle according to claim 1, wherein after extracting the region of interest in the region image, the method further comprises: Continuously optimize the region of interest; specifically: The obstacle location is input into a learning algorithm model as a feedback signal to adjust the area of interest. 3 . The method for detecting obstacles around a vehicle according to claim 2 , wherein, by detecting the region of interest, determining an obstacle image block in the region image, comprising: 3 . By performing obstacle detection on the region of interest, an obstacle image block is determined in the region image; specifically, it includes: Inputting the region of interest in the convolutional neural network model, and judging the probability that the region of interest contains obstacles; When the probability is greater than or equal to a preset probability threshold, a part of the region image corresponding to the region of interest is determined as an obstacle image block. 4 . The method for detecting obstacles around a vehicle according to claim 1 , wherein, before determining the obstacle image block in the area image by detecting the region of interest, the method further comprises: 5 . Determine the first obstacle image block by performing cluster regression on the area image; The determining of the obstacle image block in the region image by detecting the region of interest includes: By performing obstacle detection on the region of interest, a second obstacle image block is determined; Setting different weights for the first obstacle image block and the second obstacle image block, and calculating the expected value of the obstacle image block according to the weight; A threshold is set to truncate the expected value of the obstacle image block to determine the obstacle image block in the area image. 5 . The method for detecting obstacles around a vehicle according to claim 4 , wherein the determining the first obstacle image block by performing cluster regression on the regional images, comprising: 6 . Divide and compress the area image to obtain several image blocks; extracting feature values from the image block; Judging whether the image block contains obstacle features according to the feature value; When it is determined that the image block contains an obstacle feature, a part of the area image corresponding to the image block is determined as the first obstacle image block. 6 . The method for detecting obstacles around a vehicle according to claim 5 , wherein the determining the first obstacle image block by performing cluster regression on the regional images, further comprising: 6 . For each frame of the input area image, first process an image block in the area image; Loop through each image block in each frame of region image. 7. The method for detecting an obstacle around a vehicle according to any one of claims 1 to 6, wherein the obstacle comprises a living body, and the correcting the obstacle image block comprises: Determine the core tracking point in the obstacle image block, and use the filtering method to predict the trajectory of the organism; If the error between the predicted biological movement trajectory and the currently detected biological movement trajectory is within the preset error range, the core tracking point is used as the correct obstacle position; If the error between the predicted movement trajectory of the organism and the currently detected movement trajectory of the organism is outside the preset error range, the core tracking point is used as the wrong obstacle position; The correct obstacle position is displayed, and the wrong obstacle position is deleted. 8. The method for detecting obstacles around a vehicle according to claim 7, wherein the displaying the correct obstacle position comprises: The correct obstacle location is affine into the coordinate system where the vehicle is located to show the correct obstacle location in the coordinate system. 9. A device for detecting obstacles around a vehicle, wherein the device comprises: an acquisition module for acquiring regional images of at least two areas around the vehicle; an extraction module for extracting the region of interest in the region image; a detection module, configured to determine an obstacle image block in the region image by detecting the region of interest; The correction module is used for correcting the obstacle image block to output the correct obstacle position. 10. A storage medium, wherein a computer program is stored in the storage medium, the computer program is used to update an algorithm model, the algorithm model includes a first algorithm module, and the first algorithm module is used to update the algorithm model. Preset tasks are processed, and when the computer program is run on a computer, the computer is made to perform the following steps: Obtain area images of at least two areas around the vehicle; extracting a region of interest in the region image; By detecting the region of interest, an obstacle image block is determined in the region image; The obstacle image blocks are corrected to output the correct obstacle location. 11. An electronic device, wherein the electronic device comprises a processor and a memory, a computer program is stored in the memory, the computer program is used to update an algorithm model, the algorithm model includes a first algorithm module, The first algorithm module is used to process a preset task, and the processor is used to execute the following steps by calling the computer program stored in the memory: Obtain area images of at least two areas around the vehicle; extracting a region of interest in the region image; By detecting the region of interest, an obstacle image block is determined in the region image; The obstacle image blocks are corrected to output the correct obstacle location. The first obstacle image block.

Images