CN114332941A - Alarm prompting method and device based on riding object detection and electronic equipment - Google Patents

Abstract

The disclosure relates to an alarm prompting method, an alarm prompting device and electronic equipment based on riding object detection, wherein the alarm prompting method comprises the following steps: acquiring an image of a to-be-detected area of a vehicle under the condition that the vehicle is parked; detecting a riding object of the vehicle based on the image; and generating an alarm prompt when the riding object of the vehicle is determined to comprise a child passenger or a pet and not comprise an adult passenger according to the detection result of the riding object. According to the embodiment of the disclosure, due to the adoption of the image detection mode, the camera installed in the vehicle can be directly called to acquire the image, so that the redundancy degree of the parts is reduced. In addition, the embodiment of the disclosure can identify adults, children and pets in the image, children or pets exist in the image, and when no adult exists, an alarm prompt is generated, so that the probability of misinformation is reduced.

Description

Alarm prompting method, device and electronic device based on vehicle object detection

technical field

The present disclosure relates to the technical field of detection, and in particular, to an alarm prompting method, device and electronic device based on vehicle object detection.

Background technique

In recent years, due to the rapid development of the automobile industry, most adults will drive with children and pets, but if adults do not intend to leave children or pets in the car, it will easily cause physical or psychological discomfort to children or pets, especially In the case of hot weather, children or pets are prone to heat stroke, hypoxia and other situations that endanger their lives.

SUMMARY OF THE INVENTION

The present disclosure proposes a technical solution for alarm prompting.

According to an aspect of the present disclosure, there is provided an alarm prompting method based on vehicle object detection, the alarm prompting method comprising: when the vehicle is parked, acquiring an image of a to-be-detected area of the vehicle; detecting an area based on the image The riding object of the vehicle; when it is determined according to the detection result of the riding object that the riding object of the vehicle includes child passengers or pets, but does not include adult passengers, generate an alarm prompt.

In a possible implementation manner, the detecting a ride-on object of the vehicle based on the image includes: detecting a ride-on object of the vehicle based on the image, where the ride-on object to be taken care of includes a child and/or pets; when it is determined according to the detection result of the occupants of the vehicle that the occupants of the vehicle include child occupants or pets, an adult occupant of the vehicle is detected based on the image.

In a possible implementation manner, when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant, generate an alarm prompt, including: : determining the opening and closing state of the windows of the vehicle when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant; When the car window is in an open state, a first alarm prompt is generated; when the car window is in a closed state, a second alarm prompt is generated; the urgency of the second alarm prompt is higher than that of the first alarm prompt degree of urgency.

In a possible implementation manner, the determining the opening and closing state of the window of the vehicle includes: detecting the opening and closing state of the window of the vehicle based on the image, and/or based on the The sensor of the window acquires the opening and closing state of the window.

In a possible implementation manner, the generating an alarm prompt when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant, and further The method includes: after generating the first alarm prompt when the vehicle window is in an open state, and when detecting an abnormality in the physical state of a child occupant and/or pet of the vehicle according to the image, generating a first alarm prompt. Two alarm prompts.

In a possible implementation manner, the acquiring an image of the to-be-detected area of the vehicle includes: acquiring an image of at least one viewing angle inside the vehicle captured by at least one camera disposed in the vehicle.

In a possible implementation manner, in the case of acquiring images from multiple viewing angles of the vehicle captured by multiple cameras disposed on the vehicle, the detecting a riding object of the vehicle based on the images includes: : Detect the riding object of the vehicle based on the images from the multiple perspectives, respectively, and fuse the detection results of the images from the multiple perspectives; or fuse the images from the multiple perspectives, and detect based on the fused images. The ride object of the vehicle.

In a possible implementation manner, the to-be-detected area includes: a seat area and a rear area of the vehicle; the detecting a riding object of the vehicle based on the image includes: detecting the vehicle based on an image of the seat area A riding object in a vehicle; in the case where the riding object in the vehicle includes a child occupant or a pet, an adult occupant of the vehicle is detected based on the image of the seating area and the image of the rear area of the vehicle.

In a possible implementation, the detecting an adult occupant of the vehicle based on the image of the seat area and the image of the rear area of the vehicle includes: detecting the adult occupant based on the image of the rear area of the vehicle. In the case where there is an adult near the trunk at the rear of the vehicle and the obtained state of the trunk is open, it is determined that the riding object of the vehicle includes an adult occupant; In the case where the trunk at the rear of the vehicle is open and there are adults carrying items near the trunk at the rear of the vehicle, it is determined that the riding objects of the vehicle include adult passengers.

In a possible implementation manner, the detecting the riding object of the vehicle based on the image includes: performing face detection based on the image, and determining a candidate object in the image; The extracted body features and/or facial features of the candidate object determine the type of the vehicle's riding object, and the type is one of children, pets and adults.

According to a second aspect of the present disclosure, there is provided an alarm prompting device based on vehicle object detection, the alarm prompting device comprising: an image acquisition module, configured to acquire an image of a to-be-detected area of the vehicle when the vehicle is parked an image; an object detection module for detecting the riding object of the vehicle based on the image; a prompt generation module for determining that the riding object of the vehicle includes a child occupant or a pet according to the detection result of the riding object , and does not include adult occupants, generate an alarm prompt.

According to a third aspect of the present disclosure, there is provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute Any one of the alarm prompting methods described.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing any one of the alarm prompting methods when executed by a processor.

In the embodiment of the present disclosure, when the vehicle is parked, an image of the to-be-detected area of the vehicle can be acquired, and then the riding object of the vehicle can be detected based on the image, and finally the riding object can be detected according to the detection result of the riding object. If it is determined that the riding object of the vehicle includes a child passenger or a pet, and does not include an adult passenger, an alarm prompt is generated. Since the embodiment of the present disclosure adopts the method of image detection, the camera installed in the vehicle can be directly called to acquire the image, thereby reducing the redundancy of components. In addition, the embodiments of the present disclosure can identify adults, children, and pets in the image, and when there are children or pets in the image but no adults, an alarm prompt is generated, thereby reducing the probability of false alarms.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure.

FIG. 1 is a flowchart of an alarm prompting method provided by an embodiment of the present disclosure.

FIG. 2 is a flowchart of an alarm prompting method according to an embodiment of the present disclosure.

FIG. 3 is a schematic reference diagram of an alarm prompting method provided by an embodiment of the present disclosure.

FIG. 4 is a block diagram of an alarm prompting device provided by an embodiment of the present disclosure.

FIG. 5 is a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are set forth in the following detailed description. It will be understood by those skilled in the art that the present disclosure may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

In related technologies, infrared monitoring equipment is usually installed in the car to monitor children, but it is easy to cause the following two problems: 1. Redundancy of parts, that is, car owners often need to install infrared monitoring equipment to monitor children, occupying the car. Inner space, and infrared monitoring equipment is used too little. 2. It is prone to false positives, that is, this method will monitor living bodies indiscriminately, and false positives are very likely to occur. For example, if an adult is resting in the car or there are pedestrians passing by around the car, the infrared monitoring device will think that the detected child is a child, resulting in a false alarm.

In view of this, an embodiment of the present disclosure provides an alarm prompting method based on the detection of a riding object. When the vehicle is parked, an image of the to-be-detected area of the vehicle is acquired, and then based on the image, the riding of the vehicle is detected. The vehicle object finally generates an alarm prompt when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant. Since the embodiment of the present disclosure adopts the method of image detection, the camera installed in the vehicle can be directly called to acquire the image, thereby reducing the redundancy of components. In addition, the embodiments of the present disclosure can identify adults, children, and pets in the image, and when there are children or pets in the image but no adults, an alarm prompt is generated, thereby reducing the probability of false alarms.

Exemplarily, the above-mentioned alarm prompting method can be executed by a vehicle-mounted terminal (also referred to as a perception processor), and the vehicle-mounted terminal can interact with the vehicle-mounted camera to obtain the above-mentioned image. For example, the above vehicle cameras may include: OMS (Occupancy Monitoring System) cameras (also called passenger monitoring system cameras), DMS (Driver Monitoring System) cameras (also called driver monitoring system cameras) and other arbitrary vehicle cameras. Exemplarily, the above-mentioned vehicle-mounted camera can also be represented as an external camera, for example, an image can be acquired through a camera in a user's mobile phone, and the vehicle-mounted terminal can be interacted with through a wireless network.

Referring to FIG. 1, FIG. 1 is a flowchart of an alarm prompting method provided by an embodiment of the present disclosure, including:

Step S10, when the vehicle is parked, acquire an image of the to-be-detected area of the vehicle. Exemplarily, whether the vehicle is in a parked state can be determined by a vehicle speed sensor in the vehicle, for example, if the vehicle speed sensor detects that the vehicle speed is 0 km/h, it is determined that the vehicle is in a parked state. In an example, the vehicle speed sensor and the vehicle lock sensor can also be used to determine whether the vehicle is in a parked state. For example, if the vehicle speed sensor detects that the vehicle speed is 0km/h and the vehicle lock sensor is in a locked state, it is determined that the vehicle is in a parked state. state. Combined with the actual situation, when the vehicle speed is 0km/h, there may be two situations: 1. The vehicle stops and the driver does not leave the vehicle. 2. The vehicle is stopped and the driver has left the vehicle. Therefore, in combination with the state of the car lock sensor, when the car lock sensor is locked, it is determined that the driver has left the car with a high probability, and then the alarm prompting method provided by the embodiment of the present disclosure is executed, thereby saving computing power, and more efficient. In line with practical application scenarios.

In an example, the above-mentioned area to be detected may include: at least one of the seating area and the rear area of the vehicle. In a possible implementation, step S10 may include: acquiring the interior of the vehicle captured by at least one camera disposed in the vehicle images of at least one viewing angle. In other words, the above-mentioned in-vehicle camera can be arranged in the seat area and the rear area of the vehicle. Exemplarily, the number of the above-mentioned in-vehicle cameras may be multiple, so as to reduce the probability that children and pets cannot be detected because they are blocked by the seat. For example, at least one OMS camera can be set up in the rearview mirror, reading lights in the rear of the cabin, etc. to realize joint detection, and the vehicle terminal can also retrieve images from the DMS camera. In other words, the number of images at the same time can be greater than one.

Step S20 , detecting the riding object of the vehicle based on the image.

In a possible implementation manner, the occupant of the vehicle may be detected by performing human detection and/or animal detection on the image. The image features can be matched with a predetermined human feature template and/or animal feature template, or the image can be input into a neural network model for human detection and/or an image is input into a neural network model for animal detection, so as to obtain The detection result of the riding object of the vehicle.

In a possible implementation manner, step S20 may include: performing face detection based on the image, and determining a candidate object in the image. Based on the body features and/or facial features of the candidate object extracted from the image, the type of the riding object of the vehicle is determined, and the type is one of children, pets and adults. Exemplarily, the candidate object extracted above can be represented as an image region (or called a region of interest) with candidate objects in the image, and then the object of the ride is determined by extracting the body features and/or facial features of the image region. type. By extracting the region of interest, the subsequently extracted body features and facial features can be more representative, and the influence of the background image on them can be reduced. In addition, this method can effectively save computing power, which is conducive to improving the efficiency of the alarm prompting process.

Exemplarily, body features and/or facial features of candidate objects can be extracted through a key point extraction network in the related art. For example, body features can be expressed as the relative distribution state between skeleton key points, and facial features can be expressed as facial key points. Relative distribution status between points. In the model training stage, the keypoints marked with different types of bones or facial keypoints can be input into the model containing the keypoint extraction network, so that the trained model can determine the corresponding multiplication factor of the candidate object according to the body features and facial features. The type of the car object.

In one example, vehicle object detection can be performed based on a trained neural network model, the model can include the above-mentioned key point extraction network, and/or a posture feature discrimination network and a facial feature discrimination network, a posture discrimination network and a facial feature discrimination network The networks respectively include an adult discriminant sub-network, a child discriminant sub-network, and a pet discriminant sub-network. The model obtains the confidence score of each type of the candidate object in the facial features by inputting the facial features into the three sub-networks in the facial feature discrimination network, and determines the candidate object by inputting the posture features into the three sub-networks in the posture feature discrimination network. Confidence scores for each type in postural traits. Then the model can obtain the average score corresponding to each type by weighting and averaging the confidence scores of the same type, and use the type with the highest average score as the type corresponding to the candidate object.

Exemplarily, the training data of the model can also be preprocessed, for example, the skeletal key points and/or facial key points of a specific age group can be used as training data, so that the trained model can target the specific age group in the candidate object. of children for accurate classification. Another example: the above model may include multiple pet identification sub-networks, different pet identification sub-networks can accurately identify different types of pets, the user can specify a specific type of pet in the vehicle terminal, and then the machine model calls the corresponding pet of the specific type. Pet discrimination sub-network to improve the accuracy of candidate object type determination. The above model confirms the type of the candidate object by means of multi-classification. In an example, the above-mentioned sub-networks can also be divided into multiple models, and then the type confirmation can be carried out by performing two classifications respectively. The present disclosure The embodiment is not limited here.

Referring to FIG. 2, FIG. 2 is a flowchart of an alarm prompting method according to an embodiment of the present disclosure. In a possible implementation, step S20 may include:

Step S21 , based on the image, detect a ride-on object of the vehicle to be taken care of, where the ride-on object to be taken care of includes children and/or pets.

Step S22 , in a case where it is determined according to the detection result of the boarding object to be taken care of that the boarding object of the vehicle includes a child occupant or a pet, detect an adult occupant of the vehicle based on the image. Combined with the actual situation, if it is determined in the detection results that the riding objects include child passengers or pets, it proves that the riding objects that need to be taken care of may be left alone in the car. In this case, it is necessary to further confirm whether the image An adult occupant is present. This step can be obtained through the above-mentioned adult discriminating sub-network or its corresponding two-class model, which is not described in detail in this embodiment of the present disclosure.

In a possible implementation manner, in the case of acquiring images from multiple perspectives of the vehicle captured by multiple cameras disposed on the vehicle, step S20 may include: detecting, respectively, based on the images from the multiple perspectives The boarding object of the vehicle is fused, and the detection results of the images from multiple perspectives are fused; or the images from the multiple perspectives are fused, and the boarding object of the vehicle is detected based on the fused image. For example, fusing the detection results of images from multiple perspectives may be: taking the collection of detection results of images from multiple perspectives as the detection result of the riding object, or the detection results of each image include corresponding confidence information. In the case of conflicting detection results of images from different viewing angles, the detection result with higher confidence is selected. Exemplarily, images from multiple perspectives can also be fused through an image fusion algorithm to highlight image features and enhance detection robustness. For example, the image fusion algorithm may be a pixel-level fusion algorithm, a feature-level fusion algorithm, a decision-level fusion algorithm, etc., which is not limited in this embodiment of the present disclosure. The embodiments of the present disclosure can improve the accuracy of the detection results by fusing images from multiple viewing angles or the detection results thereof.

In a possible implementation manner, when the area to be detected includes a seat area and a rear area of the vehicle, step S20 may include: detecting a riding object in the vehicle based on an image of the seat area. In a case where the occupant of the vehicle includes a child occupant or a pet, an adult occupant of the vehicle is detected based on the image of the seat area and the image of the rear area of the vehicle.

In combination with the actual situation, the embodiment of the present disclosure takes into account the situation of adults carrying objects at the rear of the vehicle. In this case, since the adults are still near the object to be taken care of, it is still safe to take care of the object to be taken care of. That is, in the embodiment of the present disclosure, it is possible to accurately know whether the object to be taken care of remains in the car without being accompanied or cared by an adult by collecting an image of the rear area of the car. In one example, this step may be: in the case that an adult is detected near the trunk at the rear of the vehicle based on the image of the rear area of the vehicle, and the obtained state of the trunk is an open state, determine the The occupants of the vehicle include adult occupants. Exemplarily, the state of the trunk can be obtained by analyzing a cabin image collected by a camera in the vehicle, or acquired by a sensor of the trunk, which is not described in detail in this embodiment of the present disclosure.

By determining the trunk state, the embodiments of the present disclosure can further determine the actual leftover state of the ride-on object to be taken care of. For example: if the trunk is closed, but there is an adult in the rear area of the car, the adult is more likely to be a passerby, that is, in this case, the person who needs to take care of the car is still left alone, posing a safety risk . In one example, the step may further be: in a case where it is detected based on the image of the rear area of the vehicle that the trunk of the rear of the vehicle is opened and there is an adult carrying an item near the trunk of the rear of the vehicle, determining The occupants of the vehicle include adult occupants. Exemplarily, whether there are adults carrying items near the trunk at the rear of the vehicle can be determined by a machine learning model. Exemplarily, the input of the machine learning model during training is: images of adults without items and carrying An adult image of the item, so that the trained machine learning model can determine whether there is an adult with an item near the trunk based on the input image. In this way, when the trunk is opened and there is an adult carrying items near the trunk, the adult is considered to be an occupant object who temporarily leaves the vehicle and may quickly return to the vehicle.

In one example, the age of the candidate object may be detected by a trained neural network model, and the age attribute of the candidate object or the age group attribute to which the candidate object belongs may be determined by comparing with an age threshold. In this case, the neural network model predicts age as a regression problem or a classification problem. If the regression problem is used for age prediction, the input of the model during training is a face image or a human body image marked with a specific age value, so that the trained model can output a specific age value. If the age prediction is performed with a classification problem, the input of the model during training is a face image or a human body image marked with an age group, so that the trained model can classify the object in the face image or human body image into a specific age group. The above-mentioned age threshold may be set by a developer or a user, which is not limited in this embodiment of the present disclosure. The embodiment of the present disclosure can perform alarm detection for occupants of a specific age or specific age group by detecting the age or age group attribute of the candidate object, so as to improve the user's operability and better fit the user's actual application situation. For example, the age thresholds include 12 years old and 18 years old. The occupant objects older than or equal to 18 years old are considered as adult objects, and the occupant objects under 12 years old are considered as children who need to be taken care of.

Continuing to refer to FIG. 1 , in step S30, an alarm prompt is generated when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet, and does not include an adult occupant.

Exemplarily, the above-mentioned warning prompts may be represented as vehicle lights flashing and/or alarms sending an alarm tone, that is, the vehicle-mounted terminal can be electrically connected to the vehicle lights and the alarm. In one example, the above-mentioned vehicle-mounted terminal can also be bound to a user-portable mobile terminal, and then send a vibration command, a prompt tone command, etc. to the mobile terminal, and remind the user through the mobile terminal, so as to reduce the redundancy of the line connection of the vehicle-mounted terminal. degree.

Continuing to refer to FIG. 2 , in a possible implementation manner, step S30 may include:

Step S31, when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant, determine the opening and closing state of the window of the vehicle.

Exemplarily, the opening and closing state of the vehicle window can be obtained in at least one of the following ways: Solution 1: Detect the opening and closing state of the vehicle window of the vehicle based on the image. Solution 2: Obtain the open/close state of the window based on a sensor provided on the window. The above two methods correspond to different connection states of the vehicle-mounted terminal, respectively. In scheme 1, the vehicle-mounted terminal does not need to be electrically connected to the sensor of the vehicle window, and the detection of the opening and closing state of the vehicle window can be realized through the camera connected to it. Exemplarily, the above image can be processed by a machine learning model to determine the state of the car window, and the input of the machine learning model during training can be an image marked as the open state of the car window, and an image marked as the closed state of the car window. , so that the trained machine learning model can perform binary classification on the input images to determine the opening and closing status of the windows. This solution can effectively reduce the redundancy of line connections. In scheme 2, the vehicle terminal can be electrically connected to the sensor of the car window, and then the opening and closing state of the car window can be determined directly through the working state of the sensor, because the opening and closing state of the car window is directly sent by the sensor of the car window , so scheme 2 can effectively improve the accuracy of the window opening and closing state. Developers can decide by themselves the solution to be used according to actual needs, which is not limited in this embodiment of the present disclosure.

Step S32, when the vehicle window is in an open state, generate a first alarm prompt.

In the embodiment of the present disclosure, if the vehicle window is in the open state, the possibility of heat stroke and hypoxia in the passenger who needs to be taken care of is low. Accident means that the safety level of the object to be taken care of is relatively high, so the first alarm prompt can be generated. The first alarm prompt is lower in urgency than the second alarm prompt described later. Exemplarily, the urgency of the first alarm prompt and the second alarm prompt may be reflected in the following manners: the flashing frequency of the vehicle lights, the alarm volume of the alarm, and the like. For example: the higher the emergency, the faster the flashing frequency of the lights and the higher the alarm volume.

In a possible implementation manner, after the first alarm prompt is generated, the alarm prompt method may further include: when an abnormal physical state of the child occupant and/or pet of the vehicle is detected according to the image Next, a second alarm prompt is generated. Exemplarily, whether the above-mentioned body state is abnormal can be determined by a machine learning model. In one example, the input of the above-mentioned machine learning model during training is an image with a car ride subject to be attended, marked as abnormal, and an image with a car ride subject to be taken care of, marked as normal, so that the trained machine learning According to the input image, the model can classify whether there is an abnormality in the object in the image that needs to be taken care of, so as to determine whether to generate a second alarm prompt.

The various possible implementation manners above can be arranged and combined according to actual needs. For example, the combined alarm prompting method can be directly integrated into the vehicle-mounted terminal to reduce the user's operational difficulty, or the various possible implementations above can be displayed on the screen of the vehicle-mounted terminal, and the user can customize the alarming and prompting method to improve the To meet the actual needs of users, the embodiments of the present disclosure are not limited herein.

Exemplarily, the embodiment of the present disclosure provides an optional alarm prompting method for reference. Referring to FIG. 3 , FIG. 3 is a reference schematic diagram of the alarm prompting method provided by the embodiment of the present disclosure. The in-vehicle terminal determines whether the vehicle is turned off and the locks are closed (ie, whether the vehicle is parked). If so, the information in the vehicle cabin (that is, the above-mentioned images) is collected by the OMS camera. The post-processor (that is, the above-mentioned vehicle-mounted terminal) then processes the information to determine whether there are children or pets under the age of 12 in the vehicle. If it does not exist, it will not alarm. If there is, it is determined whether there is a face frame older than 12 years old (that is, it is determined whether the riding object of the vehicle includes an adult occupant). If it exists, it will not alarm. If it does not exist, it will alarm.

It can be understood that the above-mentioned method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method of the specific embodiment, the specific execution order of each step should be determined by its function and possible internal logic.

In addition, the present disclosure also provides an alarm prompting device, electronic device, computer-readable storage medium, and program based on vehicle object detection, all of which can be used to implement any alarm prompting method provided by the present disclosure, and the corresponding technical solutions and descriptions and Please refer to the corresponding records in the method section, which will not be repeated.

FIG. 4 is a block diagram of an alarm prompting device provided by an embodiment of the present disclosure. As shown in FIG. 4 , the alarm prompting device 100 includes: an image acquisition module 110 for acquiring an image of a to-be-detected area of the vehicle when the vehicle is parked The object detection module 120 is used to detect the riding object of the vehicle based on the image; the prompt generation module 130 is used to determine that the riding object of the vehicle includes a child occupant or a child according to the detection result of the riding object. In the case of pets and excluding adult occupants, an alarm prompt is generated.

In a possible implementation manner, the detecting a ride-on object of the vehicle based on the image includes: detecting a ride-on object of the vehicle based on the image, where the ride-on object to be taken care of includes a child and/or pets; when it is determined according to the detection result of the occupants of the vehicle that the occupants of the vehicle include child occupants or pets, an adult occupant of the vehicle is detected based on the image.

In a possible implementation manner, when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant, generate an alarm prompt, including: : determining the opening and closing state of the windows of the vehicle when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant; When the car window is in an open state, a first alarm prompt is generated; when the car window is in a closed state, a second alarm prompt is generated; the urgency of the second alarm prompt is higher than that of the first alarm prompt degree of urgency.

In a possible implementation manner, the determining the opening and closing state of the window of the vehicle includes: detecting the opening and closing state of the window of the vehicle based on the image, and/or based on the The sensor of the window acquires the opening and closing state of the window.

In a possible implementation manner, the generating an alarm prompt when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant, and further The method includes: after generating the first alarm prompt when the vehicle window is in an open state, and when detecting an abnormality in the physical state of a child occupant and/or pet of the vehicle according to the image, generating a first alarm prompt. Two alarm prompts.

In a possible implementation manner, the acquiring an image of the to-be-detected area of the vehicle includes: acquiring an image of at least one viewing angle inside the vehicle captured by at least one camera disposed in the vehicle.

In a possible implementation manner, in the case of acquiring images from multiple viewing angles of the vehicle captured by multiple cameras disposed on the vehicle, the detecting a riding object of the vehicle based on the images includes: : Detect the riding object of the vehicle based on the images from the multiple perspectives, respectively, and fuse the detection results of the images from the multiple perspectives; or fuse the images from the multiple perspectives, and detect based on the fused images. The ride object of the vehicle.

In a possible implementation manner, the to-be-detected area includes: a seat area and a rear area of the vehicle; the detecting a riding object of the vehicle based on the image includes: detecting the vehicle based on an image of the seat area A riding object in a vehicle; in the case where the riding object in the vehicle includes a child occupant or a pet, an adult occupant of the vehicle is detected based on the image of the seating area and the image of the rear area of the vehicle.

In a possible implementation, the detecting an adult occupant of the vehicle based on the image of the seat area and the image of the rear area of the vehicle includes: detecting the adult occupant based on the image of the rear area of the vehicle. In the case where there is an adult near the trunk at the rear of the vehicle and the obtained state of the trunk is open, it is determined that the riding object of the vehicle includes an adult occupant; In the case where the trunk at the rear of the vehicle is open and there are adults carrying items near the trunk at the rear of the vehicle, it is determined that the riding objects of the vehicle include adult passengers.

In a possible implementation manner, the detecting the riding object of the vehicle based on the image includes: performing face detection based on the image, and determining a candidate object in the image; The extracted body features and/or facial features of the candidate object determine the type of the vehicle's riding object, and the type is one of children, pets and adults.

In some embodiments, the functions or modules included in the apparatuses provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments. For specific implementation, reference may be made to the descriptions of the above method embodiments. For brevity, here No longer.

Embodiments of the present disclosure further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented. Computer-readable storage media can be volatile or non-volatile computer-readable storage media.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

Embodiments of the present disclosure also provide a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

FIG. 5 is a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a User Equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, an in-vehicle device, Terminal devices such as wearable devices.

5, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and communication component 816.

The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operation at electronic device 800 . Examples of such data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. Memory 804 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply assembly 806 provides power to various components of electronic device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when electronic device 800 is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of electronic device 800 . For example, the sensor assembly 814 can detect the open/closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the electronic device 800 or the electronic device 800. Changes in the position of components, presence or absence of user contact with the electronic device 800 , orientation or acceleration/deceleration of the electronic device 800 and changes in the temperature of the electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 can access a wireless network based on communication standards, such as wireless network (Wi-Fi), second generation mobile communication technology (2G), third generation mobile communication technology (3G), fourth generation mobile communication technology (4G) ), Long Term Evolution (LTE) of Universal Mobile Telecommunications Technology, Fifth Generation Mobile Telecommunications Technology (5G), or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

In an exemplary embodiment, a non-volatile computer-readable storage medium, such as a memory 804 comprising computer program instructions executable by the processor 820 of the electronic device 800 to perform the above method is also provided.

The present disclosure may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code, written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims (13)

1. an alarm prompting method based on the detection of a vehicle object, is characterized in that, the alarm prompting method comprises: When the vehicle is parked, obtain an image of the to-be-detected area of the vehicle; Detecting a ride-on object of the vehicle based on the image; In a case where it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant, an alarm prompt is generated. 2 . The alarm prompting method according to claim 1 , wherein the detecting the riding object of the vehicle based on the image comprises: 2 . Detecting a ride-on object of the vehicle based on the image, where the ride-on object to be taken care of includes children and/or pets; In a case where it is determined according to the detection result of the boarding object to be taken care of that the boarding object of the vehicle includes a child occupant or a pet, an adult occupant of the vehicle is detected based on the image. 3 . The alarm prompting method according to claim 1 or 2 , characterized in that, determining that the riding objects of the vehicle include child passengers or pets, and do not include adults, according to the detection result of the riding objects. 4 . In the case of an occupant, an alarm prompt is generated, including: In the case where it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet, and does not include an adult occupant, determining the opening and closing state of the window of the vehicle; generating a first alarm prompt when the vehicle window is in an open state; generating a second alarm prompt when the vehicle window is in a closed state; The urgency of the second alarm prompt is higher than the urgency of the first alarm prompt. 4 . The alarm prompting method according to claim 3 , wherein the determining the opening and closing state of the window of the vehicle comprises: 4 . Detecting the opening and closing states of the windows of the vehicle based on the image, and/or The opening and closing state of the window is acquired based on a sensor provided in the window. 5 . The alarm prompting method according to claim 3 or 4 , characterized in that, determining that the riding object of the vehicle according to the detection result of the riding object includes a child occupant or a pet, and does not include an adult. 6 . In the case of an occupant, an alarm prompt is generated, which also includes: After the first alarm prompt is generated when the vehicle window is in an open state, a second alarm is generated when an abnormal physical state of the child occupant and/or pet of the vehicle is detected according to the image hint. 6. The alarm prompting method according to any one of claims 1 to 5, wherein the acquiring the image of the to-be-detected area of the vehicle comprises: An image of at least one viewing angle of the interior of the vehicle captured by at least one camera disposed in the vehicle is acquired. 7 . The alarm prompting method according to claim 6 , wherein, in the case of acquiring images from multiple viewing angles of the vehicle collected by multiple cameras disposed on the vehicle, the detection based on the images is performed. 8 . The passengers of the vehicle described above include: Detect the riding object of the vehicle based on the images from the multiple perspectives, and fuse the detection results of the images from the multiple perspectives; or The images of the multiple viewing angles are fused, and the riding object of the vehicle is detected based on the fused images. 8. The alarm prompting method according to any one of claims 1 to 7, wherein the to-be-detected area comprises: a seat area and a vehicle rear area; The detecting the riding object of the vehicle based on the image includes: Detecting a ride-on object in the vehicle based on the image of the seating area; In a case where the occupant of the vehicle includes a child occupant or a pet, an adult occupant of the vehicle is detected based on the image of the seat area and the image of the rear area of the vehicle. 9 . The alarm prompting method according to claim 8 , wherein the detecting an adult occupant of the vehicle based on the image of the seat area and the image of the rear area of the vehicle comprises: 10 . In the case where an adult is detected near the trunk at the rear of the vehicle based on the image of the rear area of the vehicle and the obtained state of the trunk is an open state, it is determined that the riding object of the vehicle includes an adult occupant ;or In the case where it is detected based on the image of the rear area of the vehicle that the trunk of the rear of the vehicle is open and there are adults carrying items near the trunk of the rear of the vehicle, it is determined that the riding object of the vehicle includes an adult occupant . 10. The alarm prompting method according to any one of claims 1 to 9, wherein the detecting a riding object of the vehicle based on the image comprises: Perform face detection based on the image to determine candidate objects in the image; Based on the posture features and/or facial features of the candidate object extracted from the image, the type of the riding object of the vehicle is determined, and the type is one of a child, a pet, and an adult. 11. An alarm prompting device based on vehicle object detection, characterized in that the alarm prompting device comprises: The image acquisition module is used to acquire the image of the to-be-detected area of the vehicle when the vehicle is parked; an object detection module for detecting the riding object of the vehicle based on the image; The prompt generating module is configured to generate an alarm prompt when it is determined according to the detection result of the riding object that the riding object of the vehicle includes a child occupant or a pet and does not include an adult occupant. 12. An electronic device, characterized in that, comprising: processor; memory for storing processor-executable instructions; Wherein, the processor is configured to invoke the instructions stored in the memory to execute the alarm prompting method according to any one of claims 1 to 10. 13 . A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the alarm prompting method according to any one of claims 1 to 10 when the computer program instructions are executed by a processor. 14 .

Images