CN118864793A - A method, device, computer equipment and storage medium for collecting vehicle cabin images - Google Patents

Abstract

The present application provides a method, device, computer equipment and storage medium for collecting cabin images, wherein the cabin image is collected in real time by a driver monitoring camera, and whether there is a face in the current cabin image is detected; if there is a face in the current cabin image, face recognition and key point positioning are performed on the current cabin image to obtain the key points of the face in the current cabin image and the occlusion status of each key point; the photometric area of the current cabin image is determined according to the occlusion status of each key point of the face in the current cabin image; the target adjustment direction and target adjustment amount of the exposure time of the driver monitoring camera are determined according to the pixel grayscale of the photometric area of the current cabin image; the exposure time of the driver monitoring camera is adjusted according to the target adjustment direction and target adjustment amount; and the cabin image of the next moment is collected by the driver monitoring camera with the adjusted exposure time. The above method is used to improve the clarity of the collected cabin image.

Description

A method, device, computer equipment and storage medium for collecting vehicle cabin images

Technical Field

The present invention relates to the technical field of intelligent cockpits, and in particular to a method, device, computer equipment and storage medium for acquiring cabin images.

Background Art

A near-infrared (NIR) driver monitoring camera is a camera specifically designed to monitor the driver's status (Driver Monitoring System/DMS) inside a vehicle, which uses the near-infrared spectrum to acquire images. This technology is able to provide clear images in a variety of lighting conditions, including nighttime or backlit environments, and is therefore essential to ensuring the driver's attention and safety.

In the prior art, when using a near-infrared driver monitoring camera to collect images of the vehicle cabin, the near-infrared driver monitoring camera usually collects images of the vehicle cabin based on a pre-set exposure time. However, it is found in research that due to different driving environments of the vehicle, the lighting in the vehicle cabin may change, with high brightness, low brightness, or uneven brightness, which may cause the image collected after the image collection of the vehicle cabin based on the pre-set exposure time to be over-exposed, under-exposed, or unevenly exposed, thereby failing to obtain a clear and effective image of the vehicle cabin, or causing the clarity of the collected image of the vehicle cabin to be reduced.

Summary of the invention

In view of this, an object of the present invention is to provide a method, an apparatus, a computer device and a storage medium for acquiring a vehicle cabin image, so as to improve the clarity of the acquired vehicle cabin image.

In a first aspect, an embodiment of the present application provides a method for acquiring a cabin image, the method comprising:

The driver monitoring camera collects the cabin image in real time and detects whether there is a human face in the current cabin image;

If there is a face in the current cabin image, face recognition and key point positioning are performed on the current cabin image to obtain key points of the face in the current cabin image and the occlusion status of each key point;

Determine the photometric area of the current cabin image according to the occlusion status of each key point of the face in the current cabin image;

Determining a target adjustment direction and a target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image;

adjusting the exposure time of the driver monitoring camera according to the target adjustment direction and the target adjustment amount;

The vehicle cabin image at the next moment is collected by the driver monitoring camera with the adjusted exposure time.

Optionally, the performing face recognition and key point positioning on the current cabin image to obtain key points of the face in the current cabin image and the occlusion status of each key point includes:

Input the current cabin image into the face recognition model to obtain a face bounding box containing a face in the current cabin image;

The face area included in the face circumscribed frame is input into the face key point positioning model to obtain the key points of the face in the current cabin image and the occlusion status of each key point.

Optionally, determining the photometric area of the current cabin image according to the occlusion state of each key point of the face in the current cabin image includes:

Determining the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image;

Determining the occlusion state of the mouth according to the occlusion states of each mouth periphery key point indicating the mouth and the lower jaw in the current cabin image;

A photometric area of the current cabin image is determined according to the occlusion state of the eyes and the occlusion state of the mouth.

Optionally, determining the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image includes:

Determine whether the ratio of the number of occluded eye key points whose occlusion state is occluded in each eye key point to the first total number of eye key points exceeds a first preset threshold;

If the ratio of the number of the eye-blocking key points to the first total number exceeds the first preset threshold, determining that the eye is blocked;

If the ratio of the number of the blocked eye key points to the first total number does not exceed the first preset threshold, it is determined that the blockage state of the eye is not blocked.

Optionally, determining the occlusion state of the mouth according to the occlusion states of key points around the mouth indicating the mouth and the jaw in the current cabin image includes:

Determine whether the ratio of the number of occluded mouth periphery key points in the occlusion state of each mouth periphery key point to the second total number of mouth periphery key points exceeds a second preset threshold;

If the ratio of the number of the key points around the blocked mouth to the second total number exceeds the second preset threshold, determining that the blocking state of the mouth is blocked;

If the ratio of the number of the blocked mouth periphery key points to the second total number does not exceed the second preset threshold, it is determined that the blocking state of the mouth is not blocked.

Optionally, determining the photometric area of the current cabin image according to the occlusion state of the eyes and the occlusion state of the mouth includes:

If the occlusion state of the eyes is occluded but the occlusion state of the mouth is not occluded, the light metering area is determined to be the mouth area;

If the occlusion state of the eye is unoccluded but the occlusion state of the mouth is blocked, the light metering area is determined to be the eye area;

If the occlusion states of the eyes and the mouth are both blocked, or if the occlusion states of the eyes and the mouth are both unblocked, the light metering area is determined as a face area.

Optionally, determining the target adjustment direction and target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image includes:

Pre-configure the first parameter r and the second parameter gray_theshold;

Count the percentiles of the pixel grayscales within the rectangular area of the metering area to obtain percentile statistics results;

Determine whether P(r) in the percentile statistics exceeds the second parameter gray_theshold, where P(r) is the value of the rth percentile;

If P(r) in the percentile statistics exceeds the second parameter gray_theshold, the target adjustment direction is determined to be a decrease;

If P(r) in the percentile statistics result does not exceed the second parameter gray_theshold, determining the target adjustment direction to be increasing;

The target adjustment amount is determined based on a PID control algorithm.

In a second aspect, an embodiment of the present application provides a cabin image acquisition device, the device comprising:

An image detection module is used to collect cabin images in real time through a driver monitoring camera and detect whether there is a human face in the current cabin image;

The occlusion state determination module is used to perform face recognition and key point positioning on the current cabin image to obtain key points of the face in the current cabin image and the occlusion state of each key point if there is a face in the current cabin image;

A photometric area determination module, used to determine the photometric area of the current cabin image according to the occlusion status of each key point of the face in the current cabin image;

An adjustment strategy determination module, used to determine a target adjustment direction and a target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image;

an exposure time adjustment module, configured to adjust the exposure time of the driver monitoring camera according to the target adjustment direction and the target adjustment amount;

The image acquisition module is used to acquire the vehicle cabin image at the next moment through the driver monitoring camera with the adjusted exposure time.

Optionally, the performing face recognition and key point positioning on the current cabin image to obtain key points of the face in the current cabin image and the occlusion status of each key point includes:

Input the current cabin image into the face recognition model to obtain a face bounding box containing a face in the current cabin image;

The face area included in the face circumscribed frame is input into the face key point positioning model to obtain the key points of the face in the current cabin image and the occlusion status of each key point.

Optionally, determining the photometric area of the current cabin image according to the occlusion state of each key point of the face in the current cabin image includes:

Determining the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image;

Determining the occlusion state of the mouth according to the occlusion states of each mouth periphery key point indicating the mouth and the lower jaw in the current cabin image;

A photometric area of the current cabin image is determined according to the occlusion state of the eyes and the occlusion state of the mouth.

Optionally, determining the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image includes:

Determine whether the ratio of the number of occluded eye key points whose occlusion state is occluded in each eye key point to the first total number of eye key points exceeds a first preset threshold;

If the ratio of the number of the eye-blocking key points to the first total number exceeds the first preset threshold, determining that the eye is blocked;

If the ratio of the number of the blocked eye key points to the first total number does not exceed the first preset threshold, it is determined that the blockage state of the eye is not blocked.

Optionally, determining the occlusion state of the mouth according to the occlusion states of key points around the mouth indicating the mouth and the jaw in the current cabin image includes:

Determine whether the ratio of the number of occluded mouth periphery key points in the occlusion state of each mouth periphery key point to the second total number of mouth periphery key points exceeds a second preset threshold;

If the ratio of the number of the key points around the blocked mouth to the second total number exceeds the second preset threshold, determining that the blocking state of the mouth is blocked;

If the ratio of the number of the blocked mouth periphery key points to the second total number does not exceed the second preset threshold, it is determined that the blocking state of the mouth is not blocked.

Optionally, determining the photometric area of the current cabin image according to the occlusion state of the eyes and the occlusion state of the mouth includes:

If the occlusion state of the eyes is occluded but the occlusion state of the mouth is not occluded, the light metering area is determined to be the mouth area;

If the occlusion state of the eye is unoccluded but the occlusion state of the mouth is blocked, the light metering area is determined to be the eye area;

If the occlusion states of the eyes and the mouth are both blocked, or if the occlusion states of the eyes and the mouth are both unblocked, the light metering area is determined as a face area.

Optionally, determining the target adjustment direction and target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image includes:

Pre-configure the first parameter r and the second parameter gray_theshold;

Count the percentiles of the pixel grayscales within the rectangular area of the metering area to obtain percentile statistics results;

Determine whether P(r) in the percentile statistics exceeds the second parameter gray_theshold, where P(r) is the value of the rth percentile;

If P(r) in the percentile statistics exceeds the second parameter gray_theshold, the target adjustment direction is determined to be a decrease;

If P(r) in the percentile statistics result does not exceed the second parameter gray_theshold, determining the target adjustment direction to be increasing;

The target adjustment amount is determined based on a PID control algorithm.

In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the cabin image acquisition method described in any optional implementation manner of the first aspect are performed.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the cabin image acquisition method described in any optional implementation manner in the first aspect above are executed.

The technical solution provided by this application includes but is not limited to the following beneficial effects:

The present application uses a driver monitoring camera to collect cabin images in real time, detects whether there is a human face in the current cabin image, and if there is a human face in the current cabin image, performs face recognition and key point positioning on the current cabin image to obtain the key points of the face in the current cabin image and the occlusion status of each key point. The driver monitoring camera without exposure time adjustment can be used to collect the original cabin image for determining whether exposure time adjustment is required, and detects the key points of the face in the original cabin image and the occlusion status of each key point, providing a reference for determining the subsequent exposure time adjustment strategy.

Then, the present application determines the photometric area of the current cabin image according to the occlusion status of each key point of the human face in the current cabin image, determines the target adjustment direction and target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image, and can determine the image area used to adjust the exposure time, and determine the exposure adjustment strategy based on the grayscale data in the area.

Finally, the exposure time of the driver monitoring camera is adjusted according to the target adjustment direction and the target adjustment amount. The driver monitoring camera collects the cabin image at the next moment with the adjusted exposure time, which can achieve reasonable adjustment of the exposure time of the driver monitoring camera, thereby improving the clarity of the cabin image collected by the driver monitoring camera.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

FIG1 shows a flow chart of a method for collecting vehicle cabin images provided in Embodiment 1 of the present invention;

FIG2 shows a flow chart of a key point occlusion state detection method provided by Embodiment 1 of the present invention;

FIG3 shows a flow chart of a method for determining a light metering area provided in Embodiment 1 of the present invention;

FIG4 shows a flow chart of a method for detecting eye occlusion status provided by Embodiment 1 of the present invention;

FIG5 shows a flow chart of a method for detecting a mouth occlusion state provided by Embodiment 1 of the present invention;

FIG6 shows a flowchart of a specific method for determining a light metering area provided in Embodiment 1 of the present invention;

FIG7 shows a flow chart of a method for determining an exposure time adjustment strategy provided by Embodiment 1 of the present invention;

FIG8 shows a schematic structural diagram of a vehicle cabin image acquisition device provided by Embodiment 2 of the present invention;

FIG9 shows a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

Embodiment 1

To facilitate understanding of the present application, the first embodiment of the present application is described in detail below in conjunction with the contents of the flowchart of a cabin image acquisition method provided by the first embodiment of the present invention shown in FIG. 1 .

Referring to FIG. 1 , FIG. 1 shows a flow chart of a method for acquiring a vehicle cabin image according to a first embodiment of the present invention, wherein the method comprises steps S101 to S105:

S101: Collecting a vehicle cabin image in real time through a driver monitoring camera to detect whether there is a human face in the current vehicle cabin image.

Specifically, the cabin image collected in real time is input into the face detection model to obtain a detection result of whether there is a face in the current cabin image.

S102: If there is a human face in the current cabin image, perform face recognition and key point positioning on the current cabin image to obtain key points of the human face in the current cabin image and the occlusion status of each key point.

Specifically, if there is a face in the current cabin image, the current cabin image is input into the face recognition model to obtain a detection frame indicating the face, the area indicated by each detection frame is clipped to obtain a face area image, and the face area image is input into the key point positioning model to obtain the position information of each key point of the face in the current cabin image (standard 68 key points of the face) and the occlusion status of each key point.

S103: Determine a light metering area of the current cabin image according to the occlusion status of each key point of the human face in the current cabin image.

Specifically, the key points are grouped according to the areas indicated by the key points to obtain key point groups for various parts of the face. For example, key point 1 to key point 17 among the standard 68 key points of the face indicate the jaw, and the 17 key points of key point 1 to key point 17 are used as the jaw key point group; for another example, key point 49 to key point 68 among the standard 68 key points of the face indicate the mouth, and the 20 key points of key point 49 to key point 68 are used as the mouth key point group; for another example, key point 37 to key point 48 among the standard 68 key points of the face indicate the eyes, and the 12 key points of key point 37 to key point 48 are used as the eye key point group.

The occlusion state of the facial part corresponding to each key point group is determined according to the number of occluded key points in each key point group, and the photometric area of the current cabin image is determined according to the occlusion state of different facial parts.

S104: Determine a target adjustment direction and a target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the light metering area of the current cabin image.

Specifically, the grayscale value of each pixel in the photometric area of the current cabin image is counted (the value range is 0-255, 0 represents pure black, and 255 represents the brightest), and the target adjustment direction and target adjustment amount when adjusting the exposure time of the driver monitoring camera are determined according to the data distribution characteristics of the grayscale value of each pixel in the photometric area of the current cabin image. The target adjustment direction is increase or decrease, and the target adjustment amount is the time length.

S105: adjusting the exposure time of the driver monitoring camera according to the target adjustment direction and the target adjustment amount.

Specifically, the exposure time of the driver monitoring camera is adjusted from the initial amount when the current cabin image is collected in step S101, and the initial amount is adjusted by the target adjustment amount in the target adjustment direction to obtain the final amount.

S106: Capturing the vehicle cabin image at the next moment with the adjusted exposure time through the driver monitoring camera.

Specifically, the vehicle cabin image at the next moment is collected by the driver monitoring camera with the adjusted exposure time (corresponding to the final amount of exposure time in step S105 ), so as to realize automatic exposure of the driver monitoring camera.

In an optional implementation, referring to FIG. 2 , FIG. 2 shows a flow chart of a key point occlusion state detection method provided in Example 1 of the present invention, wherein the step of performing face recognition and key point positioning on the current cabin image to obtain key points of the face in the current cabin image and the occlusion state of each key point includes steps S201 to S202:

S201: Input the current cabin image into a face recognition model to obtain a face bounding box containing a face in the current cabin image.

S202: Inputting the face area included in the face circumscribed frame into a face key point positioning model to obtain key points of the face in the current cabin image and the occlusion status of each key point.

Specifically, the face recognition model and the face key point location model are obtained after pre-training using a face image dataset.

In an optional implementation, referring to FIG. 3 , FIG. 3 shows a flow chart of a method for determining a photometric area provided in Embodiment 1 of the present invention, wherein the method for determining the photometric area of the current cabin image according to the occlusion state of each key point of the face in the current cabin image comprises steps S301 to S303:

S301: Determine the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image.

Specifically, the eye key points indicating the eyes are key points 37 to 48 among the standard 68 key points of the face.

S302: Determine the occlusion state of the mouth according to the occlusion states of key points around the mouth indicating the mouth and the jaw in the current cabin image.

Specifically, the mouth key points indicating the mouth are key points 49 to 68 among the standard 68 key points of the face, and the mouth key points indicating the jaw are key points 1 to 17 among the standard 68 key points of the face.

S303: Determine a light metering area of the current cabin image according to the occlusion state of the eyes and the occlusion state of the mouth.

Specifically, the situation where a person's face is blocked usually occurs when wearing glasses or a mask. Wearing glasses will cause the eyes of the person to be blocked, and wearing a mask will cause the mouth of the person to be blocked. Therefore, when wearing glasses and a mask at the same time, both the eyes and the mouth will be blocked; when not wearing glasses and no mask, the eyes and the mouth will not be blocked; when wearing glasses but not wearing a mask, the eyes will be blocked but the mouth will not be blocked; when not wearing glasses but wearing a mask, the eyes will not be blocked but the mouth will be blocked. Based on the various possible situations where the face is blocked, the light metering area of the current cabin image is determined according to different combinations of eye blocking states and mouth blocking states.

In an optional implementation, referring to FIG. 4 , FIG. 4 shows a flow chart of an eye occlusion state detection method provided in Example 1 of the present invention, wherein the method of determining the eye occlusion state according to the occlusion state of each eye key point indicating the eye in the current cabin image comprises steps S401 to S403:

S401: Determine whether a ratio of the number of occluded eye key points whose occlusion state is occluded among the eye key points to a first total number of eye key points exceeds a first preset threshold.

Specifically, the number of occluded eye key points whose occlusion state is occluded is counted, the ratio of the number of occluded eye key points whose occlusion state is occluded to the first total number of eye key points is calculated, and the relationship between the ratio and the first preset threshold is determined. For example, if the number of occluded eye key points is 3 and the first total number of eye key points is 12, the ratio is calculated to be 25%.

S402: If the ratio of the number of the eye-blocking key points to the first total number exceeds the first preset threshold, it is determined that the eye is blocked.

Specifically, if the ratio of the number of occluded eye key points to the first total number exceeds a first preset threshold, it means that the eye is likely to be occluded, and the occlusion state of the eye is determined to be occluded. For example, when the first preset threshold is 20%, and the calculated ratio is 25%, it can be determined that the occlusion state of the eye is occluded.

S403: If the ratio of the number of the blocked eye key points to the first total number does not exceed the first preset threshold, determining that the blockage state of the eye is unblocked.

Specifically, if the ratio of the number of blocked eye key points to the first total number does not exceed the first preset threshold, it means that the eye is likely not blocked, and the occlusion state of the eye is determined to be not blocked. For example, when the first preset threshold is 30%, and the calculated ratio is 25%, it can be determined that the occlusion state of the eye is not blocked.

In an optional implementation, referring to FIG. 5 , FIG. 5 shows a flow chart of a method for detecting a mouth occlusion state provided in Example 1 of the present invention, wherein the method for determining the mouth occlusion state according to the occlusion state of each mouth periphery key point indicating the mouth and the lower jaw in the current cabin image comprises steps S501 to S503:

S501: Determine whether a ratio of the number of occluded mouth periphery key points in the occlusion state of being occluded among the mouth periphery key points to a second total number of mouth periphery key points exceeds a second preset threshold.

Specifically, the occluded mouth key points are key points indicating the mouth and the jaw, and the occluded mouth key points are the mouth key points and the jaw key points whose occlusion state is occluded. The number of the mouth key points and the jaw key points whose occlusion state is occluded is counted, and the ratio of the occluded mouth key points to the second total number of mouth key points is calculated, and the relationship between the ratio and the second preset threshold is determined. For example, if the number of occluded mouth key points is 10 and the second total number of mouth key points is 37, then the ratio is calculated to be about 27%.

S502: If the ratio of the number of the key points around the blocked mouth to the second total number exceeds the second preset threshold, it is determined that the blocking state of the mouth is blocked.

Specifically, if the ratio of the number of key points around the mouth that are blocked to the second total number exceeds the second preset threshold, it means that the mouth is likely to be blocked, and the occlusion state of the mouth is determined to be blocked. For example, when the second preset threshold is 20%, and the calculated ratio is 27%, it can be determined that the occlusion state of the mouth is blocked.

S503: If the ratio of the number of the blocked mouth periphery key points to the second total number does not exceed the second preset threshold, determining that the blocking state of the mouth is unblocked.

Specifically, if the ratio of the number of key points around the mouth that are blocked to the second total number does not exceed the second preset threshold, it means that the mouth is likely not blocked, and the blocking state of the mouth is determined to be not blocked. For example, when the second preset threshold is 95% and the calculated ratio is 27%, it can be determined that the blocking state of the mouth is not blocked.

In an optional implementation, referring to FIG. 6 , FIG. 6 shows a flowchart of a specific method for determining a photometric area provided in Example 1 of the present invention, wherein the photometric area of the current cabin image is determined according to the occlusion state of the eyes and the occlusion state of the mouth, including steps S601 to S603:

S601: If the occlusion state of the eyes is occluded but the occlusion state of the mouth is not occluded, the light measurement area is determined to be the mouth area.

Specifically, if the occlusion state of the eyes is blocked but the occlusion state of the mouth is unblocked, the unblocked mouth area is used as the metering area, or the lower half of the face is used as the metering area, or the lower two-thirds of the face is used as the metering area.

S602: If the occlusion state of the eyes is unoccluded but the occlusion state of the mouth is blocked, the light metering area is determined to be the eye area.

Specifically, if the occlusion state of the eyes is unoccluded but the occlusion state of the mouth is blocked, the unoccluded eye area is used as the metering area, or the upper half of the face is used as the metering area, or the upper third of the face is used as the metering area.

S603: If the occlusion states of the eyes and the mouth are both blocked, or if the occlusion states of the eyes and the mouth are both unblocked, determine the light metering area as a face area.

Specifically, if the occlusion states of the eyes and the mouth are both blocked, or the occlusion states of the eyes and the mouth are both unblocked, the entire face area is used as the light measurement area.

If there is no face in the current cabin image, the face area in the cabin image that was collected most recently within a short period of time (within 5 seconds) is used as the light measurement area. If there is no face in the cabin image that was collected within a short period of time (within 5 seconds), the preset area is used as the light measurement area.

In an optional implementation, referring to FIG. 7 , FIG. 7 shows a flow chart of a method for determining an exposure time adjustment strategy provided by Example 1 of the present invention, wherein the target adjustment direction and target adjustment amount of the exposure time of the driver monitoring camera are determined according to the pixel grayscale of the photometric area of the current cabin image, including steps S701 to S706:

S701: pre-configure a first parameter r and a second parameter gray_theshold.

S702: Count the percentiles of the grayscales of pixels within the light metering area rectangle to obtain percentile statistics results.

S703: Determine whether P(r) in the percentile statistics result exceeds the second parameter gray_theshold, where P(r) is the value of the rth percentile.

S704: If P(r) in the percentile statistics result exceeds the second parameter gray_theshold, the target adjustment direction is determined to be decreasing.

S705: If P(r) in the percentile statistics result does not exceed the second parameter gray_theshold, the target adjustment direction is determined to be increasing.

Specifically, two parameters r and gray_theshold are preset. If the percentile statistical result P(r) is greater than gray_theshold, the exposure time is reduced by setting the register sensor; otherwise, the exposure time is increased (the driver monitoring camera register sensor provides an interface function for adjusting the exposure time, which can be called to adjust the exposure time).

S706: Determine the target adjustment amount based on the PID control algorithm.

Specifically, a PID control algorithm is used to calculate the target adjustment amount Δu(k) that needs to be adjusted, where Δu(k)= _kp *( _ek - _ek-1 )+ _ki * _ek + _kd *(ek- _{2ek -1} + _ek-2 ), where _kp is the kth proportional control coefficient, _ki is the kth integral control coefficient, _kd is the kth differential control coefficient, and _ek is the kth deviation; the above-mentioned proportional coefficient, integral coefficient, and differential coefficient are set according to different exposure time segments. By using different coefficients in different exposure time intervals, better control performance can be obtained, and the stability and convergence speed can be improved. The exposure time intervals are set to 0-100us, 100-200us, 200-400us, 400-800us, and above 800us.

Embodiment 2

Referring to FIG. 8 , FIG. 8 shows a schematic diagram of the structure of a vehicle cabin image acquisition device provided in Embodiment 2 of the present invention, wherein the device comprises:

The image detection module 801 is used to collect the cabin image in real time through the driver monitoring camera and detect whether there is a human face in the current cabin image;

The occlusion state determination module 802 is used to perform face recognition and key point positioning on the current cabin image to obtain key points of the face in the current cabin image and the occlusion state of each key point if there is a face in the current cabin image;

A photometric region determination module 803 is used to determine the photometric region of the current cabin image according to the occlusion status of each key point of the face in the current cabin image;

An adjustment strategy determination module 804, configured to determine a target adjustment direction and a target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image;

An exposure time adjustment module 805, configured to adjust the exposure time of the driver monitoring camera according to the target adjustment direction and the target adjustment amount;

The image acquisition module 806 is used to acquire the cabin image at the next moment through the driver monitoring camera with the adjusted exposure time.

In an optional implementation, the step of performing face recognition and key point positioning on the current cabin image to obtain key points of the face in the current cabin image and the occlusion status of each key point includes:

Input the current cabin image into the face recognition model to obtain a face bounding box containing a face in the current cabin image;

The face area included in the face circumscribed frame is input into the face key point positioning model to obtain the key points of the face in the current cabin image and the occlusion status of each key point.

In an optional implementation manner, determining the photometric area of the current cabin image according to the occlusion state of each key point of the face in the current cabin image includes:

Determining the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image;

Determining the occlusion state of the mouth according to the occlusion states of key points around the mouth indicating the mouth and the jaw in the current cabin image;

A photometric area of the current cabin image is determined according to the occlusion state of the eyes and the occlusion state of the mouth.

In an optional implementation, determining the occlusion state of the eye according to the occlusion state of each eye key point indicating the eye in the current cabin image includes:

Determine whether the ratio of the number of occluded eye key points whose occlusion state is occluded in each eye key point to the first total number of eye key points exceeds a first preset threshold;

If the ratio of the number of the eye-blocking key points to the first total number exceeds the first preset threshold, determining that the eye is blocked;

If the ratio of the number of the blocked eye key points to the first total number does not exceed the first preset threshold, it is determined that the blockage state of the eye is not blocked.

In an optional implementation, the determining the occlusion state of the mouth according to the occlusion states of each mouth periphery key point indicating the mouth and the lower jaw in the current cabin image includes:

Determine whether the ratio of the number of occluded mouth periphery key points in the occlusion state of each mouth periphery key point to the second total number of mouth periphery key points exceeds a second preset threshold;

If the ratio of the number of the key points around the blocked mouth to the second total number exceeds the second preset threshold, determining that the blocking state of the mouth is blocked;

If the ratio of the number of the blocked mouth periphery key points to the second total number does not exceed the second preset threshold, it is determined that the blocking state of the mouth is not blocked.

In an optional implementation, determining the photometric area of the current cabin image according to the occlusion state of the eyes and the occlusion state of the mouth includes:

If the occlusion state of the eyes is occluded but the occlusion state of the mouth is not occluded, the light metering area is determined to be the mouth area;

If the occlusion state of the eye is unoccluded but the occlusion state of the mouth is blocked, the light metering area is determined to be the eye area;

If the occlusion states of the eyes and the mouth are both blocked, or if the occlusion states of the eyes and the mouth are both unblocked, the light metering area is determined as a face area.

In an optional implementation, determining the target adjustment direction and target adjustment amount of the exposure time of the driver monitoring camera according to the pixel grayscale of the photometric area of the current cabin image includes:

Pre-configure the first parameter r and the second parameter gray_theshold;

Count the percentiles of the pixel grayscales within the rectangular area of the metering area to obtain percentile statistics results;

Determine whether P(r) in the percentile statistics exceeds the second parameter gray_theshold, where P(r) is the value of the rth percentile;

If P(r) in the percentile statistics exceeds the second parameter gray_theshold, the target adjustment direction is determined to be a decrease;

If P(r) in the percentile statistics result does not exceed the second parameter gray_theshold, determining the target adjustment direction to be increasing;

The target adjustment amount is determined based on a PID control algorithm.

Embodiment 3

Based on the same application concept, referring to FIG. 9 , FIG. 9 shows a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. As shown in FIG. 9 , a computer device 900 provided in Embodiment 3 of the present application includes:

A processor 901, a memory 902 and a bus 903, wherein the memory 902 stores machine-readable instructions executable by the processor 901. When the computer device 900 is running, the processor 901 communicates with the memory 902 via the bus 903. When the processor 901 is running, the machine-readable instructions execute the steps of the cabin image acquisition method shown in the above-mentioned embodiment 1.

Embodiment 4

Based on the same application concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cabin image acquisition method described in any one of the above embodiments are executed.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

The computer program product for training a face recognition model provided in an embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

The face recognition model training device provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for the parts not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features thereof; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A vehicle cabin image acquisition method, the method comprising:

acquiring a car cabin image in real time through a driver monitoring camera, and detecting whether a face exists in the current car cabin image;

If the face exists in the current car cabin image, carrying out face recognition and key point positioning on the current car cabin image to obtain key points of the face in the current car cabin image and shielding states of the key points;

Determining a light metering area of the current car cabin image according to the shielding state of each key point of the face in the current car cabin image;

determining a target adjustment direction and a target adjustment amount of the exposure time of the driver monitoring camera according to the pixel gray level of the light measuring area of the current cabin image;

adjusting the exposure time of the driver monitoring camera according to the target adjustment direction and the target adjustment amount;

and acquiring a cabin image at the next moment by the driver monitoring camera according to the adjusted exposure time.

2. The method according to claim 1, wherein the performing face recognition and key point positioning on the current cabin image to obtain each key point of the face in the current cabin image and a shielding state of each key point comprises:

inputting the current car cabin image into a face recognition model to obtain a face external frame containing a face in the current car cabin image;

And inputting the face region contained in the face external frame into a face key point positioning model to obtain each key point of the face in the current car cabin image and the shielding state of each key point.

3. The method according to claim 1, wherein the determining the photometric area of the current car image according to the occlusion state of each key point of the face in the current car image includes:

Determining the shielding state of the eyes according to the shielding state of each eye key point of the indication eyes in the current car cabin image;

determining the shielding state of the mouth according to the shielding state of key points around the mouth, which indicate the mouth and the chin, in the current car cabin image;

And determining a photometric area of the current car cabin image according to the shielding state of the eyes and the shielding state of the mouth.

4. The method of claim 3, wherein the determining the occlusion status of the eye based on the occlusion status of each eye keypoint in the current vehicle cabin image indicative of the eye comprises:

Judging whether the ratio of the number of the shielded eye key points to the first total number of the eye key points, which are in the shielding state of each eye key point, exceeds a first preset threshold value;

if the ratio of the number of the key points of the shielded eyes to the first total number exceeds the first preset threshold, determining that the shielding state of the eyes is shielded;

And if the ratio of the number of the key points of the shielded eyes to the first total number does not exceed the first preset threshold, determining that the shielding state of the eyes is non-shielding.

5. A method according to claim 3, wherein said determining the occlusion status of the mouth based on the occlusion status of the respective peri-mouth keypoints in the current cabin image indicative of the mouth and chin comprises:

Judging whether the ratio of the number of the shielded peripheral key points to the second total number of the peripheral key points exceeds a second preset threshold value or not, wherein the shielding state of the peripheral key points is the shielded peripheral key points;

if the ratio of the number of the key points around the shielding mouth to the second total number exceeds the second preset threshold, determining that the shielding state of the mouth is shielded;

And if the ratio of the number of the key points around the shielding mouth to the second total number does not exceed the second preset threshold value, determining that the shielding state of the mouth is not shielding.

6. The method of claim 3, wherein the determining the photometric area of the current cabin image based on the occlusion status of the eyes and the occlusion status of the mouth comprises:

If the shielding state of the eyes is shielded but the shielding state of the mouth is not shielded, the photometric area is determined to be a mouth area;

if the shielding state of the eyes is not shielded but the shielding state of the mouth is shielded, the photometric area is determined to be an eye area;

and if the shielding states of the eyes and the mouth are shielded or the shielding states of the eyes and the mouth are not shielded, determining the photometry area as a face area.

7. The method according to claim 1, wherein the determining the target adjustment direction and the target adjustment amount of the exposure time of the driver monitor camera according to the pixel gray scale of the light metering area of the current cabin image includes:

Pre-configuring a first parameter r and a second parameter gray_ theshold;

Counting the percentile of the pixel gray scales in the rectangle of the light measuring area to obtain a percentile counting result;

Judging whether P (r) in the percentile statistical result exceeds the second parameter gray_ theshold, wherein P (r) is the value of the r-th percentile;

If P (r) in the percentile statistics exceeds the second parameter gray_ theshold, determining the target adjustment direction to be reduced;

If P (r) in the percentile statistics does not exceed the second parameter gray_ theshold, determining the target adjustment direction as increasing;

and determining the target adjustment amount based on a PID control algorithm.

8. A cabin image acquisition device, the device comprising:

The image detection module is used for acquiring the car cabin image in real time through the driver monitoring camera and detecting whether a face exists in the current car cabin image;

the shielding state determining module is used for carrying out face recognition and key point positioning on the current car cabin image to obtain each key point of the face and shielding state of each key point in the current car cabin image if the face exists in the current car cabin image;

The light measurement area determining module is used for determining the light measurement area of the current car cabin image according to the shielding state of each key point of the face in the current car cabin image;

the adjustment strategy determining module is used for determining a target adjustment direction and a target adjustment amount of the exposure time of the driver monitoring camera according to the pixel gray level of the light measuring area of the current vehicle cabin image;

The exposure time adjusting module is used for adjusting the exposure time of the driver monitoring camera according to the target adjusting direction and the target adjusting amount;

and the image acquisition module is used for acquiring the vehicle cabin image at the next moment through the driver monitoring camera according to the adjusted exposure time.

9. A computer device, comprising: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory in communication via the bus when the computer device is running, the machine readable instructions when executed by the processor performing the steps of the cabin image acquisition method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the cabin image acquisition method according to any one of claims 1 to 7.