CN114120484A - Face recognition system and vehicle - Google Patents

Abstract

The invention provides a face recognition system and a vehicle, which solve the problem that the existing 3D camera equipment is too large in size and inconvenient to apply, and comprise a 3D module and a controller module, wherein the 3D module is connected with the controller module, and the controller module comprises: an information calculation unit for acquiring an image of the 3D module and calculating data; the face recognition unit is used for judging whether the user is a legal user or not according to the data; the control unit is used for controlling the external terminal according to the legal user information; the information calculation unit, the face recognition unit, and the control unit are provided on the same chip.

Description

Face recognition system and vehicle

Technical Field

The invention relates to the field of face recognition, in particular to a face recognition system and a vehicle.

Background

The existing 3D camera device needs to process data calculation and image processing, so an additional high-performance processor chip and a peripheral circuit thereof need to be added in the 3D device to implement the functions of a 3D information calculation unit and 3D information transmission, and the like, that is, the 3D module includes a "3D information calculation unit" and a "3D information transmission submodule", so that the 3D camera device itself has a large volume and cannot be installed in a vehicle door or a vehicle B-pillar and the like.

Disclosure of Invention

The invention aims to solve the problem that the existing 3D camera equipment is too large in size and inconvenient to apply.

In order to achieve the above object, the present invention provides a face recognition system, including a 3D module and a controller module, wherein the 3D module is connected to the controller module, and the controller module includes: an information calculation unit for acquiring an image of the 3D module and calculating data; the face recognition unit is used for judging whether the user is a legal user or not according to the data; and the control unit is used for controlling an external terminal according to the legal user information, and the information calculation unit, the face recognition unit and the control unit are arranged on the same chip.

Preferably, an input end of the face recognition unit is connected with an output end of the information calculation unit, and an output end of the face recognition unit is connected with an input end of the control unit.

Preferably, the 3D module includes an image sensor unit for acquiring an image and outputting an image signal.

Preferably, the information calculating unit is configured to receive the image signal output by the image sensor unit and acquire an original data frame, and if the original data of a preset frame number is not received within a preset time, wait for receiving the original data of the preset frame number and then receive the temperature data; and if the original data with the preset frame number is received within the preset time, directly receiving the temperature data.

Preferably, the information calculation unit alternately receives the raw data and the temperature data for the preset number of frames.

Preferably, the controller module further includes a serializer unit, and the 3D module further includes a deserializer module, and an output end of the serializer unit is connected to an input end of the deserializer module.

Preferably, the calculation data comprises: image data for identifying the person who is the principal, and depth data for calculating whether the person is a living body.

Preferably, if the face recognition unit determines that the user is a valid user, the face recognition unit sends information of the valid user to the control unit.

Preferably, if the face recognition unit determines that the user is an illegal user, the 3D module acquires an image again after sending information of the illegal user to the control unit to obtain a rejection alert.

The invention also provides a vehicle comprising any one of the face recognition systems described above.

The scheme adopted by the patent is used for carrying out depth customization development on the 3D module, and removing a 3D information calculation unit and a 3D information sending submodule in the 3D module, so that the volume of the 3D module is effectively reduced, and the 3D module is convenient to install at the positions of a vehicle door or a vehicle B column and the like; meanwhile, the heat generation of the 3D module is reduced, and the precision of the depth data of the dimensionality of the 3D module is ensured.

Drawings

FIG. 1 is a diagram of a controller module of an embodiment of a facial recognition system of the present invention;

FIG. 2 is a schematic diagram of face recognition in an embodiment of a face recognition system of the present invention;

FIG. 3 is a flow chart of the operation of the control module of one embodiment of the facial recognition system of the present invention;

FIG. 4 is a flow chart of an information computation unit suppressing temperature drift for an embodiment of a face recognition system of the present invention;

FIG. 5 is a diagram of raw data frame packing for an embodiment of a face recognition system of the present invention;

fig. 6 is a diagram of interaction between a face recognition system and a whole vehicle according to an embodiment of the face recognition system of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative only and should not be construed as limiting the invention.

In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments, it is to be understood that the specific embodiments described herein are only for the purpose of explaining the present invention, and are not to be used as limitations of the present invention.

As shown in fig. 1 and 2, the system includes a 3D module and a controller module, wherein the 3D module is connected to the controller module, and the controller module includes: an information calculation unit for acquiring an image of the 3D module and calculating data; a face recognition unit for judging whether the user is a legal user according to the data; and the control unit is used for controlling the external terminal according to the legal user information, wherein the information calculation unit, the face recognition unit and the control unit are arranged on the same chip, the input end of the face recognition unit is connected with the output end of the information calculation unit, and the output end of the face recognition unit is connected with the input end of the control unit.

Specifically, the invention relates to a vehicle-mounted face recognition system, which comprises two parts: the first component is a 3D module, the second component is a controller module, and the 3D module and the controller are connected by LVDS signal lines.

The scheme adopted by the invention is used for carrying out depth customized development on the 3D module, and removing the 'information calculation unit' and the 'information sending submodule' in the 3D module, thereby effectively reducing the volume of the 3D module and ensuring that the 3D module is convenient to install at the positions of a vehicle door or a vehicle B column and the like; meanwhile, the heating of the 3D module is reduced, and the precision of the depth data of the dimensionality of the 3D module is ensured, namely, the prior patented technology needs two independent chips to finish image transmission and face recognition, and the two chips are arranged in two different modules, so that the volume of the 3D module is larger.

In an embodiment, the 3D module includes an image sensor unit for acquiring an image and outputting an image signal. The controller module further comprises a serializer unit, the 3D module further comprises a deserializer unit, and the output end of the serializer unit is connected with the input end of the deserializer unit.

Specifically, for the 3D module, the inside of the module mainly includes two sub-units, which are an image sensor unit for acquiring an image and outputting an image signal and a serializer unit, respectively, where the serializer is an LVDS serializer, and a deserializer in the same deserializer unit is an LVDS deserializer.

In a specific implementation principle, the LVDS serializer inside the 3D module is responsible for converting parallel signals output by the image sensor into serial differential signals for long-distance transmission. It should be noted that the LVDS serializer must be used in cooperation with the LVDS deserializer at the rear end, that is, the LVDS deserializer converts the serial differential signal transmitted from a long distance into a parallel signal, and finally delivers the parallel signal to the high-performance processor chip in the vehicle-mounted controller module for subsequent processing.

And connecting the 3D module developed by depth customization to a controller module at the back end through LVDS signal lines. The vehicle-mounted controller module performs operations such as power-on initialization, triggering of original data frame acquisition, suppression and adjustment of a temperature drift phenomenon and the like on the 3D module through I2C data transmitted in the LVDS signals. The 3D module only executes the acquisition process of the original data frame, and does not have the information calculation and information transmission process, if the acquisition process is in a USB (universal serial bus), Ethernet or Wifi (wireless fidelity) mode, an operating system does not need to be operated, so that the power-on starting speed of the 3D module is greatly increased, and the user experience is also greatly improved.

In a specific implementation principle, the 3D module actively emits light with a certain wavelength, for example, 940nm infrared light has few components in natural light, the light is reflected after striking the surface of an object, then a 3D image sensor in the 3D module receives the reflected light, and converts received light information into voltage information through a photoelectric effect, and finally generates data of an "original data frame". It should be noted that, the most commonly used 2D camera device in daily life is different from the 3D module, and the 2D camera device does not actively emit light with a specific wavelength to the outside, but passively receives all visible light reflected by the surface of an object.

As shown in fig. 2, the signal line extension scheme of the 3D image sensor may regard 3 parts of the LVDS serializer, the LVDS transmission harness, the LVDS deserializer, and the like in a dashed line frame in the figure as a black box, and then an input signal and an output signal of the black box are identical, so the black box may be equivalent to a "bundle of signal extension lines". Based on the principle, the vehicle-mounted face recognition system omits an information sending submodule and an information receiving submodule in the prior art. The link of a data channel is simplified, the delay time of data transmission is shortened, and the performance of the whole vehicle-mounted face recognition system is favorably improved.

The invention relates to a vehicle-mounted face recognition system, wherein a vehicle-mounted controller module mainly comprises: LVDS deserializers and high performance processor chips. As mentioned above, the LVDS deserializer and the LVDS serializer located in the 3D module realize "extension" of the signal line of the 3D image sensor and introduction onto the high performance processor chip in the on-board controller module. The high-performance processor chip has strong computing power and is responsible for realizing functions of an information calculation unit, a face recognition unit, a vehicle control unit and the like.

In one embodiment, if the face recognition unit judges that the user is a legal user, the face recognition unit sends information of the legal user to the control unit; if the face recognition unit judges that the user is an illegal user, sending information of the illegal user to the control unit, acquiring an refusal warning, and then the 3D module acquires an image again, wherein the information of the illegal user comprises: unregistered user information or a non-living body.

Specifically, the feedback result information is used to determine whether the face shot by the 3D module is a registered valid user. If the 'face recognition unit' in the vehicle-mounted controller module judges that the face shot by the 3D module is a registered legal user, the 'control unit' running on the high-performance processor chip interacts with a BCM (vehicle body control module) through a CAN (controller area network) bus, and finally the BCM module executes the operations of unlocking a vehicle door, starting a transmitter and the like, and if the 'face recognition unit' in the controller module judges that the face shot by the 3D module is not a legal user, possibly is not registered, and possibly is a non-living body, such as a 3D head model or a human skin mask, and the like, the 'control unit' running on the high-performance processor chip adopts an HMI (human-machine interaction) mode to reject reminding, such as voice prompt, visual prompt, alarm and the like.

In an embodiment, acquiring images of a 3D module and computing data comprises: image data for identifying the person who is the principal, and depth data for calculating whether the person is a living body.

Specifically, the face information refers to a data structure obtained by acquiring an image of a 3D module and performing calculation, and mainly includes two types of data: infrared image data and depth data characterizing the 3 rd dimension. It should be noted that, if there is no human face in the current 3D module shooting range, the data structure of "human face information" is generated, but it represents the actual shot object rather than the human face.

In a specific implementation, the controller module workflow is as shown in FIG. 3. The following is a detailed description of the various steps in the workflow:

step S01, which indicates that the "information computing unit" in the vehicle-mounted controller module performs power-on initialization on the 3D module through the I2C data transmitted in the LVDS signal. As previously described, the use of an LVDS serializer and an LVDS deserializer in combination also enables the I2C bus signal of the image sensor to be extended onto a high performance processor chip in the vehicle controller module. The process of 3D module initialization is essentially that the I2C control register and status register on the 3D image sensor are operated by the "information calculation unit".

Step S02, it indicates that the "information computing unit" in the vehicle-mounted controller module triggers the image sensor through the GPIO signal transmitted in the LVDS signal, and after triggering, the 3D module will emit a light pulse with a certain wavelength, for example, 940nm infrared light, which has a good effect of resisting the interference of sunlight, and at the same time, the 3D image sensor in the 3D module starts to receive the reflected light and outputs a digital voltage signal through photoelectric conversion, that is, data of a so-called "original data frame" is generated. The data of the original data frame is output through a parallel port on the image sensor and then reaches the parallel port of the high-performance processor chip in the vehicle-mounted controller module by means of an extended line formed by the LVDS serializer and the LVDS deserializer.

Alternatively, in step S02, the image sensor may be triggered by the "information computing unit" in the vehicle-mounted controller module through the I2C data transmitted in the LVDS signal. The effect is equal to that the image sensor is triggered by the GPIO signals transmitted in the LVDS signals.

Step S03, the "information calculating unit" in the controller module starts to perform operations and processing on the data of all the "original data frames" generated by the trigger operation 1 time after receiving the data of all the "original data frames" to obtain the face information. The face information is a data structure obtained by calculating data of an original data frame, and mainly comprises two types of data: infrared image data and depth data characterizing the 3 rd dimension. It should be noted that if no human face exists in the current 3D module shooting range, the data structure of "human face information" is generated, but it represents an actual shot object instead of a human face.

Step S04, which indicates that the "face recognition unit" in the controller module performs further calculation, i.e., an algorithm for face recognition, based on the face information generated in step S03. The 3D face recognition algorithm is the key of the whole vehicle-mounted face recognition system, and it must ensure a very high correct recognition rate, e.g. 99.99%, and a low error recognition rate to a limit value, e.g. one millionth, and at the same time be able to resist 3D head models and human masks, etc., so as to be able to be applied in the scenes of brushing the face to open the vehicle door, allowing the engine to start, etc. The execution of the face recognition generally comprises a series of operations such as living body detection, face detection and tracking, face feature extraction and comparison, and the like, and after the execution of the face recognition is finished, result information is fed back.

Step S05, indicating that the "face recognition unit" in the vehicle-mounted controller module judges whether the face currently photographed by the 3D module is a registered valid user according to the result information fed back in step S03.

Step S06, indicating that the "face recognition unit" in the onboard controller module has already determined that the face currently captured by the 3D module is a registered valid user, the "vehicle control unit" running on the high performance processor chip interacts with the BCM module (body control module) through the CAN bus, and finally the BCM module executes operations such as unlocking the vehicle door, allowing the transmitter to start.

Step S07, which means that the "face recognition unit" in the vehicle-mounted controller module determines that the face currently captured by the 3D module is not a valid user, may not be registered, and may also be a non-living body, such as a 3D head model or a human skin mask, etc., then the "control unit" operating on the high performance processor chip will use HMI (human machine interaction) to perform rejection reminding, such as voice and visual prompting, and alarm, etc.

In an embodiment, the information calculating unit is configured to receive an image signal output by the image sensor unit and acquire an original data frame, and if the original data of a preset frame number is not received within a preset time, wait for receiving the original data of the preset frame number and then receive temperature data; and if the original data with the preset frame number is received within the preset time, directly receiving the temperature data, and alternately receiving the original data with the preset frame number and the temperature data by the information calculation unit.

In a specific implementation, when the "information calculating unit" in the controller module executes the "triggering of acquiring the original data frame" in step S02 and the "calculating of the face information" in step S03, the "information calculating unit" further performs a temperature compensation operation according to the chip temperature of the image sensor monitored in real time to suppress an influence of a temperature drift phenomenon on the accuracy of the depth data of the 3 rd dimension in the "face information", thereby ensuring that the function and performance of the entire vehicle-mounted face recognition system are not affected.

The working principle is shown in fig. 4, and the flow of suppressing the temperature drift is specifically described as follows:

step S2301 represents triggering the image sensor to output an original data frame. In this step, there is an operation of setting a timer: suppose a timer of 1s is preset and the initial state of the timer is set to "1 s countdown has reached". It should be noted that the operation of setting the timer only needs to be executed once when the timer is triggered for the first time, and the setting does not need to be performed every time the timer is triggered.

In step S2302, it is determined whether or not the timing 1S has ended. If the aforementioned step S2301 is the first time trigger, since "1S countdown has been reached" is set, this time indicates that the timing 1S has ended, so step S2303 is executed next. If the aforementioned step S2301 is not triggered for the first time, there are two possibilities: if the timing 1S is not finished, directly jumping to the step S2304 for execution; if the timing 1S has ended, step S2303 is executed next.

Step S2303, which shows that, after the timing 1S is determined to have ended in step S2302, the "information calculating unit" in the controller module first reads the temperature data of the image sensor chip through the I2C control signal transmitted in the LVDS signal; the 1s countdown is then reset to begin. It should be noted that the chip temperature of the image sensor fluctuates, but does not jump greatly, so the chip temperature can be obtained every 1 s.

Step S2304, which represents that the "information calculation unit" in the controller module determines whether the data of the consecutive 4 frames of original data frames have been completely received after the trigger operation is performed 1 time. If the reception is not completed, the reception of the data is continuously waited for to be completed. It should be noted that, the 3D image sensor continuously outputs data of 4 frames of original data frames after being triggered each time; the data of the 4 original data frames need to be accurately stored according to the received sequence, otherwise, the subsequent calculation of the 'face information' based on the data of the 4 original data frames will generate an error result.

Step S2305, as shown in fig. 5; indicating that the current image sensor temperature information is subjected to a packing operation with the received data of 4 frames of original data frames, and then the whole data packet is fed into the FIFO memory area. It should be noted that, the data is packed in the manner shown in fig. 5, and time t, t + Δ t, t +2 Δ t, and t +3 Δ t represent the data of 4 frames of original data frames received after triggering, respectively, where Δ t represents the data transmission time of 1 frame of original data frame.

Step S2306, representing that the 'information calculating unit' in the vehicle-mounted controller module takes out the data packet from the FIFO storage area and starts to calculate the 3D face information; meanwhile, a feedback coefficient for restraining the temperature drift is obtained in the calculation process. It should be noted that, the "information computing unit" is used as a software sub-module running on a high-performance processor chip in the vehicle-mounted controller module, and a multithreading mechanism is adopted in the software sub-module, and the use of the FIFO storage area not only realizes synchronization among multiple threads, but also plays a role in improving the performance of the whole software sub-module.

Step S2307, which shows that the "information calculating unit" in the vehicle-mounted controller module adjusts the relevant control register on the image sensor according to the feedback coefficient for suppressing the temperature drift obtained in step S2306, so as to implement closed-loop control. Specifically, it is the "information computing unit" that performs control by the I2C data transmitted in the LVDS signal.

As shown in fig. 6, the vehicle-mounted face recognition system according to the present invention fully considers the safety, and besides the high safety level requirement of the "face recognition algorithm unit" in the vehicle-mounted controller module, the safety design is performed on the interaction between the whole vehicle-mounted face recognition system and the entire vehicle, and the scheme of the interaction between the vehicle-mounted face recognition system and the entire vehicle is specifically described as follows:

the vehicle-mounted controller module of the self vehicle-mounted face recognition system is not directly connected with the whole vehicle CAN network, but indirectly interacts with the whole vehicle CAN network through the BCM module. Namely, the vehicle-mounted controller module and the BCM module form a CAN sub-network, and the interactive CAN messages in the CAN sub-network are invisible to an OBD port on the whole vehicle; meanwhile, two independent CAN sub-modules are adopted on the BCM module to be respectively connected with the vehicle-mounted face recognition system and the whole vehicle CAN network; the method means that anyone CAN not crack the request information between the vehicle-mounted face recognition system and the BCM module by collecting the CAN message information of the OBD port. It is worth to be noted that a series of operations including door lock motor locking, engine starting permission and the like are independently completed by the BCM module; all the execution components are independently controlled by the BCM module, and control instructions of the BCM module aiming at each execution component are also invisible to an OBD port on the whole vehicle. Therefore, the safety of the whole vehicle is further improved.

The full name of the shorthand words which the patent requires to explain includes 3D: 3-Dimension, meaning three-dimensional. The 3D image sensor is different from a conventional camera, which is a 2D device, and captures an image projected onto a plane. The 3D camera/image sensor has not only image pixels projected onto a plane but also depth information for each pixel point, and the depth information can be simply understood as distance information. LVDS: abbreviation of Low Voltage Differential Signaling, Low Voltage Differential Signaling. The anti-interference capability is strong, and long-distance transmission can be realized. I2C: a simple, bi-directional two-wire synchronous serial bus developed by Philips corporation. Is widely adopted in the vehicle-mounted embedded device. And (3) OBD: in the On Board Diagnostics, a diagnostic port On the entire vehicle is called an OBD port, which CAN monitor related message information On the entire vehicle CAN network. FIFO: first Input First Output shorthand, First-in-First-out mechanism. This mechanism is typically used in software for digital signal processing. GPIO: general-purpose input/output, generic-purpose input/output shorthand, generic-purpose IO port/pin. HMI: the shorthand for the Human Machine Interface represents a Human Machine Interface.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. A facial recognition system comprising a 3D module and a controller module, the 3D module being connected to the controller module, wherein the controller module comprises:

an information calculation unit for acquiring an image of the 3D module and calculating data;

the face recognition unit is used for judging whether the user is a legal user or not according to the data;

the control unit is used for controlling the external terminal according to the legal user information;

the information calculation unit, the face recognition unit, and the control unit are provided on the same chip.

2. A facial recognition system as claimed in claim 1, wherein an input of said facial recognition unit is connected to an output of said information calculation unit and an output of said facial recognition unit is connected to an input of said control unit.

3. A facial recognition system as claimed in claim 1, wherein the 3D module comprises an image sensor unit for capturing images and outputting image signals.

4. A face recognition system according to claim 3, wherein the information calculation unit is adapted to receive the image signal output from the image sensor unit and collect a raw data frame,

if the original data with the preset frame number is not received within the preset time, waiting for the original data with the preset frame number to be received and then receiving the temperature data;

and if the original data with the preset frame number is received within the preset time, directly receiving the temperature data.

5. A face recognition system as set forth in claim 4, wherein the information calculation unit alternately receives the raw data and the temperature data for the preset number of frames.

6. A facial recognition system as in claim 3 wherein the controller module further comprises a serializer block and the 3D module further comprises a deserializer block, the output of the serializer block being connected to the input of the deserializer block.

7. A facial recognition system as claimed in claim 1, wherein said computing data comprises: image data for identifying the person who is the principal, and depth data for calculating whether the person is a living body.

8. A face recognition system as claimed in claim 3, wherein if the face recognition unit determines that it is a valid user, it sends information of the valid user to the control unit.

9. The facial recognition system of claim 8, wherein if the facial recognition unit determines that the user is an illegal user, sending information of the illegal user to the control unit to obtain a rejection alert, and simultaneously, the 3D module re-collects the image.

10. A vehicle comprising a face recognition system as claimed in any one of claims 1 to 9.

Images