CN110191266B - Data processing method, apparatus, electronic device, and computer-readable storage medium - Google Patents
Data processing method, apparatus, electronic device, and computer-readable storage medium Download PDFInfo
- Publication number
- CN110191266B CN110191266B CN201910492085.2A CN201910492085A CN110191266B CN 110191266 B CN110191266 B CN 110191266B CN 201910492085 A CN201910492085 A CN 201910492085A CN 110191266 B CN110191266 B CN 110191266B
- Authority
- CN
- China
- Prior art keywords
- processing unit
- image
- controller
- type
- bus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 308
- 238000000034 method Methods 0.000 claims abstract description 35
- 230000001360 synchronised effect Effects 0.000 claims description 12
- 230000002457 bidirectional effect Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 13
- 238000012937 correction Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 229910052705 radium Inorganic materials 0.000 description 2
- HCWPIIXVSYCSAN-UHFFFAOYSA-N radium atom Chemical compound [Ra] HCWPIIXVSYCSAN-UHFFFAOYSA-N 0.000 description 2
- 238000004148 unit process Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000004071 biological effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
- H05B47/10—Controlling the light source
- H05B47/175—Controlling the light source by remote control
- H05B47/18—Controlling the light source by remote control via data-bus transmission
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
本申请实施例涉及一种数据处理方法、装置、电子设备及计算机可读存储介质。上述方法，包括：当第一处理单元接收到第二处理单元发送的图像采集指令时，根据所述图像采集指令确定采集的图像类型；通过双向二线制同步串行I2C总线向控制器发送与图像类型对应的控制指令，所述控制指令用于控制开启泛光灯和镭射灯中的至少一个；通过脉冲宽度调制PWM模块向所述控制器发送脉冲，点亮开启的所述泛光灯和镭射灯中的至少一个，并通过激光摄像头采集目标图像；通过第一处理单元对目标图像进行处理，并将处理后的目标图像发送给所述第二处理单元。上述数据处理方法、装置、电子设备及计算机可读存储介质，可以降低控制泛光灯和镭射灯等的复杂度，且节约成本。
The embodiments of the present application relate to a data processing method, an apparatus, an electronic device, and a computer-readable storage medium. The above method includes: when the first processing unit receives the image acquisition instruction sent by the second processing unit, determining the type of the image to be acquired according to the image acquisition instruction; A control command corresponding to the type, the control command is used to control at least one of the floodlight and the laser light to be turned on; send a pulse to the controller through the pulse width modulation PWM module to light the turned on of the floodlight and the laser light At least one of the lamps is used, and the target image is collected by the laser camera; the target image is processed by the first processing unit, and the processed target image is sent to the second processing unit. The above data processing method, device, electronic device and computer-readable storage medium can reduce the complexity of controlling floodlights and laser lights, and save costs.
Description
The application is a divisional application with the application number of 201810402999.0, application date of 2018, 4-month and 28-month, and the title of data processing method, device, electronic equipment and computer readable storage medium.
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method and apparatus, an electronic device, and a computer-readable storage medium.
Background
3D (3Dimensions, three-dimensional) human faces play an important role in different application scenes such as face recognition, facial beautification, 3D model establishment and the like. The electronic equipment can emit laser through lasers such as a laser lamp, the face image irradiated by the laser is collected through the camera, and the 3D face is constructed through structured light. In the conventional method, the method for controlling the laser to emit laser by the electronic device is complicated and has high cost.
Disclosure of Invention
Embodiments of the present application provide a data processing method, an apparatus, an electronic device, and a computer-readable storage medium, which can reduce the complexity of controlling a floodlight, a laser light, and the like, and save the cost.
A method of data processing, comprising:
when the first processing unit receives an image acquisition instruction sent by the second processing unit, determining the type of an acquired image according to the image acquisition instruction;
sending a control instruction corresponding to the image type to a controller through a bidirectional two-wire system synchronous serial I2C bus, wherein the control instruction is used for controlling to start at least one of a floodlight and a laser lamp;
sending a pulse to the controller through a Pulse Width Modulation (PWM) module, lighting at least one of the floodlight and the radium-shine lamp which are started, and collecting a target image through a laser camera;
and processing the target image through the first processing unit, and sending the processed target image to the second processing unit.
A data processing apparatus comprising:
the instruction sending module is used for determining the type of the acquired image according to the image acquisition instruction when the first processing unit receives the image acquisition instruction sent by the second processing unit; sending a control instruction corresponding to the image type to a controller through a bidirectional two-wire system synchronous serial I2C bus, wherein the control instruction is used for controlling to start at least one of a floodlight and a laser lamp;
the pulse sending module is used for sending pulses to the controller through the Pulse Width Modulation (PWM) module, lightening at least one of the started floodlight and the started laser light and collecting a target image through the laser camera;
and the processing module is used for processing the target image through the first processing unit and sending the processed target image to the second processing unit.
An electronic device comprises a camera module, a first processing unit, a second processing unit and a controller, wherein the first processing unit is respectively connected with the second processing unit and the camera module, and is connected with the controller through an I2C bus; the camera module comprises a laser camera, a floodlight and a laser lamp, and the floodlight and the laser lamp are respectively connected with the controller; the first processing unit comprises a PWM module, and the first processing unit is connected with the controller through the PWM module;
the first processing unit is used for determining the type of the acquired image according to the image acquisition instruction when receiving the image acquisition instruction sent by the second processing unit, sending a control instruction corresponding to the image type to the controller through a bidirectional two-wire system synchronous serial I2C bus, wherein the control instruction is used for controlling to start at least one of a floodlight and a laser lamp, sending a pulse to the controller through a Pulse Width Modulation (PWM) module, lighting the started at least one of the floodlight and the laser lamp, acquiring a target image through a laser camera, processing the target image and sending the processed target image to the second processing unit.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the above-mentioned data processing method.
According to the data processing method, the data processing device, the electronic equipment and the computer readable storage medium, when the first processing unit receives the image acquisition instruction sent by the second processing unit, the type of the acquired image is determined according to the image acquisition instruction, the control instruction corresponding to the image type is sent to the controller through the I2C bus, at least one of the floodlight and the laser lamp is controlled to be started, the PWM module sends a pulse to the controller, at least one of the started floodlight and the started laser lamp is lightened, the target image is processed after the target image is acquired, the floodlight and the laser lamp can be controlled through one controller, the complexity of controlling the floodlight, the laser lamp and the like can be reduced, and the cost is saved.
Drawings
FIG. 1 is a diagram of an exemplary implementation of a data processing method;
FIG. 2 is a diagram of an application scenario of a data processing method in another embodiment;
FIG. 3 is a block diagram of an electronic device in one embodiment;
FIG. 4 is a flow diagram illustrating a data processing method according to one embodiment;
FIG. 5 is a schematic diagram of a first processing unit, a laser camera, and a controller connected to the same I2C bus in one embodiment;
FIG. 6 is a schematic flow chart illustrating a process for a first processing unit sending control instructions to a controller according to image capture instructions in one embodiment;
FIG. 7 is a flow diagram illustrating processing of a target image by a first processing unit in one embodiment;
FIG. 8 is a block diagram of a data processing apparatus in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first client may be referred to as a second client, and similarly, a second client may be referred to as a first client, without departing from the scope of the present application. Both the first client and the second client are clients, but they are not the same client.
Fig. 1 is a diagram illustrating an application scenario of a data processing method according to an embodiment. As shown in fig. 4, the electronic device includes a laser camera 102, a floodlight 104, a laser lamp 106, a first processing unit 110, a second processing unit 120, and a controller 130. The first Processing Unit 110 may be an MCU (micro controller Unit) module, etc., and the second Processing Unit 120 may be a CPU (Central Processing Unit) module, etc. The first processing unit 110 may be connected to the laser camera 102 and the second processing unit 120, and the first processing unit 110 may be connected to the controller 130 through an I2C bus. The first processing unit 110 may include a PWM (Pulse Width Modulation) module 112, and is connected to the controller 130 through the PWM module 112, and the controller 130 may be connected to the floodlight 104 and the laser light 106 respectively.
When the first processing unit 110 receives the image acquisition instruction sent by the second processing unit 120, a control instruction is sent to the controller 130 through the I2C bus, and the control instruction can be used for controlling to turn on at least one of the floodlight 104 and the laser light 106. The first processing unit 110 may send a pulse to the controller 130 through the PWM module 112, illuminate at least one of the floodlight 104 and the laser light 106 that are turned on, and capture an image of the object through the laser camera 102. The first processing unit 110 may process the target image and transmit the processed target image to the second processing unit 120.
Fig. 2 is an application scenario diagram of a data processing method in another embodiment. As shown in fig. 2, the electronic device 200 may include a camera module 210, a second processing unit 220, and a first processing unit 230. The second processing unit 220 may be a CPU module. The first processing unit 230 may be an MCU module. The first processing unit 230 is connected between the second processing unit 220 and the camera module 210, the first processing unit 230 can control the laser camera 212, the floodlight 214 and the laser light 218 in the camera module 210, and the second processing unit 220 can control the RGB camera 216 in the camera module 210.
The camera module 210 includes a laser camera 212, a floodlight 214, an RGB camera 216, and a laser light 218. The laser camera 212 may be an infrared camera for acquiring infrared images. The floodlight 214 is a surface light source capable of emitting infrared light; the laser lamp 218 is a point light source capable of emitting laser light and is a point light source with a pattern. When the floodlight 214 emits a surface light source, the laser camera 212 can obtain an infrared image according to the reflected light. When the laser lamp 218 emits a point light source, the laser camera 212 may obtain a speckle image according to the reflected light. The speckle image is an image of the pattern deformation after the point light source with the pattern emitted by the laser lamp 218 is reflected.
The second processing unit 220 may include a CPU core operating in a TEE (Trusted Execution Environment) Environment and a CPU core operating in a REE (natural Execution Environment) Environment. The TEE environment and the REE environment are both running modes of an ARM module (Advanced RISC Machines, Advanced reduced instruction set processor). The security level of the TEE environment is higher, and only one CPU core in the second processing unit 220 can operate in the TEE environment at the same time. Generally, the operation behavior with higher security level in the electronic device 200 needs to be executed in the CPU core in the TEE environment, and the operation behavior with lower security level can be executed in the CPU core in the REE environment.
The first processing unit 230 includes a PWM module 232, an SPI/I2C (Serial Peripheral Interface/Inter-Integrated Circuit) Interface 234, a RAM (Random Access Memory) module 236, and a depth engine 238. The first processing unit 230 may be connected via the PWM module 232 to a controller for the floodlight 214 and the laser light 218, which controller may be connected to the floodlight 214 and the laser light 218, respectively, for controlling the floodlight 214 and the laser light 218. The first processing unit 230 is also connected to the controller via an I2C bus, and the floodlight 214 or the laser light 218 can be controlled to be turned on via the connected I2C bus, and the PWM module 232 can transmit a pulse to the camera module to light the turned-on floodlight 214 or laser light 218. The first processing unit 230 may collect an infrared image or a speckle image through the laser camera 212. The SPI/I2C interface 234 is used for receiving the image capturing instruction sent by the second processing unit 220. The depth engine 238 may process the speckle images to obtain a depth disparity map.
When the second processing unit 220 receives a data acquisition request of an application program, for example, when the application program needs to perform face unlocking and face payment, an image acquisition instruction may be sent to the first processing unit 230 through the CPU core operating in the TEE environment. After the first processing unit 230 receives the image acquisition command, a control command can be sent to the controller through the I2C bus, the floodlight 214 in the control camera module 210 is controlled to be turned on, the pulse wave is sent to the controller through the PWM module 232 to light the floodlight 214, the laser camera 212 is controlled to acquire the infrared image through the I2C bus, the control command can be sent to the controller through the I2C bus, the laser lamp 218 in the camera module 210 is controlled to be turned on, the pulse wave is sent to the controller through the PWM module 232 to light the laser lamp 218, and the laser camera 212 is controlled to acquire the speckle image through the I2C bus. The camera module 210 may send the collected infrared image and speckle image to the first processing unit 230. The first processing unit 230 may process the received infrared image to obtain an infrared disparity map; and processing the received speckle images to obtain a speckle parallax image or a depth parallax image. The processing of the infrared image and the speckle image by the first processing unit 230 refers to correcting the infrared image or the speckle image and removing the influence of internal and external parameters in the camera module 210 on the image. The first processing unit 230 can be set to different modes, and the images output by the different modes are different. When the first processing unit 230 is set to the speckle pattern mode, the first processing unit 230 processes the speckle image to obtain a speckle disparity map, and a target speckle image can be obtained according to the speckle disparity map; when the first processing unit 230 is set to the depth map mode, the first processing unit 230 processes the speckle images to obtain a depth disparity map, and obtains a depth image according to the depth disparity map, where the depth image is an image with depth information. The first processing unit 230 may send the infrared disparity map and the speckle disparity map to the second processing unit 220, and the first processing unit 230 may also send the infrared disparity map and the depth disparity map to the second processing unit 220. The second processing unit 220 may obtain an infrared image of the target according to the infrared disparity map and obtain a depth image according to the depth disparity map. Further, the second processing unit 220 may perform face recognition, face matching, living body detection, and depth information acquisition on the detected face according to the target infrared image and the depth image.
The communication between the first processing unit 230 and the second processing unit 220 is through a fixed security interface to ensure the security of the transmitted data. As shown in fig. 1, the data sent by the second processing unit 220 to the first processing unit 230 is through a SECURE SPI/I2C 240, and the data sent by the first processing unit 230 to the second processing unit 220 is through a SECURE MIPI (Mobile Industry Processor Interface) 250.
In an embodiment, the first processing unit 230 may also obtain a target infrared image according to the infrared disparity map, calculate and obtain a depth image according to the depth disparity map, and then send the target infrared image and the depth image to the second processing unit 220.
FIG. 3 is a block diagram of an electronic device in one embodiment. As shown in fig. 3, the electronic device includes a processor, a memory, a display screen, and an input device connected through a system bus. The memory may include, among other things, a non-volatile storage medium and a processor. The non-volatile storage medium of the electronic device stores an operating system and a computer program, and the computer program is executed by a processor to implement a data processing method provided in the embodiment of the present application. The processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The internal memory in the electronic device provides an environment for the execution of the computer program in the nonvolatile storage medium. The display screen of the electronic device may be a liquid crystal display screen or an electronic ink display screen, and the input device may be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a housing of the electronic device, or an external keyboard, a touch pad or a mouse. The electronic device may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device, etc. Those skilled in the art will appreciate that the architecture shown in fig. 3 is a block diagram of only a portion of the architecture associated with the subject application, and does not constitute a limitation on the electronic devices to which the subject application may be applied, and that a particular electronic device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
As shown in fig. 4, in one embodiment, there is provided a data processing method comprising the steps of:
And step 420, sending a control instruction corresponding to the image type to the controller through a bidirectional two-wire system synchronous serial I2C bus, wherein the control instruction is used for controlling to start at least one of the floodlight and the radium lamp.
When an application program in the electronic device needs to acquire face data, the application program may send a data acquisition request to the second processing unit, where the face data may include, but is not limited to, data that needs to be subjected to face verification in scenes such as face unlocking and face payment, face depth information, and the like. After receiving the data acquisition request, the second processing unit may send an image acquisition instruction to the first processing unit, where the first processing unit may be an MCU module, and the second processing unit may be a CPU module.
The electronic equipment can comprise a controller, the controller can be respectively connected with the floodlight and the laser lamp, and the floodlight and the laser lamp can be controlled by the same controller. The controller controls the floodlight and the laser light, and can comprise controlling and starting the floodlight or the laser light, controlling the switching between the floodlight and the laser light, controlling the emission power of the floodlight and the laser light, and the like. The first processing unit may be connected to the controller via an I2C bus, and the I2C bus may implement data transmission between devices connected to the I2C bus via a data line and a clock line. When the first processing unit receives the image acquisition instruction sent by the second processing unit, a control instruction can be sent to the controller through the I2C bus, and after the controller receives the control instruction, at least one of the floodlight and the laser lamp can be started according to the control instruction.
And 430, sending a pulse to the controller through the pulse width modulation PWM module, lighting at least one of the started floodlight and the started radium-shine lamp, and collecting a target image through the laser camera.
The first processing unit can be connected with the controller through the PWM module, when at least one of the floodlight and the laser lamp is controlled to be started, the first processing unit can send pulses to the controller through the PWM module, and at least one of the started floodlight and the started laser lamp is lightened. Optionally, the PWM module may control to turn on at least one of the floodlight and the laser light according to a pulse signal continuously sent to the controller at a certain voltage amplitude and a certain time interval.
The first processing unit may collect a target image through the laser camera, and the target image may include an infrared image, a speckle image, and the like. If what open is floodlight, the PWM module can send the pulse to the controller, lights floodlight, and floodlight can be one kind to the pointolite of all directions uniform irradiation, when floodlight was lighted, can emit the infrared light, and the laser camera can gather the people face and obtain infrared image. If the laser lamp is started, the PWM module can send pulses to the controller to light the laser lamp. When the laser lamp is lighted, the emitted laser can be diffracted by a lens and a DOE (diffraction optical elements) to generate a pattern with speckle particles, the pattern with the speckle particles is projected to a target object, the offset of the speckle pattern is generated due to different distances between each point of the target object and electronic equipment, and a laser camera acquires the target object to obtain a speckle image.
The laser camera can send the target image of gathering to first processing unit, and first processing unit can be handled the target image, and wherein, the target image can include infrared image, speckle image etc.. After the first processing unit determines the image type according to the image acquisition instruction, a target image corresponding to the image type can be acquired according to the determined image type, and corresponding processing is carried out on the target image. When the image type is an infrared image, the first processing unit can send pulses to the first controller through the first PWM module, light the floodlight, acquire the infrared image through the laser camera and process the infrared image to obtain an infrared parallax image. When the image type is the speckle image, the first processing unit can send pulses to the second controller through the second PWM module, light the laser lamp, collect the speckle image through the laser camera, and can process the speckle image to obtain the speckle parallax image. When the image type is a depth image, the first processing unit can collect the speckle image and process the collected speckle image to obtain a depth parallax image.
In one embodiment, the first processing unit may perform a correction process on the target image, where the correction process is performed to correct an image content shift of the target image due to internal and external parameters of the laser camera and the RGB camera, for example, an image content shift due to a laser camera deflection angle, a placement position between the laser camera and the RGB camera, and the like. After the target image is corrected, a disparity map of the target image can be obtained, for example, an infrared disparity map can be obtained by correcting an infrared image, and a speckle disparity map or a depth disparity map can be obtained by correcting a speckle image. The correction processing is performed on the target image, so that the situation that the image finally presented on the screen of the electronic equipment is ghosted can be prevented.
The first processing unit processes the target image, and can send the processed target image to the second processing unit. The second processing unit can obtain required images such as infrared images, speckle images, depth images and the like according to the processed target images. The second processing unit can process the required image according to the requirement of the application program.
For example, when the application program needs to perform face verification, the second processing unit may perform face detection on the obtained required image, where the face detection may include face recognition, face matching, and living body detection. The human face recognition means that whether a human face exists in an image required for recognition, the human face matching means that the human face in the image required for recognition is matched with a pre-stored human face, and the living body detection means that whether the human face in the image required for detection has biological activity or not. If the application program needs to acquire the depth information of the face, the generated depth image can be uploaded to the application program, and the application program can perform beautifying processing, three-dimensional modeling and the like according to the received depth image.
In this embodiment, when the first processing unit receives the image acquisition instruction that the second processing unit sent, send control instruction to the controller through I2C bus, at least one in floodlight and laser lamp is opened in the control to send the pulse to the controller through the PWM module, light at least one in floodlight and laser lamp opened, handle the target image again after gathering the target image, can realize the control to floodlight and laser lamp through a controller, can reduce the complexity of controlling floodlight and laser lamp etc. and practice thrift the cost.
In one embodiment, the step of acquiring an image of the target by a laser camera comprises: and the laser camera is controlled by an I2C bus to acquire a target image.
The first processing unit can be connected with the laser camera through an I2C bus, and controls the laser camera to acquire a target image through the connected I2C bus. In one embodiment, the first processing unit, the laser camera, and the controller may be connected to the same I2C bus. After the first processing unit receives the image acquisition command sent by the second processing unit, the floodlight or the laser light can be controlled to be turned on through the I2C bus, the pulse is sent to the controller through the PWM module, the turned-on floodlight or the laser light is lightened, and the laser camera is controlled to acquire target images such as infrared images or speckle images through the connected I2C bus.
In one embodiment, the first processing unit can address the controller through the connected I2C bus, send a control command to the controller, control to turn on the floodlight or the radium light, address the laser camera through the connected I2C bus, control the laser camera to collect a target image, and multiplex the same connected I2C bus at different times, so that resources are saved.
FIG. 5 is a schematic diagram of the first processing unit, the laser camera, and the controller connected to the same I2C bus in one embodiment. As shown in fig. 5, the electronic device includes a laser camera 102, a floodlight 104, a laser lamp 106, a first processing unit 110, a second processing unit 120, and a controller 130. The first processing unit 110 may be connected with the second processing unit 120. The first processing unit 110 may include a PWM module 112 and is connected to the controller 130 through the PWM module 112, and the controller 130 may be connected to the floodlight 104 and the laser light 106 respectively. The laser camera 102, the first processing unit 110, and the controller 130 may be connected to the same I2C bus.
After receiving the image acquisition command sent by the second processing unit 120, the first processing unit 110 can send a control command to the controller 130 through the I2C bus to control the floodlight 104 or the laser light 106 to be turned on, send a pulse to the controller 130 through the PWM module 112 to light the turned-on floodlight 104 or the laser light 106, and control the laser camera 102 to acquire target images such as infrared images or speckle images through the connected I2C bus.
In this embodiment, the floodlight, the laser light and the laser camera are controlled by the same I2C bus, and the I2C bus is multiplexed, so that the complexity of the control circuit can be reduced, and the cost can be reduced.
As shown in FIG. 6, in one embodiment, the step of sending a control instruction corresponding to the image type to the controller via the I2C bus comprises the steps of:
The first processing unit receives the image acquisition instruction sent by the second processing unit, and can determine the type of the acquired image according to the image acquisition instruction, wherein the image type can be one or more of an infrared image, a speckle image, a depth image and the like. The image type can be determined according to the face data required by the application program, and after the second processing unit receives the data acquisition request, the image type can be determined according to the data acquisition request, and an image acquisition instruction containing the image type is sent to the first processing unit. For example, if the data required to unlock the face is the infrared image and the speckle image, the image type may be determined to be the depth image if the face depth information is required, but the present invention is not limited thereto.
If the image type is the first type, in this embodiment, the first type may be an infrared image, the first processing unit may send a first control instruction to the controller through the connected I2C, and the controller may switch to the floodlight according to the first control instruction and turn on the floodlight. The first processing unit may transmit a pulse to the controller via the PWM module to illuminate the floodlight. Alternatively, the first processing unit may address the controller via I2C and send a first control instruction to the controller.
And step 604, if the image type is the second type, the first processing unit sends a second control instruction to the controller through the I2C bus, and the second control instruction is used for instructing the controller to turn on the laser lamp.
If the image type is the second type, in this embodiment, the second type may be a speckle image, a depth image, or the like, the first processing unit may send a second control instruction to the controller through the connected I2C, and the controller may switch to the laser lamp according to the second control instruction and turn on the laser lamp. The first processing unit can emit pulses to the controller through the PWM module to light the laser lamp.
The first processing unit determines the image types according to the image acquisition instruction, and the image types can include at least two types, for example, the image types can include a first type and a second type at the same time. When the image type includes an infrared image and a speckle image, or includes an infrared image and a depth image, the infrared image and the speckle image need to be acquired simultaneously. The first processing unit can collect the infrared image firstly or the speckle image firstly, and does not limit the collecting sequence. The first processing unit can send a first control instruction to the controller through the I2C bus, turn on the floodlight, transmit pulse to the controller through the PWM module, light the floodlight, and then control the laser camera to collect infrared images through the I2C bus. After the first processing unit controls the laser camera to collect the target image corresponding to the first type, a second control instruction can be sent to the controller through the I2C bus, the laser lamp is started, pulses are transmitted to the controller through the PWM module, the laser lamp is lightened, and then the laser camera is controlled through the I2C bus to collect the speckle image.
In one embodiment, when the image type may include both the first type and the second type, the first processing unit may also first send a second control instruction to the controller through the I2C bus, turn on the laser lamp, transmit a pulse to the controller through the PWM module, illuminate the laser lamp, and control the laser camera to collect a speckle image through the I2C bus. After the first processing unit controls the laser camera to collect the target image corresponding to the second type, a first control instruction can be sent to the controller through the I2C bus, the floodlight is started, the pulse is transmitted to the controller through the PWM module, the floodlight is lightened, and then the infrared image is collected by the laser camera through the I2C bus.
Optionally, the first processing unit may send the first control instruction and the second control instruction to the controller at different times, a time interval between the time of sending the first control instruction and the time of sending the second control instruction may be smaller than a time threshold, and the laser camera may collect the speckle images at the time interval smaller than the time threshold after collecting the infrared images, so that image contents of the collected infrared images and the speckle images are relatively consistent, and subsequent processing such as face detection is facilitated. The time threshold may be set according to actual requirements, for example, 20 milliseconds, 30 milliseconds, and the like. The image content of the collected infrared image is consistent with that of the speckle image, and the accuracy of follow-up face detection can be improved.
In this embodiment, the floodlight and the laser light can be switched and controlled by one controller, so that the complexity of the control circuit can be reduced, and the cost can be reduced.
As shown in FIG. 7, in one embodiment, step 430 processes the target image by the first processing unit and sends the processed target image to the second processing unit, comprising the steps of:
In the camera coordinate system, a straight line which is vertical to the imaging plane and passes through the center of the mirror surface is taken as a Z axis, and if the coordinates of the object in the camera coordinate system are (X, Y, Z), the Z value is the depth information of the object in the imaging plane of the camera. If the application program needs to acquire the depth information of the face, a depth image containing the face depth information needs to be acquired. The first processing unit can control the laser lamp to be started through an I2C bus and control the laser camera to collect speckle images through an I2C bus. The first processing unit can be pre-stored with a reference speckle pattern, the reference speckle pattern can be provided with reference depth information, and the depth information of each pixel point contained in the speckle image can be acquired according to the collected speckle image and the reference speckle image.
And step 704, matching the reference speckle image with the speckle image to obtain a matching result.
The first processing unit may sequentially select a pixel block of a predetermined size, for example, 31 pixels by 31 pixels, centering on each pixel point included in the collected speckle image, and search for a block matching the selected pixel block on the reference speckle image. The first processing unit can find two points on the same laser light path in the speckle image and the reference speckle image respectively from the pixel block selected from the acquired speckle images and the block matched with the reference speckle image, wherein the speckle information of the two points on the same laser light path is consistent, and the two points on the same laser light path can be identified as corresponding pixel points. The depth information of the points on each laser path in the reference speckle image is known. The first processing unit can calculate the offset between two corresponding pixel points of the target speckle image and the reference speckle image on the same laser light path, and calculate the depth information of each pixel point contained in the acquired speckle pattern according to the offset.
In one embodiment, the first processing unit calculates an offset between the collected speckle image and the reference speckle pattern, and calculates depth information of each pixel point included in the speckle image according to the offset, where a calculation formula may be as shown in formula (1):
wherein Z isDRepresenting the depth information of the pixel points, namely the depth values of the pixel points; l is the distance between the laser camera and the laser; f is the focal length of the lens in the laser camera, Z0The reference speckle image is acquired by comparing the depth value of a reference plane with the depth value of a laser camera of the electronic equipment, and P is the offset between the acquired speckle image and the corresponding pixel point in the reference speckle image. P can be obtained by multiplying the amount of pixels of the offset of the pixels in the target speckle pattern and the reference speckle pattern by the actual distance of one pixel. When the distance between the target object and the laser camera is larger than the distance between the reference plane and the laser camera, P is a negative value, and when the distance between the target object and the laser camera is smaller than the distance between the reference plane and the laser camera, P is a positive value.
And step 706, generating a depth disparity map according to the reference depth information and the matching result, sending the depth disparity map to the second processing unit, and processing the depth disparity map through the second processing unit to obtain the depth map.
The first processing unit obtains the depth information of each pixel point contained in the acquired speckle image, can correct the acquired speckle image, and corrects the acquired speckle image to have image content offset caused by internal and external parameters of the laser camera and the RGB camera. The first processing unit can generate a depth parallax map according to the corrected speckle images and the depth values of all pixel points in the speckle images, and sends the depth parallax map to the second processing unit. The second processing unit may obtain a depth map according to the depth disparity map, and the depth map may include depth information of each pixel point. The second processing unit can upload the depth map to an application program, and the application program can perform beautifying, three-dimensional modeling and the like according to the depth information of the face in the depth map. The second processing unit can also carry out living body detection according to the depth information of the face in the depth map, and can prevent the collected face from being a two-dimensional plane face and the like.
Optionally, the second processing unit in the electronic device may include two operation modes, where the first operation mode may be a TEE, the TEE is a trusted operation environment, and the security level is high; the second operation mode may be REE, which is a natural operation environment, and the security level of the REE is low. After receiving a data acquisition request sent by an application program, the second processing unit can send an image acquisition instruction to the first processing unit through the first operation mode. When the second processing unit is a CPU with a single core, the single core can be directly switched from the second operation mode to the first operation mode; when the second processing unit has multiple cores, one core can be switched from the second operation mode to the first operation mode, other cores still operate in the second operation mode, and an image acquisition instruction is sent to the first processing unit through the core operating in the first operation mode.
After the first processing unit processes the acquired target image, the processed target image can be sent to the kernel operating in the first operation mode, so that the first processing unit can be ensured to always operate in a trusted operation environment, and the safety is improved. The second processing unit may obtain a required image according to the processed target image in the kernel operating in the first operating mode, and process the required image according to a requirement of the application program. For example, the second processing unit may perform face detection on a desired image in the kernel operating in the first operating mode. The image acquisition instruction is sent to the first processing unit through the kernel with high safety of the second processing unit, so that the first processing unit can be ensured to be in an environment with high safety, and the data safety is improved.
In one embodiment, since the kernel operating in the first operating mode is unique, the second processing unit performs face detection on the target image in the TEE environment, and can perform face recognition, face matching, live body detection and the like on the target image one by one in a serial manner. The second processing unit may perform face recognition on the required image, and when a face is recognized, match the face included in the required image with a face stored in advance, and determine whether the faces are the same face. If the face is the same face, then the living body detection is carried out on the face according to the required image, and the collected face is prevented from being a two-dimensional plane face and the like. When the face is not recognized, face matching and live body detection may not be performed, and the processing pressure of the second processing unit may be reduced.
In this embodiment, the depth information of the acquired image can be accurately obtained through the first processing unit, the data processing efficiency is high, and the accuracy of image processing is improved.
In one embodiment, a data processing method is provided, comprising the steps of:
and (1) when the first processing unit receives an image acquisition instruction sent by the second processing unit, sending a control instruction to the controller through a bidirectional two-wire system synchronous serial I2C bus, wherein the control instruction is used for controlling to start at least one of the floodlight and the laser lamp.
In one embodiment, step (1) comprises: determining the type of the acquired image according to the image acquisition instruction; if the image type is a first type, the first processing unit sends a first control instruction to the controller through the I2C bus, and the first control instruction is used for instructing the controller to turn on the floodlight; if the image type is the second type, the first processing unit sends a second control instruction to the controller through the I2C bus, and the second control instruction is used for instructing the controller to start the laser lamp.
In one embodiment, after the step of determining the type of the captured image according to the image capturing instruction, the method further comprises: when the image types comprise a first type and a second type, the first processing unit sends a first control instruction to the controller through the I2C bus, and the floodlight is turned on; after the target image corresponding to the first type is collected through the laser camera, a second control instruction is sent to the controller through an I2C bus, and the laser lamp is started.
In one embodiment, after the step of determining the type of the captured image according to the image capturing instruction, the method further comprises: when the image types comprise a first type and a second type, the first processing unit sends a second control instruction to the controller through the I2C bus, and the laser lamp is started; after the target image of the second type is collected through the laser camera, a first control instruction is sent to the controller through an I2C bus, and the floodlight is turned on.
In one embodiment, a time interval between a time when the first processing unit transmits the first control instruction and a time when the second control instruction is transmitted is less than a time threshold.
And (2) sending pulses to the controller through the pulse width modulation PWM module, lighting at least one of the started floodlight and the started laser light, and collecting a target image through the laser camera.
In one embodiment, the first processing unit, the controller and the laser camera are connected to the same I2C bus; the method comprises the following steps of collecting a target image through a laser camera, wherein the method comprises the following steps: and the laser camera is controlled by an I2C bus to acquire a target image.
And (3) processing the target image through the first processing unit, and sending the processed target image to the second processing unit.
In one embodiment, the target image includes a speckle image; step (3), comprising: acquiring a stored reference speckle image, wherein the reference speckle image is provided with reference depth information; matching the reference speckle image with the speckle image to obtain a matching result; and generating a depth parallax map according to the reference depth information and the matching result, sending the depth parallax map to a second processing unit, and processing the depth parallax map through the second processing unit to obtain the depth map.
In this embodiment, when the first processing unit receives the image acquisition instruction that the second processing unit sent, send control instruction to the controller through I2C bus, at least one in floodlight and laser lamp is opened in the control to send the pulse to the controller through the PWM module, light at least one in floodlight and laser lamp opened, handle the target image again after gathering the target image, can realize the control to floodlight and laser lamp through a controller, can reduce the complexity of controlling floodlight and laser lamp etc. and practice thrift the cost.
It should be understood that, although the steps in the respective flow charts described above are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the various flow diagrams described above may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, an electronic device is provided and includes a camera module, a first processing unit, a second processing unit, and a controller, wherein the first processing unit is connected to the second processing unit and the camera module, respectively. The first processing unit is connected to the controller via an I2C bus. The camera module comprises a laser camera, a floodlight and a laser lamp, and the floodlight and the laser lamp are respectively connected with the controller. The first processing unit comprises a PWM module, and the first processing unit is connected with the controller through the PWM module.
The first processing unit is used for determining the type of the acquired image according to the image acquisition instruction when receiving the image acquisition instruction sent by the second processing unit, sending a control instruction corresponding to the image type to the controller through an I2C bus, wherein the control instruction is used for controlling to start at least one of the floodlight and the laser lamp, sending a pulse to the controller through the Pulse Width Modulation (PWM) module, lighting at least one of the started floodlight and the started laser lamp, acquiring a target image through the laser camera, processing the target image and sending the processed target image to the second processing unit.
In this embodiment, when the first processing unit receives the image acquisition instruction that the second processing unit sent, send control instruction to the controller through I2C bus, at least one in floodlight and laser lamp is opened in the control to send the pulse to the controller through the PWM module, light at least one in floodlight and laser lamp opened, handle the target image again after gathering the target image, can realize the control to floodlight and laser lamp through a controller, can reduce the complexity of controlling floodlight and laser lamp etc. and practice thrift the cost.
In one embodiment, the first processing unit, the controller, and the laser camera are connected to the same I2C bus.
The first processing unit is also used for controlling the laser camera to acquire a target image through an I2C bus.
In this embodiment, the floodlight, the laser light and the laser camera are controlled by the same I2C bus, and the I2C bus is multiplexed, so that the complexity of the control circuit can be reduced, and the cost can be reduced.
In one embodiment, the first processing unit is further configured to determine a type of the captured image according to the image capturing instruction, and if the type of the image is a first type, the first processing unit sends a first control instruction to the controller through an I2C bus, where the first control instruction is used to instruct the controller to turn on the floodlight, and if the type of the image is a second type, the first processing unit sends a second control instruction to the controller through an I2C bus, and the second control instruction is used to instruct the controller to turn on the laser light.
In one embodiment, the first processing unit is further configured to, when the image types include a first type and a second type, send a first control instruction to the controller through an I2C bus, turn on the floodlight, and after a target image corresponding to the first type is acquired by the laser camera, send a second control instruction to the controller through an I2C bus, and turn on the laser light.
In one embodiment, the first processing unit is further configured to, when the image types include a first type and a second type, send a second control instruction to the controller through the I2C bus, turn on the laser light, and after acquiring a target image of the second type through the laser camera, send a first control instruction to the controller through the I2C bus, and turn on the floodlight.
In one embodiment, a time interval between a time when the first processing unit transmits the first control instruction and a time when the second control instruction is transmitted is less than a time threshold.
In this embodiment, the floodlight and the laser light can be switched and controlled by one controller, so that the complexity of the control circuit can be reduced, and the cost can be reduced.
In one embodiment, the target image includes a speckle image.
The first processing unit is further used for acquiring a stored reference speckle image, matching the reference speckle image with the speckle image to obtain a matching result, generating a depth parallax map according to the reference depth information and the matching result, and sending the depth parallax map to the second processing unit, wherein the reference speckle image has the reference depth information;
and the second processing unit is also used for processing the depth parallax map to obtain a depth map.
In this embodiment, the depth information of the acquired image can be accurately obtained through the first processing unit, the data processing efficiency is high, and the accuracy of image processing is improved.
In one embodiment, a data processing apparatus 800 is provided that includes an instruction sending module 810, a pulse sending module 820, and a processing module 830.
And the instruction sending module 810 is configured to, when the first processing unit receives an image acquisition instruction sent by the second processing unit, determine an acquired image type according to the image acquisition instruction, send a control instruction corresponding to the image type to the controller through a bidirectional two-wire system synchronous serial I2C bus, where the control instruction is used to control to turn on at least one of the floodlight and the laser light.
And the pulse sending module 820 is used for sending pulses to the controller through the pulse width modulation PWM module, lightening at least one of the started floodlight and the started laser light, and collecting a target image through the laser camera.
The processing module 830 is configured to process the target image through the first processing unit, and send the processed target image to the second processing unit.
In this embodiment, when the first processing unit receives the image acquisition instruction that the second processing unit sent, send control instruction to the controller through I2C bus, at least one in floodlight and laser lamp is opened in the control to send the pulse to the controller through the PWM module, light at least one in floodlight and laser lamp opened, handle the target image again after gathering the target image, can realize the control to floodlight and laser lamp through a controller, can reduce the complexity of controlling floodlight and laser lamp etc. and practice thrift the cost.
In one embodiment, the first processing unit, the controller, and the laser camera are connected to the same I2C bus.
And the pulse sending module 820 is further used for controlling the laser camera to acquire a target image through the I2C bus.
In this embodiment, the floodlight, the laser light and the laser camera are controlled by the same I2C bus, and the I2C bus is multiplexed, so that the complexity of the control circuit can be reduced, and the cost can be reduced.
In one embodiment, the instruction sending module 810 includes a type determining unit, a first sending unit, and a second sending unit.
And the type determining unit is used for determining the type of the acquired image according to the image acquisition instruction.
And the first sending unit is used for sending a first control instruction to the controller through the I2C bus if the image type is the first type, and the first control instruction is used for instructing the controller to turn on the floodlight.
And if the image type is the second type, the first processing unit sends a second control instruction to the controller through the I2C bus, and the second control instruction is used for instructing the controller to turn on the laser lamp.
In one embodiment, the first sending unit is further configured to send a first control instruction to the controller through the I2C bus to turn on the floodlight when the image types include a first type and a second type.
And the second sending unit is also used for sending a second control instruction to the controller through an I2C bus after the target image corresponding to the first type is collected through the laser camera, and starting the laser lamp.
In one embodiment, the second sending unit is further configured to send a second control instruction to the controller through the I2C bus to turn on the laser lamp when the image type includes the first type and the second type.
And the first sending unit is also used for sending a first control instruction to the controller through the I2C bus and starting the floodlight after the target image of the second type is collected through the laser camera.
In one embodiment, a time interval between a time when the first processing unit transmits the first control instruction and a time when the second control instruction is transmitted is less than a time threshold.
In this embodiment, the floodlight and the laser light can be switched and controlled by one controller, so that the complexity of the control circuit can be reduced, and the cost can be reduced.
In one embodiment, the processing module 830 includes an image acquisition unit, a matching unit, and a generation unit.
And the image acquisition unit is used for acquiring the stored reference speckle images, and the reference speckle images are provided with reference depth information.
And the matching unit is used for matching the reference speckle image with the speckle image to obtain a matching result.
And the generating unit is used for generating a depth parallax map according to the reference depth information and the matching result, sending the depth parallax map to the second processing unit, and processing the depth parallax map through the second processing unit to obtain the depth map.
In this embodiment, the depth information of the acquired image can be accurately obtained through the first processing unit, the data processing efficiency is high, and the accuracy of image processing is improved.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the above-mentioned data processing method.
In an embodiment, a computer program product is provided, comprising a computer program, which, when run on a computer device, causes the computer device to carry out the above-mentioned data processing method when executed.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.
Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (16)
1. A data processing method, comprising:
when the first processing unit receives an image acquisition instruction sent by the second processing unit, determining the type of an acquired image according to the image acquisition instruction;
the first processing unit sends a control instruction corresponding to the image type to a controller through a bidirectional two-wire system synchronous serial I2C bus, and the control instruction is used for controlling to start at least one of a floodlight and a laser lamp; the first processing unit is connected with the controller through the I2C bus;
sending a pulse to the controller through a Pulse Width Modulation (PWM) module, lighting at least one of the floodlight and the radium-shine lamp which are started, and collecting a target image through a laser camera;
and processing the target image through the first processing unit, and sending the processed target image to the second processing unit.
2. The method of claim 1, wherein the first processing unit, controller and laser camera are connected on the same I2C bus;
gather the target image through the laser camera, include:
and controlling the laser camera to acquire a target image through the I2C bus.
3. The method of claim 2, wherein sending control instructions corresponding to the image type to a controller via a bidirectional two-wire synchronous serial I2C bus comprises:
if the image type is a first type, the first processing unit sends a first control instruction corresponding to the first type to the controller through the I2C bus, and the first control instruction is used for instructing the controller to turn on the floodlight; the first type is an infrared image;
if the image type is a second type, the first processing unit sends a second control instruction corresponding to the second type to the controller through the I2C bus, and the second control instruction is used for instructing the controller to turn on the laser lamp; the second type is a speckle image or a depth image.
4. The method of claim 1, wherein after said determining a type of image to capture from said image capture instructions, said method further comprises:
when the image types comprise a first type and a second type, the first processing unit sends a first control instruction to the controller through the I2C bus, and the floodlight is turned on;
after a target image corresponding to the first type is collected through a laser camera, a second control instruction is sent to the controller through the I2C bus, and the laser lamp is started.
5. The method of claim 1, wherein after said determining a type of image to capture from said image capture instructions, said method further comprises:
when the image types comprise a first type and a second type, the first processing unit sends a second control instruction to the controller through the I2C bus, and the laser lamp is started;
and after a target image corresponding to the second type is collected through the laser camera, sending a first control instruction to the controller through the I2C bus, and starting the floodlight.
6. The method according to claim 4 or 5, wherein the time interval between the moment when the first processing unit sends the first control instruction and the moment when the second control instruction is sent is smaller than a time threshold.
7. The method of claim 1, wherein the target image comprises a speckle image;
the processing the target image by the first processing unit and sending the processed target image to the second processing unit includes:
acquiring a stored reference speckle image with reference depth information;
matching the reference speckle image with the speckle image to obtain a matching result;
and generating a depth parallax map according to the reference depth information and the matching result, sending the depth parallax map to the second processing unit, and processing the depth parallax map through the second processing unit to obtain the depth map.
8. A data processing apparatus, comprising:
the device comprises an instruction sending module, a first processing unit and a second processing unit, wherein the instruction sending module is used for determining the type of an acquired image according to an image acquisition instruction sent by the second processing unit when the first processing unit receives the image acquisition instruction, the first processing unit sends a control instruction corresponding to the image type to a controller through a bidirectional two-wire system synchronous serial I2C bus, and the control instruction is used for controlling the opening of at least one of a floodlight and a laser light; the first processing unit is connected with the controller through the I2C bus;
the pulse sending module is used for sending pulses to the controller through the Pulse Width Modulation (PWM) module, lightening at least one of the started floodlight and the started laser light and collecting a target image through the laser camera;
and the processing module is used for processing the target image through the first processing unit and sending the processed target image to the second processing unit.
9. An electronic device is characterized by comprising a camera module, a first processing unit, a second processing unit and a controller, wherein the first processing unit is respectively connected with the second processing unit and the camera module, and is connected with the controller through an I2C bus; the camera module comprises a laser camera, a floodlight and a laser lamp, and the floodlight and the laser lamp are respectively connected with the controller; the first processing unit comprises a PWM module, and the first processing unit is connected with the controller through the PWM module;
the first processing unit is used for determining the type of the acquired image according to the image acquisition instruction when receiving the image acquisition instruction sent by the second processing unit, sending a control instruction corresponding to the image type to a controller through a bidirectional two-wire system synchronous serial I2C bus, wherein the control instruction is used for controlling to start at least one of a floodlight and a laser light, sending a pulse to the controller through a Pulse Width Modulation (PWM) module, lighting the started at least one of the floodlight and the laser light, acquiring a target image through a laser camera, processing the target image and sending the processed target image to the second processing unit.
10. The electronic device of claim 9, wherein the first processing unit, the controller, and the laser camera are connected to the same I2C bus;
the first processing unit is further used for controlling the laser camera to acquire a target image through the I2C bus.
11. The electronic device of claim 10, wherein the first processing unit is further configured to send a first control instruction to the controller through the I2C bus if the image type is a first type, the first control instruction being configured to instruct the controller to turn on the floodlight; the first type is an infrared image; if the image type is a second type, the first processing unit sends a second control instruction to the controller through the I2C bus, and the second control instruction is used for instructing the controller to turn on the laser lamp; the second type is a speckle image or a depth image.
12. The electronic device of claim 9, wherein the first processing unit is further configured to, when the image types include a first type and a second type, send a first control instruction to the controller through the I2C bus, turn on the floodlight, and after a target image corresponding to the first type is captured by the laser camera, send a second control instruction to the controller through the I2C bus, turn on the radium-shine light.
13. The electronic device of claim 9, wherein the first processing unit is further configured to, when the image types include a first type and a second type, send a second control instruction to the controller through the I2C bus, turn on the laser light, and after a target image corresponding to the second type is captured by the laser camera, send a first control instruction to the controller through the I2C bus, turn on the floodlight.
14. The electronic device according to claim 12 or 13, wherein a time interval between a time when the first processing unit transmits the first control instruction and a time when the second control instruction is transmitted is smaller than a time threshold.
15. The electronic device of claim 9, wherein the target image comprises a speckle image;
the first processing unit is further used for acquiring a stored reference speckle image, and the reference speckle image is provided with reference depth information; matching the reference speckle image with the speckle image to obtain a matching result, generating a depth parallax map according to the reference depth information and the matching result, and sending the depth parallax map to the second processing unit, wherein the reference speckle image carries reference depth information;
and the second processing unit is also used for processing the depth parallax map to obtain a depth map.
16. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910492085.2A CN110191266B (en) | 2018-04-28 | 2018-04-28 | Data processing method, apparatus, electronic device, and computer-readable storage medium |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810402999.0A CN108696682B (en) | 2018-04-28 | 2018-04-28 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN201910492085.2A CN110191266B (en) | 2018-04-28 | 2018-04-28 | Data processing method, apparatus, electronic device, and computer-readable storage medium |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810402999.0A Division CN108696682B (en) | 2018-04-28 | 2018-04-28 | Data processing method and device, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110191266A CN110191266A (en) | 2019-08-30 |
| CN110191266B true CN110191266B (en) | 2021-08-31 |
Family
ID=63845252
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810402999.0A Active CN108696682B (en) | 2018-04-28 | 2018-04-28 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN201910492085.2A Active CN110191266B (en) | 2018-04-28 | 2018-04-28 | Data processing method, apparatus, electronic device, and computer-readable storage medium |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810402999.0A Active CN108696682B (en) | 2018-04-28 | 2018-04-28 | Data processing method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN108696682B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108696682B (en) * | 2018-04-28 | 2019-07-09 | Oppo广东移动通信有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| EP3672223B1 (en) | 2018-04-28 | 2022-12-28 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Data processing method, electronic device, and computer-readable storage medium |
| CN108924421B (en) * | 2018-07-16 | 2020-09-11 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer-readable storage medium and electronic equipment |
| CN110335303B (en) * | 2019-06-24 | 2021-10-26 | Oppo广东移动通信有限公司 | Image processing method and apparatus, and storage medium |
| CN112307798A (en) * | 2019-07-24 | 2021-02-02 | 云从科技集团股份有限公司 | Detection equipment |
| CN111896974A (en) * | 2020-08-20 | 2020-11-06 | 昆山丘钛微电子科技有限公司 | Time delay control system and method for TOF module swing error calibration |
| CN113014782B (en) * | 2021-03-19 | 2022-11-01 | 展讯通信（上海）有限公司 | Image data processing method and device, camera equipment, terminal and storage medium |
| CN116033265B (en) * | 2023-01-04 | 2024-08-06 | 浙江吉利控股集团有限公司 | Camera sharing method, system, equipment and computer readable storage medium |
| CN117746802A (en) * | 2023-11-27 | 2024-03-22 | 北京显芯科技有限公司 | Method for controlling command transmission, dimmer and backlight unit |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009262344A (en) * | 2008-04-22 | 2009-11-12 | Ricoh Co Ltd | Image formation device and image correcting method |
| CN105407610A (en) * | 2015-12-09 | 2016-03-16 | 深圳天珑无线科技有限公司 | Method and apparatus for controlling dual flash lamps |
| CN106331453A (en) * | 2016-08-24 | 2017-01-11 | 深圳奥比中光科技有限公司 | Multi-image acquisition system and image acquisition method |
| CN106973251A (en) * | 2017-03-23 | 2017-07-21 | 移康智能科技（上海）股份有限公司 | Image data transfer method and device |
| CN107105217A (en) * | 2017-04-17 | 2017-08-29 | 深圳奥比中光科技有限公司 | Multi-mode depth calculation processor and 3D rendering equipment |
| CN107341481A (en) * | 2017-07-12 | 2017-11-10 | 深圳奥比中光科技有限公司 | It is identified using structure light image |
| CN108696682B (en) * | 2018-04-28 | 2019-07-09 | Oppo广东移动通信有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622591B (en) * | 2012-01-12 | 2013-09-25 | 北京理工大学 | 3D (three-dimensional) human posture capturing and simulating system |
| US9462253B2 (en) * | 2013-09-23 | 2016-10-04 | Microsoft Technology Licensing, Llc | Optical modules that reduce speckle contrast and diffraction artifacts |
| CN103971405A (en) * | 2014-05-06 | 2014-08-06 | 重庆大学 | Method for three-dimensional reconstruction of laser speckle structured light and depth information |
| CN106851927B (en) * | 2017-04-19 | 2018-06-12 | 慈溪锐恩电子科技有限公司 | A kind of multichannel light modulation toning LED drive circuit of speech recognition |
| CN107944422B (en) * | 2017-12-08 | 2020-05-12 | 业成科技（成都）有限公司 | Three-dimensional camera device, three-dimensional camera method and face recognition method |
| CN108573170B (en) * | 2018-04-12 | 2020-06-12 | Oppo广东移动通信有限公司 | Information processing method and apparatus, electronic device, computer-readable storage medium |
| CN108564032B (en) * | 2018-04-12 | 2020-05-22 | Oppo广东移动通信有限公司 | Image processing method, apparatus, electronic device, and computer-readable storage medium |
-
2018
- 2018-04-28 CN CN201810402999.0A patent/CN108696682B/en active Active
- 2018-04-28 CN CN201910492085.2A patent/CN110191266B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009262344A (en) * | 2008-04-22 | 2009-11-12 | Ricoh Co Ltd | Image formation device and image correcting method |
| CN105407610A (en) * | 2015-12-09 | 2016-03-16 | 深圳天珑无线科技有限公司 | Method and apparatus for controlling dual flash lamps |
| CN106331453A (en) * | 2016-08-24 | 2017-01-11 | 深圳奥比中光科技有限公司 | Multi-image acquisition system and image acquisition method |
| CN106973251A (en) * | 2017-03-23 | 2017-07-21 | 移康智能科技（上海）股份有限公司 | Image data transfer method and device |
| CN107105217A (en) * | 2017-04-17 | 2017-08-29 | 深圳奥比中光科技有限公司 | Multi-mode depth calculation processor and 3D rendering equipment |
| CN107341481A (en) * | 2017-07-12 | 2017-11-10 | 深圳奥比中光科技有限公司 | It is identified using structure light image |
| CN108696682B (en) * | 2018-04-28 | 2019-07-09 | Oppo广东移动通信有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108696682A (en) | 2018-10-23 |
| CN110191266A (en) | 2019-08-30 |
| CN108696682B (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110191266B (en) | Data processing method, apparatus, electronic device, and computer-readable storage medium | |
| CN108419017B (en) | Method and device for controlling shooting, electronic equipment and computer-readable storage medium | |
| WO2019205742A1 (en) | Image processing method, apparatus, computer-readable storage medium, and electronic device | |
| CN111523499B (en) | Image processing method, device, electronic device, and computer-readable storage medium | |
| CN108769509A (en) | Method, device, electronic device and storage medium for controlling camera | |
| WO2019196683A1 (en) | Method and device for image processing, computer-readable storage medium, and electronic device | |
| CN110971836B (en) | Method, apparatus, electronic device, and computer-readable storage medium for controlling shooting | |
| WO2019206020A1 (en) | Image processing method, apparatus, computer-readable storage medium, and electronic device | |
| CN108804895A (en) | Image processing method, device, computer-readable storage medium, and electronic device | |
| US11050918B2 (en) | Method and apparatus for performing image processing, and computer readable storage medium | |
| CN108833887B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| WO2019205887A1 (en) | Method and apparatus for controlling photographing, electronic device, and computer readable storage medium | |
| CN108573170A (en) | Information processing method and device, electronic device, computer-readable storage medium | |
| US20200151428A1 (en) | Data Processing Method, Electronic Device and Computer-Readable Storage Medium | |
| EP3621294B1 (en) | Method and device for image capture, computer readable storage medium and electronic device | |
| CN108924421B (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| CN108846310A (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN108810516B (en) | Data processing method, apparatus, electronic device, and computer-readable storage medium | |
| CN108833885A (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| CN108965716B (en) | Image processing method and apparatus, electronic device, computer-readable storage medium | |
| CN108986153B (en) | Image processing method and apparatus, electronic device, computer-readable storage medium | |
| CN109064503B (en) | Image processing method and apparatus, electronic device, computer-readable storage medium | |
| CN108881712A (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| HK40025202A (en) | Image processing method and apparatus, and electronic device and computer-readable storage medium | |
| HK40024500B (en) | Method and device for image capture, computer readable storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |