CN106375642B - Image acquisition and processing device and object motion image acquisition system - Google Patents

Abstract

The invention relates to an image sensing device, an acquisition processing device and an object moving image acquisition system. In order to solve the shortcomings of the traditional object moving image acquisition system, such as high cost, complex system structure, and complicated signal processing process, the object moving image acquisition system, It includes an image sensing device and an image acquisition and processing device; the image sensing device converts the captured image into an LVDS signal and outputs it to the image acquisition and processing device for image processing. The image sensing device of the present invention directly converts the optical signal into a low-voltage differential signal and outputs it to the image acquisition and processing device for processing, the signal transmission process is simple, and no encapsulation and decapsulation are required. The image sensing device adopts the global shutter to collect the image of the high-speed moving object, which is clear without smear and easy to identify. The image acquisition and processing device adopts a single chip including FPGA and ARM functions, has a simple structure, and realizes the integration of image acquisition and processing.

Description

Image acquisition and processing device and object motion image acquisition system

technical field

The invention relates to the field of image acquisition, and more specifically relates to an image sensing device, an acquisition processing device and an object moving image acquisition system.

Background technique

Traditional object motion image acquisition systems have disadvantages such as high cost, complex architecture, and complex signal processing. For example, the existing patent CN105611270A discloses a binocular autostereoscopic display system, including a binocular camera, an interface conversion circuit, an FPGA acceleration circuit and an autostereoscopic display. The binocular camera includes an image processing board, a sensor board and a lens. In practice, the purchase of cameras to assemble the system would incur significant expense. The signal processing flow of the system is as follows: first convert the original signal to HDMI signal, and then convert the HDMI signal to LVDS signal through the conversion circuit. The medium includes both HDMI cable and LVDS cable. The whole signal processing process is more complicated. The existing patent CN104835163A discloses a high-speed binocular vision system, including: two high-speed cameras, a binocular bracket, an image acquisition and processing board, an embedded human-computer interaction board, etc. Its image acquisition and processing board is used to calculate image data, including two main chips: FPGA and DSP, which are used for parallel computing and serial computing respectively. Its disadvantage is that there are too many processors in the whole system. Although the embedded human-computer interaction board is used to replace the PC to complete the human-computer interaction, its processing speed will still be limited. In practical applications, its processing capacity can only achieve 200fps, 640*240 pixel image acquisition. To sum up, it is very necessary to provide a low-cost, simple-structure, and high-efficiency image processing object moving image acquisition system.

Contents of the invention

The technical problem to be solved by the present invention is to provide an image sensing device, a collection and processing device, and an object motion image collection system for the above-mentioned defects of the prior art.

The technical solution adopted by the present invention to solve the technical problem is to construct an image sensing device, including: a lens for shooting; a photoelectric converter arranged at the imaging position of the lens; and a communication interface for transmitting data; The light reflected by the photographed object is focused by the lens to form an image that falls on the photoelectric converter, and the photoelectric converter converts the image into an electrical signal and outputs it to the communication interface.

Preferably, it also includes a power circuit connected to the photoelectric converter, the communication interface is also used to connect to an external power supply device, and the power circuit is connected to the communication interface.

The present invention also provides an image acquisition and processing device, including an HDMI interface, an FPGA acquisition unit and an ARM processing unit;

The FPGA acquisition unit is connected to an external image sensing device through the HDMI interface, uses the LVDS protocol to exchange image data with the external image sensing device, and outputs the image data to an ARM processing unit;

The ARM processing unit receives and processes the image data, and outputs the processed image data.

Preferably, the FPGA acquisition unit includes a synchronous clock configuration module, a general control bus module, an LVDS image signal receiving module, a write memory module and a write bus module;

The synchronous clock configuration module configures the polarity and phase of the synchronous clock of the external image sensing device with the SPI serial protocol through the HDMI interface;

The general control bus module is connected with the synchronous clock configuration module, the LVDS image signal receiving module and the ARM processing unit, and is used to configure the synchronous clock polarity and phase of the synchronous clock configuration module and the LVDS image signal receiving module, and with ARM processing unit for low-speed data transmission;

The LVDS image signal receiving module receives the image data output by the external image sensing device and its corresponding synchronization code through the HDMI interface, and performs serial-to-parallel conversion on the image data and outputs it to the write memory module;

The write memory module caches the image data after serial-to-parallel conversion to an internal memory, and then reads the image data from the internal memory and outputs it to the write bus module;

The write bus module is connected to the ARM processing unit for high-speed data transmission with the ARM processing unit.

Preferably, the ARM processing unit includes a data processing module, a common bus interface, a first high-speed bus interface and an Ethernet module;

The common bus interface is connected to the general control bus module, and the first high-speed bus interface is connected to the write bus module;

The data processing module is connected to the common bus interface and the first high-speed bus interface;

The Ethernet module is connected with the data processing module for connecting to an external computer, and the processed image data is transmitted to the external computer through the TCP protocol.

Preferably, it also includes a data storage unit connected to the data processing module for storing images and program data, and the data storage unit includes SDRAM and FLASH flash memory.

Preferably, the ARM processing unit further includes a second high-speed bus interface.

Described FPGA acquisition unit also comprises: read bus module, read memory module, timing state generation module and TMDS signal conversion module;

The read bus module is connected to the second high-speed bus interface for high-speed data transmission with the ARM processing unit;

The read memory module reads the image data in the data storage unit through the read bus module and the second high-speed bus interface;

The timing state generation module is connected to the memory read module, and is used to generate a standard timing corresponding to the resolution, and output the image data to the TMDS signal conversion module according to the generated standard timing;

The TMDS signal conversion module is used for converting the image data into TMDS signals and outputting them to an external display for display.

The present invention also provides an object motion image acquisition system, including an image sensing device and an image acquisition and processing device;

The image sensing device converts the captured image into an LVDS signal and outputs it to the image acquisition and processing device for image processing.

Preferably, the system is a binocular vision system, comprising two image sensing devices, an image acquisition and processing device, and a plurality of HDMI connection lines; the communication interfaces of the two image sensing devices are respectively connected through independent HDMI connection lines It is connected with the HDMI interface of the image acquisition and processing device.

Preferably, a supplementary light device for improving the anti-interference performance of ambient light of the two image sensing devices is also included.

The image sensing device, acquisition processing device and object moving image acquisition system implementing the present invention have the following beneficial effects: the image sensing device directly converts the optical signal into a low-voltage differential signal and outputs it to the image acquisition processing device for processing, and the signal transmission process is simple , without encapsulation and decapsulation. The image sensing device adopts the global shutter to collect the image of the high-speed moving object, which is clear without smear and easy to identify. The image acquisition and processing device adopts a single chip including FPGA and ARM functions, has a simple structure, and realizes the integration of image acquisition and processing. The entire object moving image acquisition system has the advantages of low cost, simple structure and high processing efficiency.

Description of drawings

The present invention will be further described below in conjunction with accompanying drawing and embodiment, in the accompanying drawing:

Fig. 1 is a structural block diagram of an image sensing device of the present invention;

Fig. 2 is a structural block diagram of an image acquisition and processing device of the present invention;

Figure 3 is a structural block diagram of the binocular vision system.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, the image sensing device of the present invention includes: a lens for shooting; a photoelectric converter arranged at the imaging position of the lens; and a communication interface for transmitting data; the light reflected by the object to be photographed is focused by the lens The formed image falls on the photoelectric converter, and the photoelectric converter converts the image into an electrical signal and outputs it to the communication interface.

Concrete, the shooting lens of the present invention selects 12mm focal length for use, aperture f1.0, the lens of 1/3 inch, photoelectric converter adopts CMOS global shutter sensor, concrete model is NOIP1SN0300A of ON Semiconductor, by the combination of above-mentioned lens and CMOS sensor, Can achieve 725 frames, 640*480 pixel image acquisition. When controlling the acquisition of a single image and the detection time is less than 5 milliseconds, it is possible to monitor the motion state of an ultra-high-speed target (moving speed within the range of 50m/s to 100m/s) in real time, while retaining the low-speed target (moving speed less than 2m) /s) detection function. It has reached the detection system requirements for comprehensive coverage of medium, high and low speed moving objects.

The electrical signal converted by the sensor is a low-voltage differential signal, that is, an LVDS signal. The communication interface is a mini-HDMI interface, that is, a Type-C HDMI interface. The communication interface exchanges data with external devices through LVDS protocol.

Further, the image sensing device of the present invention further includes: a power circuit connected to the photoelectric converter. The image sensing device is connected to an external power supply through a power supply circuit, and the power supply circuit converts the external power supply into a voltage required by the photoelectric converter.

In some embodiments, the communication interface is also used to connect to an external power supply, and the power supply circuit is connected to the communication interface. When the communication interface is a mini-HDMI interface, it is a type-C HDMI interface. This type of interface has a total of 19 pins, and its 18th pin is +5V power supply, which can be used for 5V voltage supply. CMOS image sensing devices are designed for 5V or 3.3V supply voltages. Therefore, a corresponding external power supply device can be connected through the mini-HDMI interface to supply power for the entire image sensing device.

As shown in Figure 2, the present invention also provides a kind of image acquisition and processing device, comprises HDMI interface, FPGA acquisition unit and ARM processing unit; FPGA acquisition unit connects external image sensor device by HDMI interface, uses LVDS agreement and external image sensor The device exchanges image data, and outputs the image data to the ARM processing unit; the ARM processing unit receives and processes the image data, and outputs the processed image data.

FPGA and ARM are integrated in one ZYNQ chip. The chip adopts the XC7Z020-1CLG484C model of Xilinx Company. After the FPGA acquisition unit is connected to the image sensing device, it is configured with a synchronous clock, and the two exchange data under the control of the corresponding clock after configuration. The LVDS signal is a serial signal, and the FPGA converts it serially to parallel to generate effective image data, and then outputs it to the ARM processing unit, which processes the image data, including 3D reconstruction and motion parameter calculation, etc., and outputs the processed image data to obtain the result of. Its output includes: output to an external computer, data storage unit and output to FPGA for displaying data. The specific output process will be described in detail below.

Further, the corresponding modules that perform the above synchronous clock configuration, data transmission, conversion and output in FPGA include: synchronous clock configuration module, general control bus module, LVDS image signal receiving module, write memory module and write bus module; synchronous clock configuration module Configure the polarity and phase of the synchronous clock of the external image sensing device with the SPI serial protocol through the HDMI interface; the general control bus module is connected with the synchronous clock configuration module, the LVDS image signal receiving module and the ARM processing unit for configuring the synchronous clock configuration module, Synchronous clock polarity and phase of the LVDS image signal receiving module, and low-speed data transmission with the ARM processing unit; the LVDS image signal receiving module receives the image data output by the external image sensing device and its corresponding synchronization code through the HDMI interface, and The image data is serial-to-parallel converted and then output to the write memory module; the write memory module caches the image data after the serial-to-parallel conversion to the internal memory, and then reads the image data from the internal memory and outputs it to the write bus module; the write bus module communicates with the ARM Processing unit connection for high-speed data transfer with the ARM processing unit.

The synchronous clock configuration includes: the output clock configuration of the FPGA and the clock configuration of the FPGA to the image sensing device.

The general control bus module is used to read and configure the internal registers of the FPGA: the synchronous clock configuration module and the LVDS image signal receiving module. The output clock configuration of the FPGA is the clock configuration of the general control bus module, which is implemented when the FPGA system is programmed. The specific principles of clock configuration and register configuration are well known to those skilled in the art, and will not be repeated in the present invention.

Clock configuration of the FPGA to the image sensing device: the synchronous clock configuration module configures the polarity and phase of the synchronous clock of the external image sensing device with the SPI serial protocol through the HDMI interface. Specifically, the communication between two SPI devices must be controlled by the master device to the slave device. The clock of the slave device is provided by the master device through the corresponding pin, and the slave device itself cannot generate or control the clock. The FPGA generates corresponding clock pulses according to the data to be exchanged. The clock pulses form a clock signal. The clock signal controls when to exchange data between the FPGA and the image sensing device and when to process the received data through the clock polarity and clock phase. Sampling is performed to ensure synchronous transfer of data between the two devices.

After the clock configuration is completed, the image sensing device and the LVDS image signal receiving module perform data transmission. The LVDS image signal receiving module receives the image data output by the image sensor device as LVDS signal and the corresponding synchronization code, generates the line and field synchronization signals of the image sensor device, and performs serial-to-parallel conversion on the LVDS signal to generate effective image data.

The LVDS image signal receiving module outputs the generated effective image data to the writing memory module, and the writing memory module buffers the received image data into the internal memory FIFO, then takes out the image data in the FIFO, and inputs them to the ARM for processing through the writing bus module unit.

Further, the ARM processing unit includes a data processing module, an ordinary bus interface, a first high-speed bus interface and an Ethernet module; the ordinary bus interface is connected to the general control bus module, and the first high-speed bus interface is connected to the write bus module; the data processing module and the ordinary bus The interface is connected to the first high-speed bus interface; the Ethernet module is connected to the data processing module for connecting to an external computer, and the processed image data is transmitted to the external computer through the TCP protocol.

Further, the ARM processing unit also includes a data storage unit connected to the data processing module for storing images and program data, and the data storage unit includes SDRAM and FLASH flash memory. SDRAM is used to temporarily store the processed image data, and FLASH flash memory is used to store the solidified reset program.

The ARM processing unit performs low-speed data transmission with FPGA through the ordinary bus interface; high-speed data reception through the first high-speed bus interface; the data processing module will perform three-dimensional reconstruction and motion parameter calculation on the image data input by the first high-speed bus interface, and process The obtained data is output to an external computer through the Ethernet module. Users can directly view the processing results through the computer. The processor used for image data calculation in the ARM in the present invention is a Cortex-A9 dual-core processor.

The data processing module can also directly output the processing results to SDRAM for storage, and can also be read from SDRAM and output to an external computer or FPGA through the Ethernet module when needed.

The data processing module can also output the processed contacts to the FPGA for displaying the data. Correspondingly, the ARM processing unit further includes a second high-speed bus interface. The FPGA acquisition unit also includes: a read bus module, a read memory module, a timing state generation module and a TMDS signal conversion module; the read bus module is connected to the second high-speed bus interface for high-speed data transmission with the ARM processing unit; the read memory module passes The read bus module and the second high-speed bus interface read the image data in the data storage unit; the timing state generation module is connected with the read memory module to generate a standard timing corresponding to the display resolution, and convert the image data according to the generated standard The timing is output to the TMDS signal conversion module; the TMDS signal conversion module is used to convert the image data into a TMDS signal and then output it to an external display for display.

Specifically, the ARM processing unit outputs the image data stored in the SDRAM to the read memory module through the second high-speed bus interface and the read bus module of the FPGA. The read memory module caches the obtained image data in the internal memory FIFO, and outputs the data from the FIFO to the timing state generation module. The timing state generation module generates standard timing according to the resolution that needs to be displayed, and converts the image data according to the standard timing. The drive is output to the TMDS signal conversion module and converted into a TMDS standard signal. The FPGA can be connected to an external display through the corresponding HDMI interface, and output the TMDS standard signal to the external display for image display. Among them, the timing state generating module is composed of a counter and two state machines, the row counter and the row state machine are used to generate the row synchronization timing, and the field counter and the field state machine are used to generate the field synchronization timing.

The present invention also provides a system for collecting moving images of objects, including an image sensing device and an image collecting and processing device; the image sensing device converts captured images into LVDS signals and outputs them to the image collecting and processing device for image processing.

The structure and functional characteristics of the image sensing device and image acquisition and processing device in this system are the same as those of the above image sensing device and image acquisition and processing device, and will not be repeated here.

As shown in Figure 3, the system is a binocular vision system, including two image sensing devices, an image acquisition and processing device, and multiple HDMI connection lines; the communication interfaces of the two image sensing devices are respectively connected through independent HDMI connection lines Connect with the HDMI interface of the image acquisition and processing device.

Further, each image sensing device is provided with two mini-HDMI interfaces, and the image acquisition and processing device is correspondingly provided with four HDMI interfaces to connect with it.

The system can be completely packaged in the mechanical shell, and the corresponding external network interface and power interface are provided on the mechanical shell. The power interface is connected to the image acquisition and processing device, and the image acquisition and processing device is connected to the image sensing device through the HDMI interface. The device can supply power to the image sensing device through the power supply pin of the interface. Preferably, the power supply of the system is 12V, 3A.

Further, the system also includes a supplementary light device for improving the anti-interference performance of the image sensing device against ambient light. The supplementary light device of the present invention preferably selects three tungsten-halogen lamps with an input power of 12V and an input power of 50W.

The image sensing device, acquisition processing device and object moving image acquisition system implementing the present invention have the following beneficial effects: the image sensing device directly converts the optical signal into a low-voltage differential signal and outputs it to the image acquisition processing device for processing, and the signal transmission process is simple , without encapsulation and decapsulation. The image sensing device adopts the global shutter to collect the image of the high-speed moving object, which is clear without smear and easy to identify. The image acquisition and processing device adopts a single chip including FPGA and ARM functions, has a simple structure, and realizes the integration of image acquisition and processing. The entire object moving image acquisition system has the advantages of low cost, simple structure and high processing efficiency.

It can be understood that the above examples only express the preferred implementation of the present invention, and its description is relatively specific and detailed, but it should not be interpreted as limiting the patent scope of the present invention; it should be pointed out that for those of ordinary skill in the art In other words, on the premise of not departing from the concept of the present invention, the above-mentioned technical features can be freely combined, and some modifications and improvements can also be made, all of which belong to the protection scope of the present invention; All equivalent transformations and modifications should fall within the scope of the claims of the present invention.

Claims (8)

1. a kind of image acquisition and processing device, which is characterized in that including HDMI interface, FPGA acquisition unit and ARM processing are single Member；

The FPGA acquisition unit connects external image sensing device by the HDMI interface, uses LVDS agreement and described outer Portion's image sensing device exchange image data, and described image data are exported to ARM processing unit；

The ARM processing unit receives, processing image data, and the image data that exports that treated；

The FPGA acquisition unit includes synchronised clock configuration module, general controls bus module, LVDS picture signal reception mould Block writes memory modules and write bus module；

The synchronised clock configuration module configures the external image sensing device by the HDMI interface with SPI serial protocol Synchronised clock polarity and phase；

At the general controls bus module and the synchronised clock configuration module, LVDS picture signal receiving module and ARM Manage unit connection, for configure the synchronised clock configuration module, LVDS picture signal receiving module synchronised clock polarity and Phase, and slow data transmission is carried out with ARM processing unit；The general controls bus module is by reading, configuring FPGA Internal register with synchronised clock configuration module and LVDS picture signal receiving module；

The LVDS picture signal receiving module receives the figure of the external image sensing device output by the HDMI interface Memory modules are write to described as data and its corresponding synchronous code, and to output after described image data progress serioparallel exchange；

The memory modules of writing are by the described image data buffer storage after serioparallel exchange to internal storage, then from the storage inside Described image data are read in device and are exported to write bus module；

The write bus module is connect with the ARM processing unit, for carrying out high-speed data biography with the ARM processing unit It is defeated.

2. image acquisition and processing device according to claim 1, which is characterized in that the ARM processing unit includes data Processing module, common status bus interface, the first high speed bus interface and ethernet module；

The common status bus interface connects the general controls bus module, writes described in the first high speed bus interface connection total Wire module；

The data processing module is connect with the common status bus interface, first high speed bus interface；

The ethernet module is connect with the data processing module, will be described by Transmission Control Protocol for connecting outer computer Image data that treated is transmitted to the outer computer.

3. image acquisition and processing device according to claim 2, which is characterized in that further include and the data processing module Connection, for storing the data storage cell of image, program data, the data storage cell includes that SDRAM and FLASH dodges It deposits.

4. image acquisition and processing device according to claim 3, which is characterized in that the ARM processing unit further includes Two high speed bus interfaces.

The FPGA acquisition unit further include: read bus module, rdma read module, time sequence status generation module and TMDS signal Conversion module；

The read bus module is connected with second high speed bus interface, for carrying out high speed number with the ARM processing unit According to transmission；

The rdma read module reads the data by the read bus module and second high speed bus interface and stores list Image data in member；

The time sequence status generation module is connected with the rdma read module, when for generating standard corresponding with resolution ratio Sequence, and described image data are exported according to the standard time sequence of generation to the TMDS signal conversion module；

Output to external display is shown after the TMDS signal conversion module is used to being converted to described image data into TMDS signal Show.

5. a kind of object of which movement image capturing system, which is characterized in that including image sensing device and such as Claims 1 to 4 Described in any item image acquisition and processing devices；

Described image sensing device, comprising:

Camera lens for shooting；

The photoelectric converter of the lens imaging position is set；And

It is used for transmission the communication interface of data；

The image that the light of subject reflection is formed after the lens focus is fallen on the photoelectric converter, the light Electric transducer converts the image into electric signal and exports to the communication interface；

The electric signal is LVDS signal, and the communication interface is included HDMI interface and exchanged with LVDS agreement with external equipment Data.

Output to described image acquisition process fills after the image that shooting obtains is converted into LVDS signal by described image sensing device Set carry out image procossing.

6. object of which movement image capturing system according to claim 5, which is characterized in that described image sensing device also wraps The power circuit being connected with the photoelectric converter is included, the communication interface is also used to connect external power supply, the power supply Circuit is connected with the communication interface.

7. object of which movement image capturing system according to claim 5, which is characterized in that the system is binocular vision system System, including two image sensing devices, an image acquisition and processing device and a plurality of HDMI connecting line；Described two images pass The communication interface of induction device is connected by independent HDMI connecting line with the HDMI interface of described image acquisition processing device respectively.

8. object of which movement image capturing system according to claim 7, which is characterized in that further include for improving the figure As the light compensating apparatus of sensing device environment resistant light jamming performance.