JP2009077195A - Imaging apparatus, method of controlling output signal, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve a configuration capable of outputting data to the external at a low transmission frequency in an imaging apparatus having an imaging part for outputting a plurality of signals corresponding to divided areas. <P>SOLUTION: A multiplexer selectively reads out data from a plurality of line memories in which data corresponding to divided areas of the imaging part are stored and outputs the read data in the array order of pixels composing a frame image. The multiplexer inputs the data of a left image storing line memory in the order from a left image to a right image by first-in first-out (FIFO) system and outputs data of a right image storing line memory in the order from a right pixel to a left pixel by first-in last-out (FILO) system and outputs the data. An output control part outputs the pixel value data of a plurality of pixels selected in accordance with an array of frame image composing pixels in parallel to a transmission cable. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an output signal control method, and a computer program. More specifically, the present invention relates to an imaging device that generates an image signal for external output to be output from the imaging device to an external device that executes display processing and the like, an output signal control method, and a computer program.

  For example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) is widely used as an imaging device in a video camera or a still camera. For example, a conventional general CCD receives image information for one screen by a large number of photo detectors (PDs), reads charge signals obtained by photoelectric conversion by a vertical register and a horizontal register, and converts them into a data stream. And output from one output channel. Such a one-channel output type CCD and a signal processing configuration will be described with reference to FIG.

  A CCD 10 shown in FIG. 1 has a vertical register 11 that transfers charges accumulated in a photodetector (PD) serving as an image pickup device constituting the CCD in the vertical direction, and charges transferred by the vertical register 11 line by line in the horizontal direction. One horizontal register 12 to be transferred and an output amplifier 13 for converting the charge of the horizontal register 12 into a voltage are provided, and the output of the output amplifier 13 is input to the signal processing unit 21.

  The signal processing unit 21 includes a CDS circuit that performs noise removal from an input signal, an AGC circuit that performs gain adjustment, an AD conversion unit that performs AD conversion, and the like. The digital signal subjected to the signal processing in the signal processing unit 12 is accumulated in the line memory 22 and then output via the output unit 23 to obtain, for example, an output image 30 shown in the figure.

  In recent years, with the demand for high-speed signal processing and the increase in the number of CCD constituent pixels, a configuration has been proposed in which the output of the CCD is divided into a plurality of parts, and the outputs are combined and output in parallel.

  A two-channel output compatible CCD and signal processing configuration will be described with reference to FIG. FIG. 2 shows a camera module 40 that captures an image, a display module 80 that performs processing for receiving an output signal from the camera module 40 and displaying it on a monitor 82, and a transmission cable 70 that transmits a signal from the camera module 40. Is shown.

  The CCD 50 in the camera module 40 shown in FIG. 2 includes a vertical register 51 that transfers charges accumulated in the photodetector (PD) constituting the CCD in the vertical direction, and one line in the horizontal direction that is transferred by the vertical register 51. Two horizontal registers 52 and 53 for transferring data one by one are provided. The first horizontal register 52 receives the output of the left half of the photodetector constituting the CCD, and the second horizontal register 53 receives the output of the right half.

  The accumulated data in the first horizontal register 52 is converted into voltage by the output amplifier 54 and input to the signal processing unit 61. Further, the accumulated data of the second horizontal register 53 is converted into a voltage by the output amplifier 55 and input to the signal processing unit 62. The two signal processing units 61 and 62 each process in parallel the output data of half of the constituent pixels of the CCD. This process realizes high-speed processing.

  The signal-processed data is respectively input to the line memories 63 and 64, synthesized by the multiplexer 65, adjusted in the left / right image balance by the image correction processing unit 66, and outputted through the output unit 67. For example, each pixel is output as 12-bit digital data. An example of image signal transmission processing via a transmission cable is described in, for example, Japanese Patent Application Laid-Open No. 2007-116734.

  An output signal from the camera module 40 is input to the display control unit 81 of the display module 80 via the transmission cable 70. The display control unit 81 is configured by, for example, a PC and generates an image to be displayed on the monitor 82 based on a signal input via the transmission cable 70. The display control unit 81 performs a process of generating a display image of one screen by executing a synthesis process of the left area image data and the right area image data input via the transmission cable 70.

  There are various forms of signal transmission from the camera module 40 via the transmission cable 70. For example, in the case of a camera module for factory automation (FA) use, the captured image is not compressed in the transmission cable 70. In this case, the transmission block is divided and transmitted for each imaging region.

FIG. 3 shows an example of the transmission cable 70. As shown in FIG. 3, the transmission cable has 24 lines each transmitting 1-bit data, and each is connected to the following ports.
Ports A0 to A7,
Ports B0-B7,
Ports C0-C7,
Data indicating the pixel value level of the pixels constituting the image is transmitted through these ports.

  The transmission cable 70 shown in FIG. 3 can transmit 8-bit data at ports A0 to A7, and can transmit parallel data of 24 bits in total, 8 bits at ports B0 to B7 and 8 bits at ports C0 to C7. is there. An example of data transmission when transmission blocks are transmitted separately for each imaging area will be described with reference to FIG. FIG. 4 is a diagram for explaining the left image data and the right image data which are taken by the CCD 50 shown in FIG. Each pixel is set to 12-bit data, and the 12-bit data is sequentially transmitted via the transmission cable 70.

As shown in FIG.
(1) Ports A0 to A7 and B0 to B3 are used for transmission of pixel value data of the left image.
(2) Ports B4 to A7 and C0 to C7 are used to transmit pixel value data of the right image.
Thus, for each of the left image and the right image, 12-bit data indicating a pixel value in units of one pixel can be transmitted in parallel. As described above, the pixel value data of the left image and the right image are input to the display control unit 81 of the display module 80 via the transmission cable 70.

  The display control unit 81 of the display module 80 acquires pixel value data for each image region, combines the images divided into regions into one image, generates display data, and displays the display data on the monitor 82. Applied to system analysis.

  However, in such a data transmission configuration, the display control unit 81 of the display module 80 needs to perform a process of combining a plurality of areas into one, and there is a problem that the processing load on the display control unit 81 increases. is there. Particularly in factory automation (FA) applications, the PC used as the display control unit 81 is often used as a host computer for controlling the entire factory automation (FA) system, and computer resources are used to generate display data. This may cause a problem that a delay occurs in system control processing other than display control.

  In order to reduce the processing load on the display control unit 81 of the display module 80, a method of executing and transmitting a process of combining the divided images captured in a plurality of areas into an image in one area on the camera module 40 side. However, with this method, pixel value data for one pixel is sequentially transmitted according to the arrangement of the constituent pixels of the image frame, and data for two pixels is transmitted in parallel. The transmission frequency must be increased compared to As a result, there arises a problem that the possibility of occurrence of transmission loss due to the skin effect of the signal in data transmission from the camera module 40 to the display module 80 via the transmission cable 70 increases. For example, specifically, there arises a problem that the transmission distance cannot be extended.

In addition, when data transmission is performed at a high transmission frequency, the display control unit 81 on the display module 80 side also needs to be provided with a module for receiving and processing a high-speed signal, resulting in a problem of cost increase.
JP 2007-116734 A

  The present invention has been made in view of such problems, and outputs an output signal from an image sensor such as a CCD or CMOS having a plurality of divided outputs to an external device that performs display control, for example, via a transmission cable. In an imaging apparatus that outputs, an imaging apparatus that realizes high-quality data transmission by reducing the load on an external device that receives output data and performs display control and the like, and reduces data transmission problems such as transmission loss, and output It is an object to provide a signal control method and a computer program.

The first aspect of the present invention is:
An imaging unit that outputs a plurality of output signals corresponding to each of the divided regions of the imaging element;
A signal processing unit that performs signal processing on an output signal from the imaging unit and generates digital image data corresponding to each of the divided regions;
An output signal generator for generating an output signal to be externally input by inputting the generation data of the signal processing unit;
The output signal generator is
A plurality of line memories for individually storing digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing unit;
A multiplexer that selectively reads out the divided region correspondence data stored in the plurality of line memories and outputs the data in the order of arrangement of the frame image constituent pixels;
An imaging apparatus comprising: an output control unit that receives the output data of the multiplexer and outputs in parallel the pixel value data of a plurality of pixels selected according to the arrangement of the frame image constituent pixels to a transmission cable.

  Furthermore, in an embodiment of the imaging apparatus of the present invention, the imaging unit has an imaging device that is divided into a left image and a right image, and outputs a plurality of output signals corresponding to the left image and the right image, respectively. The plurality of line memories include two line memories that individually store digital image data corresponding to the left image and the right image, respectively, and the multiplexer includes the left image and the right image stored in the two line memories. The configuration is characterized in that data corresponding to each image is selectively read out and output in the order of the constituent pixels of the frame image.

  Furthermore, in one embodiment of the imaging device of the present invention, the left image storage line memory for storing the left image data in the two line memories has the pixel value data of each line of the left image in the order of the left pixel to the right pixel. The right image storage line memory for storing and storing the right image data is configured to store pixel value data of each line of the right image in order from the right pixel to the left pixel, and the multiplexer stores in the left image storage line memory After inputting the pixel value data from the left pixel to the right pixel in the first-in first-out method (FIFO), the data stored in the right image storage line memory is transferred from the right pixel to the left pixel by the first-in last-out method (LIFO). The present invention is characterized in that a process of inputting and outputting pixel value data in order is executed.

  Furthermore, in one embodiment of the imaging apparatus of the present invention, the multiplexer executes data input processing from the plurality of line memories in a clock cycle faster than a clock cycle of data writing processing to the plurality of line memories. It is characterized by being.

  Furthermore, in an embodiment of the imaging apparatus of the present invention, the output control unit sequentially selects and transmits pixel value data of two consecutive pixels according to the arrangement of frame image constituent pixels as a set of parallel output data. It is the structure which performs the process output to a cable, It is characterized by the above-mentioned.

  Furthermore, in one embodiment of the imaging device of the present invention, the imaging device further includes different transmission modes of data output in units of one pixel according to a high transmission frequency and data output in units of two pixels according to a low transmission frequency. It is characterized in that it has a configuration capable of switching data output at.

  Furthermore, in one embodiment of the imaging apparatus of the present invention, the output signal generation unit further includes an image correction processing unit that inputs the output data of the multiplexer and performs image correction, and the output control unit includes: The output data of the image correction processing unit is input, and a process of executing parallel output of pixel value data of a plurality of pixels selected according to the arrangement of the frame image constituent pixels to a transmission cable is performed.

Furthermore, the second aspect of the present invention provides
An output signal control method for executing output processing of image data acquired in an imaging device,
A signal processing step in which a signal processing unit performs signal processing on an output signal from the imaging unit that outputs a plurality of output signals corresponding to each of the divided regions of the image sensor, and generates digital image data corresponding to each of the divided regions; ,
The output signal generation unit has an output signal generation step of generating an output signal to be output to the outside by inputting the generation data of the signal processing unit,
The output signal generation step includes
Individually storing digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing unit in a line memory;
A data synthesis step in which the multiplexer selectively reads out the divided region corresponding data stored in the plurality of line memories and outputs the data in the order of arrangement of the frame image constituent pixels;
An output signal control method comprising: a data output step in which an output control unit inputs output data of the multiplexer and outputs pixel value data of a plurality of pixels selected in accordance with an arrangement of frame image constituting pixels to a transmission cable in parallel. It is in.

Furthermore, the third aspect of the present invention provides
A computer program that executes output control of image data acquired in an imaging device,
A signal processing step of causing the signal processing unit to perform signal processing on the output signals from the imaging unit that outputs a plurality of output signals corresponding to each of the divided regions of the imaging element, and generate digital image data corresponding to each of the divided regions. When,
The output signal generation unit has an output signal generation step of generating an output signal to be output to the outside by inputting the generation data of the signal processing unit,
The output signal generation step includes
Individually storing digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing unit in a line memory;
A data synthesis step in which the multiplexer selectively reads out the divided region corresponding data stored in the plurality of line memories and outputs the data in the order of arrangement of the frame image constituent pixels;
An output control unit has a data output step for inputting the output data of the multiplexer and outputting in parallel the pixel value data of a plurality of pixels selected according to the arrangement of the frame image constituting pixels to a transmission cable. is there.

  The computer program of the present invention is, for example, a computer program that can be provided by a storage medium or a communication medium provided in a computer-readable format to a general-purpose computer system that can execute various program codes. . By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of one embodiment of the present invention, in the configuration in which an output signal corresponding to each of the divided areas of the image sensor, for example, the left and right divided areas is input and output externally, the digital image data corresponding to each divided area is individually output. The divided region correspondence data stored in the plurality of line memories and selectively stored in the line memory by the multiplexer are selectively read out and output in the arrangement order of the frame image constituent pixels. The multiplexer inputs the data stored in the left image storage line memory in the order of the left pixel to the right pixel by the first-in first-out method (FIFO), and then stores the data stored in the right image storage line memory in the first-in last-out method (LIFO). ) To input and output pixel value data in order from the right pixel to the left pixel. The output control unit inputs the output data and outputs the pixel value data of a plurality of pixels selected according to the arrangement of the frame image constituent pixels in parallel to the transmission cable. With this configuration, data transmission via the transmission cable can be performed, for example, in units of two pixels, and transmission loss due to the skin effect or the like can be suppressed without increasing the transmission frequency, for example, long transmission. Stable data transmission is also realized on the line. Also, processing at a high frequency is not required on the data receiving side, and the cost of the receiving side device can be reduced.

  The details of the imaging apparatus, output signal control method, and computer program of the present invention will be described below with reference to the drawings. First, with reference to FIG. 5, a configuration example of an imaging apparatus according to an embodiment of the present invention will be described.

  In FIG. 5, an imaging device (camera module) 100 that captures an image, a display module 200 that receives an output signal from the imaging device 100 and displays an image on the monitor 202, and a signal from the imaging device 100 are shown. A transmission cable 180 for transmission is shown.

  The CCD 110 in the imaging apparatus 100 shown in FIG. 5 is divided into a plurality of parts and has an output structure for each divided region, similar to the structure of the CCD described above with reference to FIG. A detailed configuration of the CCD 110 will be described with reference to FIG. In the following embodiments, a configuration example using a CCD as an image sensor will be described. However, the present invention is not limited to a CCD, and can be applied to a case where a CMOS is used as an image sensor.

  As shown in FIG. 6, the CCD 110 has a large number of photodetectors (PD) 118 that are photoelectric conversion elements, and is configured to output a voltage signal based on the electric charge accumulated in these photodetectors (PD) 118. The CCD 110 divides an imaging area for one screen into two at the center in the horizontal direction, and outputs pixel information from different channels. The CCD 110 has a vertical register 111 and horizontal registers 112 and 113 for one line. The vertical register 111 is a register that transfers charges accumulated in the photodetector (PD) 118 in the vertical direction in units of one line.

  The horizontal registers 112 and 113 transfer charges for one line transferred from the vertical register 111 in units of pixels in the horizontal direction, and input the charge information to output amplifiers 114 and 115 that convert and amplify the voltage information. . The output amplifiers 114 and 115 output charge information corresponding to the respective divided images as voltage signals. Thus, the image information generated by the photo detector (PD) 118 of the CCD 110 is output from two output channels via the two output amplifiers 114 and 115.

  That is, the first horizontal register 112 outputs a signal based on the charge information output from the photodetector (PD) 118 included in the left image area via the first output amplifier 114, and the second horizontal register 113 A signal based on the charge information output from the photodetector (PD) 118 included in the image area is output via the second output amplifier 115.

  Returning to FIG. 5, the processing of the output signals of these output amplifiers 114 and 115 will be described. The output of the first output amplifier 114 that is image signal information corresponding to the left image of the CCD 110 is input to the signal processing unit 121. The signal processing unit 121 includes a CDS circuit that performs noise removal from an input signal, an AGC circuit that performs gain adjustment, an AD conversion unit that performs AD conversion, and the like. For example, a 12-bit digital signal (0 (min) to 4095 (max)) is generated and stored in a line memory 132 in a DSP (Digital Signal Processor) 130.

  On the other hand, the output of the second output amplifier 115 which is image signal information corresponding to the right image of the CCD 110 is input to the signal processing unit 123. The signal processing unit 123 is also configured by a CDS circuit that removes noise in the input signal, an AGC circuit that performs gain adjustment, an AD conversion unit that performs AD conversion, and the like. Signal processing is performed by each of these circuits, for example, 12 bits. Digital signals (0 (min) to 4095 (max)) are generated and stored in a line memory 134 in a DSP (Digital Signal Processor) 130.

  The DSP 130 receives the digital signals generated by the signal processing unit 121 and the signal processing unit 123, generates an output digital signal to be output to the outside, and outputs the output digital signal to the display module 200 via the transmission cable 180. Thus, the DSP 130 functions as an output signal generation unit that generates a signal to be output to the outside. The control unit (MPU) 140 executes data transfer timing and sequence control in the DSP 130, further receives a feedback control signal from the display control unit 201 of the display module 200, and performs various processing controls on the DSP 130 according to the reception control signal. I do.

  The data stored in the line memory 132 corresponds to the image data of the left half of the CCD 110, and the data stored in the line memory 134 corresponds to the image data of the right half of the CCD 110. Of the images for one line stored in the line memories 132 and 134, the left image stored in the line memory 132 is output to the multiplexer (MUX) 135 according to the internal synchronization signal by the first-in first-out method (FIFO). On the other hand, the transfer start position and transfer end position of the right image stored in the line memory 134 are switched by a first-in last-out method (LIFO), and the transfer start position of the right image area comes next to the transfer end position of the left image area. To the multiplexer (MUX) 135.

  In the imaging apparatus of the present invention, at the time of output from the multiplexer (MUX) 135, the left image and the right image are set to be output as one connected image. Details of signal input / output in the multiplexer (MUX) 135 will be described later with reference to FIG.

  The output of the multiplexer (MUX) 135 is input to the image correction processing unit 136, and pixel value correction is executed. This correction is, for example, processing such as balance adjustment processing for the left and right images, interpolation processing for defective pixels caused by noise, and the like, for example, pixel value determination processing based on surrounding pixel values using a filter.

  The image data corrected by the image correction processing unit 136 is output to the output control unit 137, and the corrected image data is output to the display control unit (for example, PC) 201 of the display module 200 via the transmission cable 180.

  The output data from the output control unit 137 is output to the display control unit (for example, PC) 201 of the display module 200 via the transmission cable 180 without being compressed. In the imaging apparatus of the present invention, as described above, at the time of output from the multiplexer (MUX) 135, the setting is output as one image in which the left image and the right image are connected, and the output data from the output control unit 137 Is not configured to output the left image and the right image separately, but according to the scanning direction at the time of display processing, for example, a set of 2 pixels, 12-bit pixel value × 2 = 24-bit data, and transmission cable 180 To the display control unit (for example, PC) 201 of the display module 200. Details of the output signal will be described later.

  The transmission data output from the output control unit 137 includes a 24-bit video signal, a signal (LVAL) indicating an effective image in a horizontal period, a signal (FVAL) indicating an effective image in a vertical period, and a signal (FVAL) indicating an effective video period ( 4 bit data (LVDS (low voltage small amplitude differential signal)) and 1 bit of a serialized 28-bit signal that combines the 4-bit control signal of each of the pixel clock signal (STROBE) It becomes a clock (LVDS signal).

  An output signal from the imaging device (camera module) 100 is input to the display control unit 201 of the display module 200 via the transmission cable 180. The display control unit 201 is configured by a PC, for example. The display control unit 201 generates an image to be displayed on the monitor 202 based on a signal input via the transmission cable 180. Also, feedback control information for image correction and data transmission is generated and output to the MPU 140 of the imaging apparatus 100.

  As described above, since the imaging apparatus 100 of the present invention does not individually output the pixel data of the left and right images, the display control unit 201 uses the left region image data and the right region image data. It is not necessary to execute the synthesis process, and the processing load is reduced.

  The output control unit 137 of the imaging apparatus 100 of the present invention outputs the constituent pixels of one frame image in a preset sequence, not the data obtained by dividing the left image and the right image. That is, pixel value data of a plurality of pixels selected according to the arrangement of the frame image constituent pixels is output in parallel to the transmission cable. For example, 12 bits of pixel value × 2 = 24 bits data is output through the transmission cable 180 by taking two consecutive pixels as a set according to the arrangement of the frame image constituent pixels. Details of this processing will be described below with reference to FIG.

  As described above, the DSP 130 of the imaging apparatus 100 functions as an output signal generation unit that generates a signal to be output externally. The DSP 130 serving as an output signal generation unit includes a plurality of line memories 132 and 134 that individually store digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing units 121 and 123, and a plurality of line memories 132, A multiplexer 135 that selectively reads out the data corresponding to the divided area stored in 134 and outputs the data in the order of the constituent pixel arrangement of one frame image, and an image correction processing unit 136 that inputs the output data of the multiplexer 135 and performs image correction. The output control unit 137 inputs the output of the image correction processing unit 136 and outputs the pixel value data of a plurality of pixels selected according to the arrangement of the constituent pixels of one frame image to the transmission cable 180 in parallel.

  First, a data input sequence for the line memories 132 and 134 of the DSP 130 will be described with reference to FIG. The output of the first output amplifier 114 which is image signal information corresponding to the left image of the CCD 110 shown in FIG. 5 is accumulated in the line memory 132 in the DSP 130 as a 12-bit digital signal, for example, via the signal processing unit 121.

  On the other hand, the output of the second output amplifier 115, which is image signal information corresponding to the right image of the CCD 110, is stored in the line memory 134 in the DSP 130 as a 12-bit digital signal, for example, via the signal processing unit 123.

  As shown in FIG. 7, the line memory 132 for inputting pixel value data of the left image sequentially inputs pixel data (for example, 12-bit pixel values) of the constituent pixels of the left image from left to right from the upper line. In accordance with the 60 MHz clock signal, 12-bit data for each pixel is sequentially stored in the line memory 132.

  On the other hand, the line memory 134 for inputting the pixel value data of the right image sequentially inputs the pixel data (for example, 12-bit pixel value) of the constituent pixels of the right image from the upper line from the right to the left. In accordance with the 60 MHz clock signal, 12-bit data for each pixel is sequentially stored in the line memory 134. In this way, the image data of horizontal N pixels on the left and right are temporarily stored in the line memories 132 and 134, respectively.

Next, a data input sequence from the line memories 132 and 134 to the multiplexer (MUX) 135 will be described with reference to FIG. FIG. 8 (2) shows transmission signals of the signal transmission lines (A), (B), and (C) having the configuration shown in FIG. 8 (1). That is,
(A) an input signal to the line memory 132 for storing the left image data;
(B) an input signal to the line memory 134 for storing the right image data;
(C) input signals from the line memories 132 and 134 to the multiplexer (MUX),
These are the input signals.

  As described above with reference to FIG. 7, the line memory 132 that inputs the pixel value data of the left image receives the pixel data (for example, 12-bit pixel value) of the constituent pixels of the left image from the upper line from the left to the right. Enter sequentially. In accordance with the 60 MHz clock signal, 12-bit data for each pixel is sequentially stored in the line memory 132. Ls to Le of the signal (A) in FIG. 8B is the input start timing (Ls) of the pixel value data of one line of the left image (for example, the pixels 1 to N of the left image in FIG. 7) to the line memory 132. And the input end timing (Le).

  On the other hand, the line memory 134 for inputting the pixel value data of the right image sequentially inputs the pixel data (for example, 12-bit pixel value) of the constituent pixels of the right image from the upper line from the right to the left. In accordance with the 60 MHz clock signal, 12-bit data for each pixel is sequentially stored in the line memory 134. Rs to Re of the signal (B) in FIG. 8B are input start timings (Rs) of pixel value data of one line of the right image (for example, pixels 1 to N of the right image in FIG. 7) to the line memory 134. And the input end timing (Re).

  The signals in FIGS. 8B and 8C indicate input signals from the line memories 132 and 134 to the multiplexer (MUX) 135.

  The left image stored in the line memory 132 is output to the multiplexer (MUX) 135 according to the internal synchronization signal (80 MHz) by a first-in first-out method (FIFO). Input order of Ls to Le, that is, pixel value data of the pixels 1 to N of the left image shown in FIG. 7 is sequentially input to the multiplexer (MUX) 135 by the first-in first-out method (FIFO).

  On the other hand, the transfer start position and transfer end position of the right image stored in the line memory 134 are switched by a first-in last-out method (LIFO), and the transfer start position of the right image area comes next to the transfer end position of the left image area. Is output to the multiplexer (MUX) 135. The input order (Rs to Re) is switched by the first-in last-out method (LIFO), and the order of Re to Rs, that is, the pixel value data of the pixels 1 to N in the right image shown in FIG. Are sequentially input to the multiplexer (MUX) 135.

  In the right image stored in the line memory 134, the transfer start position of the right image area (Re position shown in FIGS. 8 (2) and (C)) is the transfer end position of the left image area (FIG. 8 (2) (C ) To the multiplexer (MUX) 135 so as to come next.

  Pixels in a line in the left image start with Ls and end with Le, and pixels in the same line in the right image start with Rs and end with Re. Data storage in the line memories 132 and 134 is executed according to a 60 MHz clock. Data reading from the line memories 132 and 134 is executed according to an 80 MHz clock.

  (A) to (C) in FIG. 8 (2) show the data transfer timing as time (t) elapses from left to right. As can be understood from the timing charts shown in (A) to (C) of FIG. 8 (2), before the writing of one line to the line memory 132 of the left image is completed (before Le in (A)), Reading from the memory 132 is started (Ls in (C)), and immediately after the end of reading from the line memory 132 (Le in (C)), the line image 134, which is the storage memory for the right image, continues to the left image. Read out pixel data (Re). The read clock at this time is 80 MHz.

  At the end of reading from the line memory 132 (Le in (C)), writing of all one line (Rs to Re) to the line memory 134 of the right image is completed, and at this timing, Re, That is, pixel data (Re) continuous to the left image can be read from the line memory 134 and input to the multiplexer (MUX) 135. Thereafter, for the right image, the transfer start position and the transfer end position are switched by a first-in last-out method (LIFO), and the multiplexer (MUX) 135 is arranged so that the transfer start position in the right image area is next to the transfer end position in the left image area. By setting to input to all the images, it is possible to continuously configure all images as one line.

  The sequence of the input image to the multiplexer (MUX) 135 is set as shown in FIG. FIG. 9 shows an input sequence to the multiplexer (MUX) 135 of the image frame of horizontal 2N pixels and vertical S pixels.

  First, pixel value data of the pixels 1 to N corresponding to the left image of the line 1 is read from the line memory 132 and input to the multiplexer (MUX) 135, and immediately after that, the pixel N + 1 corresponding to the right image of the line 1 Pixel value data of ˜2N are read from the line memory 134 and input to the multiplexer (MUX) 135. Next, pixel value data of the pixels 1 to N corresponding to the left image of line 2 is read from the line memory 132 and input to the multiplexer (MUX) 135, and immediately thereafter, the pixels corresponding to the right image of line 2 N + 1 to 2N pixel value data are read from the line memory 134 and input to the multiplexer (MUX) 135. Thereafter, the same processing is repeated.

  As described above, the right image and the left image individually stored in the line memories 132 and 134 are multiplexed as one frame image in which the left and right images are continuous at the input timing from the line memories 132 and 134 to the multiplexer (MUX) 135. (MUX) 135 is input. The output from the multiplexer (MUX) 135 is output according to the internal synchronization signal (80 MHz) in the same sequence as the input sequence to the multiplexer (MUX) 135. These data input / output controls are controlled by a control unit (MPU) 140.

  The image data in which all the image areas are continuous by the multiplexer (MUX) 135 is output to the image correction processing unit 138, where each pixel value is corrected, and the transmission cable is transmitted through the output control unit 137 without being compressed. 180, and is input to the display control unit 201 including the PC of the display module 200 via the transmission cable 180.

  An electrical signal as transmission data via the transmission cable 180 is converted into a low voltage small amplitude differential signal (LVDS) by the output control unit 137 and output. As described above, the transmission data output from the output control unit 137 includes a 24-bit video signal, a signal (LVAL) indicating effective video in the horizontal period, a signal (FVAL) indicating effective video in the vertical period, and an effective video period. A 4-bit data (LVDS signal) and a 1-bit clock (LVDS signal) obtained by serializing the 28-bit signal including the 4-bit control signal of the signal (DVAL) indicating pixel and the 4-bit control signal of the pixel clock signal (STROBE) seven times ) These signals are input to the display control unit 201.

  As described with reference to FIG. 8, the data input from the line memories 132 and 134 to the multiplexer (MUX) 135 becomes one frame image in which the left and right images are continuous, and after the output from the multiplexer (MUX) 135 Data transfer is performed as one frame image, and the output from the output control unit 137 is also output as one frame image without distinction between left and right images.

  An output example of image data from the output control unit 137 to the transmission cable 180 in the imaging apparatus of the present invention will be described with reference to FIG. Data output of the image data is executed as a process according to a predetermined standard, for example, the Base Configuration standard of the camera link.

According to the Base Configuration standard of the camera link, a 24-bit image signal can be transmitted using one connector. That is,
(1) 8-bit port A from port A0 to A7,
(2) 8-bit port B from port B0 to B7,
(3) 8-bit port C from port C0 to C7,
A total of 24 bits of parallel data transfer is possible.

  This is because when one pixel is an image of 8-bit data, a maximum of 3 systems, a 10-bit image, a maximum of 2 systems, a 12-bit image, a maximum of 2 systems, a 14-bit image, a 1-system, a 16-bit image In this case, in the case of a one-line, 24-bit image, for example, an RGB 8-bit image, it means that one line of parallel transmission is possible.

  For example, as described above with reference to FIG. 4, for example, when transmitting two 12-bit images divided by two left and right areas, from port A0 to port A7 and from boat B0 to port B3 Channel A using a total of 12 bits and channel B using a total of 12 bits from port B4 to port B7 and from port C0 to port C7, and when the left image is used for channel A, the right image Is used for channel B and the left image is used for channel B, the right image may be assigned to channel A.

  However, when such a data transmission process is performed, it is necessary to perform a process of combining a plurality of areas into one in a display control unit such as a PC on the display module side, which causes a problem that the processing load increases.

  In the imaging apparatus of the present invention, image data transmission is not performed in units of such divided areas, and as described with reference to FIG. 9, at the stage of input to the multiplexer (MUX) 135, one frame image is converted. All of the processes after the multiplexer (MUX) 135 after conversion are configured to transfer data as one frame image. Accordingly, image data is output from the output control unit 137 to the transmission cable 180 without separately configuring the left and right images, and pixel value data is transmitted from left to right for each line according to the arrows shown in FIG. Output via 180.

  For example, when each pixel is 12-bit data, a total of 24 bits of pixel value data can be transmitted via the transmission cable 180 with two pixels as a set from left to right for each line. A specific example is shown in FIG.

FIG. 10B shows a transmission line for image data in the transmission cable 180. As described above, according to the Base Configuration standard of the camera link, it is possible to transmit a 24-bit image signal using one connector. That is,
(1) 8-bit port A from port A0 to A7,
(2) 8-bit port B from port B0 to B7,
(3) 8-bit port C from port C0 to C7,
A total of 24 bits of parallel data transfer is possible.

When each pixel is 12-bit data, the imaging apparatus 100 of the present invention transmits a total of 24-bit pixel value data via the transmission cable 180, with two pixels as a set from the left to the right for each line. That is,
12-bit data of odd-numbered pixels (1, 3, 5, 7...) Are transmitted using channel A (ports A0 to A7 and ports B0 to B3),
Using the channel B (ports B4 to A7 and ports C0 to C7), 12-bit data of even-numbered pixels (2, 4, 6, 8,...) Is transmitted.

When n = 1, 2, 3... natural numbers, as shown in FIG.
The channel A (ports A0 to A7 and ports B0 to B3) is used to sequentially transmit the odd-numbered pixels, that is, the 12-bit data of the pixel 2n-1,
The channel B (ports B4 to A7 and ports C0 to C7) is used to sequentially transmit even-numbered pixels, that is, 12-bit data of the pixel 2n.

As shown in FIG.
Channel A (ports A0 to A7 and ports B0 to B3) is used for transmission of pixel value data of pixels 1, 3, 5, 7,.
Channel B (ports B4 to A7 and ports C0 to C7) is used for transmission of pixel value data of pixels 2, 4, 6, 8,.

  First, pixels 1 and 2 are transmitted in parallel using channels A and B, respectively, and then pixel value data (12 bits each) of two pixels in the order of pixels 3 and 4 and then pixels 5 and 6. Are transmitted in parallel.

  The entire channel distribution of one frame of image data is set as shown in FIG. In each line, odd-numbered pixel value data is transmitted using channel A (ports A0 to A7 and ports B0 to B3), and even-numbered pixel value data is transmitted to channel B (ports B4 to A7 and port C0). To C7).

  By performing such data transmission in units of two pixels, unlike transmission of serial data in units of one pixel, there is no need to increase the transmission frequency. For example, parallel transmission in units of two pixels at 40 MHz achieves the same transmission efficiency as data transmission in units of one pixel at 80 MHz. As a result, the possibility of transmission loss due to the skin effect of the signal in data transmission from the camera module to the display module via the transmission cable is reduced. Specifically, there is a merit that the transmission distance can be extended. Further, the display control unit 201 on the display module 200 side does not need to increase the data reception or the processing speed of the received data, and can construct an inexpensive system.

  In the above-described embodiment, a processing example in which pixel value data of two pixels is transmitted as a set via a transmission cable has been described. However, the transmission mode is set via the input unit of the imaging apparatus 100 illustrated in FIG. As a possible configuration, a data transmission in units of one pixel in accordance with a high transmission frequency (for example, 80 Hz) and a data transmission in units of two pixels in accordance with a low transmission frequency (for example, 40 Hz) may be selected. In this way, if the 1-channel transmission method and the 2-channel transmission method are selectable, restrictions on the computer on the display module 200 side (an ancestor display control unit 201) are widened, existing settings, control computers, software Assets can be used effectively.

  Further, in the configuration of the present invention, at the stage of input to the multiplexer (MUX) 135, the processing after the multiplexer (MUX) 135 is converted into one frame image, and all the data is transferred as one frame image. Therefore, there is a merit that the image correction in the image correction processing unit 136 that executes pixel value correction or the like can be executed based on one continuous image.

  As image correction in the image correction processing unit 136, for example, there is a case where an interpolation process is performed in which a pixel value of a defective pixel for which a pixel value cannot be obtained due to noise or the like is calculated and set from surrounding pixels. In this case, for example, it is necessary to apply a 3 × 3 image filter to obtain pixel values of pixels around the defective pixel. When the image area is divided, if interpolation of pixels near the boundary is performed, pixel values are read out from the respective image areas, and the reading process becomes complicated and requires time. Then, as an image input to the image correction processing unit 136, one whole image is input instead of a divided image, and an efficient pixel value can be acquired even in the interpolation process.

  In the configuration of the present invention, data transmission via the transmission cable 180 can be performed in units of two pixels. As described above, it is not necessary to increase the transmission frequency, and the band can be effectively used without waste. In other words, the transmission clock is one-half of the conventional one (80 MHz → 40 MHz), and the probability of occurrence of transmission loss is reduced. This makes it possible to realize a longer transmission line than the conventional transmission method.

  Note that the imaging apparatus of the present invention may be either a monochrome camera or a color camera. In the case of a color image, for example, the imaging apparatus outputs RAW data via the transmission cable 180, and the display control unit 201 synthesizes the RAW data, converts it to RGB 24 bits, and displays it on the monitor 201.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run. For example, the program can be recorded in advance on a recording medium. In addition to being installed on a computer from a recording medium, the program can be received via a network such as a LAN (Local Area Network) or the Internet, and installed on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of the embodiment of the present invention, in the configuration in which output signals corresponding to the divided regions of the image sensor, for example, the left and right divided regions are input and output externally, The corresponding digital image data is individually stored in a plurality of line memories, and the division region corresponding data stored in the line memory by the multiplexer is selectively read out and output in the arrangement order of the frame image constituent pixels. The multiplexer inputs the data stored in the left image storage line memory in the order of the left pixel to the right pixel by the first-in first-out method (FIFO), and then stores the data stored in the right image storage line memory in the first-in last-out method (LIFO). ) To input and output pixel value data in order from the right pixel to the left pixel. The output control unit inputs the output data and outputs the pixel value data of a plurality of pixels selected according to the arrangement of the frame image constituent pixels in parallel to the transmission cable. With this configuration, data transmission via the transmission cable can be performed, for example, in units of two pixels, and transmission loss due to the skin effect or the like can be suppressed without increasing the transmission frequency, for example, long transmission. Stable data transmission is also realized on the line. Also, processing at a high frequency is not required on the data receiving side, and the cost of the receiving side device can be reduced.

It is a figure explaining a 1 channel output type image sensor and signal processing composition. It is a figure explaining a 2 channel output type image sensor and signal processing composition. It is a figure explaining the example of a data output from the imaging device using a 2 channel output type imaging device. It is a figure explaining the example of a data output from the imaging device using a 2 channel output type imaging device. It is a figure explaining the structural example of the imaging device which concerns on one Example of this invention. It is a figure explaining the detailed structure of CCD. It is a figure explaining the transfer processing sequence of the image data between each structure part inside the imaging device which concerns on one Example of this invention. It is a figure explaining the transfer processing sequence of the image data between each structure part inside the imaging device which concerns on one Example of this invention. It is a figure explaining the transfer processing sequence of the image data between each structure part inside the imaging device which concerns on one Example of this invention. It is a figure explaining the specific process example of the data transmission process via the transmission cable in the imaging device which concerns on one Example of this invention. It is a figure explaining the specific process example of the data transmission process via the transmission cable in the imaging device which concerns on one Example of this invention. It is a figure explaining the specific process example of the data transmission process via the transmission cable in the imaging device which concerns on one Example of this invention.

Explanation of symbols

10 CCD
DESCRIPTION OF SYMBOLS 11 Vertical register 12 Horizontal register 13 Output amplifier 21 Signal processing part 22 Line memory 23 Output part 30 Output image 40 Camera module 50 CCD
51 Vertical register 52, 53 Horizontal register 54, 55 Output amplifier 61, 62 Signal processing unit 63, 64 Line memory 65 Multiplexer 66 Image correction processing unit 67 Output unit 70 Transmission cable 80 Display module 81 Display control unit 82 Monitor 100 Imaging device ( The camera module)
110 CCD
111 Vertical register 112, 113 Horizontal register 114, 115 Output amplifier 118 Photo detector (PD)
121, 123 Signal processor 130 DSP
132, 134 Line memory 135 Multiplexer 136 Image correction processing unit 137 Output control unit 140 Control unit (MPU)
180 Transmission cable 200 Display module 201 Display control unit 202 Monitor

Claims (9)

An imaging unit that outputs a plurality of output signals corresponding to each of the divided regions of the imaging element;
A signal processing unit that performs signal processing on an output signal from the imaging unit and generates digital image data corresponding to each of the divided regions;
An output signal generator for generating an output signal to be externally input by inputting the generation data of the signal processing unit;
The output signal generator is
A plurality of line memories for individually storing digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing unit;
A multiplexer that selectively reads out the divided region correspondence data stored in the plurality of line memories and outputs the data in the order of arrangement of the frame image constituent pixels;
An image pickup apparatus, comprising: an output control unit which inputs output data of the multiplexer and outputs in parallel to a transmission cable pixel value data of a plurality of pixels selected according to an arrangement of frame image constituent pixels. The imaging unit
It has an image sensor divided into a left image and a right image, and outputs a plurality of output signals corresponding to the left image and the right image,
The plurality of line memories are
It is composed of two line memories that individually store digital image data corresponding to each of the left image and the right image,
The multiplexer is
The imaging apparatus according to claim 1, wherein data corresponding to each of the left image and the right image stored in the two line memories is selectively read and output in the order of the constituent pixels of the frame image. In the two line memories,
The left image storage line memory for storing the left image data stores the pixel value data of each line of the left image in the order of the left pixel to the right pixel,
The right image storage line memory for storing the right image data is configured to store the pixel value data of each line of the right image in order from the right pixel to the left pixel,
The multiplexer is
After the pixel value data is input from the left pixel to the right pixel in the first-in first-out method (FIFO), the storage data in the left image storage line memory is input in the first-in last-out method (LIFO). The imaging apparatus according to claim 2, wherein the imaging apparatus is configured to execute a process of inputting and outputting pixel value data in order from the right pixel to the left pixel. The multiplexer is
The imaging apparatus according to claim 1, wherein data input processing from the plurality of line memories is executed in a clock cycle faster than a clock cycle of data writing processing to the plurality of line memories. The output control unit
2. A configuration in which pixel value data of two consecutive pixels in accordance with an arrangement of frame image constituent pixels is sequentially selected as a set of parallel output data and output to a transmission cable. The imaging device described. The imaging device further includes:
2. The configuration according to claim 1, wherein the data output in a different transmission mode can be switched between a data output in units of one pixel according to a high transmission frequency and a data output in units of two pixels according to a low transmission frequency. The imaging device described. The output signal generation unit further includes:
An image correction processing unit for inputting the output data of the multiplexer and executing image correction;
The output control unit is configured to input the output data of the image correction processing unit and execute a process of outputting pixel value data of a plurality of pixels selected in accordance with an arrangement of frame image constituent pixels to a transmission cable in parallel. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. An output signal control method for executing output processing of image data acquired in an imaging device,
A signal processing step in which a signal processing unit performs signal processing on an output signal from the imaging unit that outputs a plurality of output signals corresponding to each of the divided regions of the image sensor, and generates digital image data corresponding to each of the divided regions; ,
The output signal generation unit has an output signal generation step of generating an output signal to be output to the outside by inputting the generation data of the signal processing unit,
The output signal generation step includes
Individually storing digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing unit in a line memory;
A data synthesis step in which the multiplexer selectively reads out the divided region corresponding data stored in the plurality of line memories and outputs the data in the order of arrangement of the frame image constituent pixels;
An output signal control method comprising: a data output step in which an output control unit inputs output data of the multiplexer and outputs pixel value data of a plurality of pixels selected in accordance with an arrangement of frame image constituting pixels to a transmission cable in parallel. . A computer program that executes output control of image data acquired in an imaging device,
A signal processing step of causing the signal processing unit to perform signal processing on the output signals from the imaging unit that outputs a plurality of output signals corresponding to each of the divided regions of the imaging element, and generate digital image data corresponding to each of the divided regions. When,
The output signal generation unit has an output signal generation step of generating an output signal to be output to the outside by inputting the generation data of the signal processing unit,
The output signal generation step includes
Individually storing digital image data corresponding to each of the divided regions of the image sensor generated by the signal processing unit in a line memory;
A data synthesis step in which the multiplexer selectively reads out the divided region corresponding data stored in the plurality of line memories and outputs the data in the order of arrangement of the frame image constituent pixels;
A computer program comprising: a data output step in which an output control unit inputs output data of the multiplexer and outputs pixel value data of a plurality of pixels selected in accordance with an arrangement of frame image constituting pixels to a transmission cable in parallel.