CN114556903B - Image generation system, image processing device, program, and recording medium - Google Patents

Abstract

The camera unit (1) is capable of rotating around a rotation axis, and has a plurality of detection element lines extending parallel to the rotation axis and each consisting of a plurality of detection elements. By shooting, a camera image containing pixels corresponding to each detection element of the detection element line is generated. The image processing device (3) calculates the corresponding pixel position in the panoramic coordinate system as a projection position for each pixel, and projects each pixel to the projection position. At the pixel position where more than two pixels are projected, the pixel values of the projected pixels are synthesized and the synthesized value is calculated, and a panoramic image containing a plurality of pixels whose pixel values are each synthesized value is generated. By connecting the images obtained by rotating the camera unit and shooting, at least one of a high resolution and a high SN ratio of the panoramic image can be achieved.

Description

Image generation system, image processing device, program, and recording medium

Technical Field

The present invention relates to an image generation system and an image processing device. The present invention also relates to a program and a recording medium. In particular, the present invention relates to the generation of a panoramic image.

Background technique

A panoramic image is generated by connecting a plurality of captured images. In this case, the improvement of spatial resolution and SN ratio becomes a problem.

In the field of document reading, it is known to obtain a high-resolution image by scanning with n line sensors. For example, Patent Document 1 describes that n pixel columns read by n line sensors are arranged in a coordinate system of an output image to generate a high-resolution image.

Prior art literature

Patent Literature

Patent Document 1: Japanese Patent Application Laid-Open No. 11-127319 (page 4)

Summary of the invention

Problem that the invention aims to solve

The method of Patent Document 1 requires translation of n line sensors, which increases the size of the reading device and cannot be applied to the generation of a panoramic image.

An object of the present invention is to generate a panoramic image having at least one of a high resolution and a high SN ratio.

Means used to solve problems

The image generation system of the present invention has:

A camera unit that can rotate around a rotation axis and has a plurality of detection element lines extending parallel to the rotation axis and each consisting of a plurality of detection elements, the camera unit generates a camera image containing pixels corresponding to the detection elements of the detection element lines by shooting; and an image processing device that calculates corresponding pixel positions in a panoramic coordinate system as projection positions for pixels corresponding to the detection elements, projects the pixels to the projection positions, and synthesizes pixel values of the projected two or more pixels at pixel positions in the panoramic coordinate system where two or more pixels are projected to calculate a synthesized value, thereby generating a panoramic image containing a plurality of pixels each having the synthesized value as a pixel value.

The image processing device of the present invention generates a panoramic image based on a camera image generated by shooting by a camera unit. The camera unit can rotate around a rotation axis and has a plurality of detection element lines extending parallel to the rotation axis and respectively composed of a plurality of detection elements. The camera image includes pixels corresponding to the detection elements of the detection element lines, respectively. The image processing device includes: a projection position calculation unit, which calculates the corresponding pixel position in a panoramic coordinate system as a projection position for the pixels respectively corresponding to the detection elements; and a pixel projection unit, which projects the pixel to the projection position, and at the pixel position of the panoramic coordinate system where more than two pixels are projected, the pixel values of the projected more than two pixels are synthesized to calculate a synthesized value. The image processing device generates a panoramic image including a plurality of pixels each having the synthesized value as a pixel value.

Effects of the Invention

According to the present invention, by connecting images captured while rotating the imaging unit, it is possible to achieve at least one of a higher resolution and a higher SN ratio of the panoramic image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing an image generation system according to Embodiment 1 of the present invention.

FIG. 2 is a diagram showing an example of the arrangement of image sensors in the imaging unit of FIG. 1 .

FIG. 3 is a diagram showing an example of an arrangement of detection elements in an image sensor and a portion used as a detection element line.

FIG. 4 is a perspective view showing only the portion used as the detection element line shown in FIG. 3 .

FIG. 5 is a horizontal cross-sectional view showing only the portion used as the detection element line shown in FIG. 3 .

FIG. 6 is a diagram showing an example of projection into a panoramic coordinate system.

FIG. 7 is a diagram showing another example of a portion used as a detection element line in an image sensor.

FIG. 8 is a diagram showing another example of the arrangement of image sensors in the imaging unit of FIG. 1 .

FIG. 9 is a schematic diagram showing an image generation system according to Embodiment 2 of the present invention.

FIG. 10 is a cross-sectional view showing an example of an image sensor used in Embodiment 2. FIG.

FIG. 11 is a schematic diagram showing an image generation system according to Embodiment 3 of the present invention.

FIG. 12 is a schematic diagram showing an image generation system according to a fourth embodiment of the present invention.

FIG. 13 shows a configuration example of a computer 9 that realizes all functions of the image processing device and the operation control unit in Embodiments 1 to 4 of the present invention.

14 is a flowchart showing a processing procedure in a processor when the functions of the image processing device and the motion control unit according to the fourth embodiment are realized by the computer of FIG. 13 .

FIG. 15 is a flowchart showing a process procedure of generating a real-time image in FIG. 14 .

FIG. 16 is a flowchart showing a processing procedure for generating the reference image of FIG. 14 .

Detailed ways

Implementation method 1.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Fig. 1 shows an image generation system representing Embodiment 1 of the present invention. The image generation system shown in the figure generates a panoramic image.

The image generation system shown in FIG. 1 includes an imaging unit 1 , a rotation driving system 2 , an image processing device 3 , and an operation control unit 4 .

The rotation drive system 2 rotates the imaging unit 1 around a rotation axis.

The imaging unit 1 includes an image sensor 6 and an imaging optical system 7 .

The image sensor 6 generates image data representing an image formed by the imaging optical system 7 .

The image sensor 6 includes a plurality of detection element lines extending in parallel with the rotation axis and each of which is constituted by a plurality of detection elements, and generates a captured image including pixels corresponding to the detection elements of the detection element lines by imaging.

The image processing device 3 calculates the corresponding pixel position in the panoramic coordinate system as the projection position for the pixels corresponding to each detection element of the plurality of detection element lines, and projects the pixels corresponding to each detection element of the plurality of detection element lines to the projection position. The image processing device 3 synthesizes the pixel values of the projected two or more pixels at the pixel position of the panoramic coordinate system where two or more pixels are projected, and calculates a synthesized value. The image processing device 3 generates a panoramic image including a plurality of pixels each having a synthesized value as a pixel value.

Hereinafter, each structural element will be described in detail.

Here, it is assumed that the image generation system is used to generate a thermal image. In this case, a thermal image sensor is used as the image sensor 6. The thermal image sensor has a plurality of infrared detection elements that detect infrared rays (usually electromagnetic waves with a wavelength of 8 μm to 12 μm) radiated from a subject.

In the case of an uncooled thermal infrared solid-state imaging device, detection elements using semiconductor elements such as diodes and transistors are known as detection elements in addition to bolometers of polycrystalline silicon, amorphous silicon, silicon carbide, vanadium oxide, and the like.

The image sensor 6 is constituted by, for example, a two-dimensional image sensor.

In a two-dimensional image sensor, for example, detection elements having a heat-insulating structure are arranged in row and column directions, connected by drive lines for each row and connected by signal lines for each column.

Each drive line is selected in turn by the vertical scanning circuit and the switch, and the power supply is energized to the detection element through the selected drive line. The output of the detection element is transmitted to the integration circuit via the signal line, integrated and amplified by the integration circuit, and output to the output terminal in turn through the horizontal scanning circuit and the switch.

The image sensor 6 outputs the image DIN at each imaging cycle (frame cycle) Tf. The imaging cycle Tf may be fixed or may be changed by the operation control unit 4.

FIG3 shows an example of the arrangement of detection elements in the image sensor 6. In the example shown in the figure, the image sensor 6 is composed of a detection element array of Ma rows and Na columns. In FIG3, each rectangular portion divided by grid-like lines represents a detection element. In the example shown in FIG3, each column extends in the vertical direction.

A portion of the detection elements constituting the detection element array shown in FIG3 is selected and designated as the detection elements constituting the plurality of detection element lines L1 to L4. For example, detection elements arranged along a plurality of straight lines extending parallel to the rotation axis Ac are selected and designated as the detection elements constituting the plurality of detection element lines L1 to L4. The number of detection element lines is, for example, 4, but is sometimes generalized and represented by N.

For example, detection element lines L1 to L4 are formed by N consecutive columns (N=4 in this case) of detection elements located near the center in the horizontal direction of the detection element array.

As described later, a plurality of, for example, four, pixel lines S1 to S4 corresponding to the above-mentioned detection element lines L1 to L4 in the video image DIN output from the image sensor 6 are extracted for use in the generation of a panoramic image. The arrangement of pixels in the video image DIN and the configuration of the pixel lines S1 to S4 are the same as the arrangement of the detection elements and the configuration of the detection element lines L1 to L4 shown in FIG. 3 . Therefore, the pixel lines S1 to S4 are respectively composed of pixels located on a plurality of straight lines parallel to each other in the video image DIN.

The rotation drive system 2 is controlled by the motion control unit 4 to rotate the imaging unit 1 around the rotation axis Ac as shown in Fig. 2. The rotation is performed at a constant angular velocity, for example.

The rotation drive system 2 rotates the imaging unit 1 while maintaining the plane (sensor surface) 6a on which the detection elements of the image sensor 6 are arranged parallel to the rotation axis Ac, thereby rotating the imaging direction of the imaging unit 1, that is, the direction of the optical axis Oa of the imaging unit 1, around the rotation axis Ac. For example, the image sensor 6 maintains the direction of the row or column parallel to the rotation axis Ac. The following describes the case where the direction of the column is maintained parallel to the rotation axis Ac.

In the example shown in FIG. 2 , the imaging unit 1 is supported so that the principal point Q of the imaging optical system 7 is located on the rotation axis Ac.

Hereinafter, for convenience, the direction of the rotation axis Ac is referred to as the vertical direction, and the direction perpendicular to the rotation axis is referred to as the horizontal direction. In addition, in the image obtained by photographing by the image sensor 6, the direction corresponding to the vertical direction of the image sensor 6 (the direction of the rotation axis Ac) is referred to as the vertical direction, and the direction perpendicular to the vertical direction is referred to as the horizontal direction.

In addition, for convenience, the angle in the rotation direction centered on the rotation axis Ac is called the azimuth angle, and for convenience, the angle in the vertical direction, that is, the angle around the straight line perpendicular to the rotation axis Ac and the optical axis Oa is called the pitch angle.

In FIG2 , the imaging unit 1 is rotated within a range from an azimuth (starting point) Xa to an azimuth (ending point) Xb. Such rotation is referred to as scanning. In addition, the angle θ of the imaging direction of the imaging unit 1 at each time point, i.e., the direction of the optical axis Oa, relative to the reference azimuth, such as the starting point Xa, is referred to as the rotation angle.

The action control unit 4 outputs information indicating the positions of the detection element lines L1 to L4 (detection element line position information) Gp, information indicating the positions of the pixel lines S1 to S4 (pixel line position information) Gs, information indicating the field of view angle _βH of each detection element line, and information indicating the rotation angle θ of the imaging unit 1 at each time point.

The operation control unit 4 controls the optical magnification of the imaging optical system 7, and calculates and outputs the field angle _βH of each detection element line based on the optical magnification. The field angle _βH of each detection element line is obtained from the horizontal field angle _αH of each detection element.

The horizontal field angle _αH of each detection element is obtained by dividing the horizontal field angle of the imaging unit 1 by the number of detection elements in the horizontal direction of the image sensor 6, that is, the number of columns. For example, if the horizontal field angle of the imaging unit 1 is 80 degrees and the number of detection elements in the horizontal direction of the image sensor 6 is 160, the horizontal field angle _αH of each detection element becomes

α _H =80÷160=0.5 degrees Formula (1).

The system of this embodiment has a function of generating a panoramic image by connecting pixel lines extracted from the camera image DIN in the horizontal direction, and a function of adjusting the resolution in the horizontal direction. On the other hand, there is no function of connecting images and a function of adjusting the resolution in the vertical direction. With respect to the vertical direction, the pixels representing the subject portion in the camera image DIN at a certain pitch angle are directly projected onto the pixels at the same pitch angle in the panoramic image.

Therefore, the vertical direction will be briefly described below, and the horizontal direction will be described in detail.

The rotation angle θ of the imaging unit 1 changes as scanning proceeds, that is, as the imaging unit 1 rotates. The motion control unit 4 receives notification of the rotation angle θ from the rotation drive system 2, for example, and outputs the rotation angle θ to the pixel line extraction unit 31. When the motion control unit 4 provides the rotation drive system 2 with a command value of the rotation angle θ, the motion control unit 4 may internally generate and output the command value of the rotation angle θ to the pixel line extraction unit 31.

The image processing device 3 generates a panoramic image DOUT from the captured image DIN based on the detection element line position information Gp and pixel line position information Gs output from the operation control unit 4 , the field angle _βH of each detection element line, and the rotation angle θ of the imaging unit 1 .

The image processing device 3 includes a pixel line extraction unit 31 , a projection position calculation unit 32 , and a pixel projection unit 33 .

The pixel line extraction unit 31 extracts a plurality of pixel lines S1 to S4 from the captured image DIN output from the image sensor 6 in each imaging cycle based on the pixel line position information Gs supplied from the motion control unit 4. The number of extracted pixel lines is, for example, 4, but is sometimes generalized and represented by N. The pixel lines S1 to S4 are respectively composed of pixels located on straight lines extending in the vertical direction at different horizontal positions in the captured image DIN, for example, pixels belonging to the same column.

The pixel line position information Gs indicates the positions of the pixel lines S1 to S4 to be extracted in the captured image DIN. The detection element line position information Gp indicates the positions of the detection element lines L1 to L4 in the image sensor 6 , that is, the detection element lines corresponding to the pixel lines S1 to S4 .

Which column in the detection element array of the image sensor 6 is used as the detection element lines L1 to L4 is determined in advance, for example. In this case, the detection element line position information Gp is stored in a parameter storage unit (not shown) in the action control unit 4, and the action control unit 4 notifies the projection position calculation unit 32 of the above-mentioned detection element line position information Gp, and generates pixel line position information Gs based on the detection element line position information Gp, and notifies it to the pixel line extraction unit 31. The pixel line extraction unit 31 extracts pixel lines S1 to S4 according to the pixel line position information Gs.

The detection element line position information Gp may be information indicating a relative position with respect to the optical axis Oa, or may be information indicating the number of the detection element line counted from a vertical plane including the optical axis Oa.

The pixel line position information Gs may be information indicating the relative position with respect to a center line extending in the vertical direction at the horizontal center position of the captured image DIN, or may be information indicating the number of the detection element line counted from the center line of the captured image DIN.

The projection position calculation unit 32 calculates the corresponding pixel position in the panoramic coordinate system as the projection position for each pixel of the plurality of pixel lines S1 to S4 extracted from the captured image DIN based on the detection element line position information Gp, the _field angle β H of each detection element line, and the rotation angle θ of the imaging unit 1 .

The panoramic coordinate system uses the azimuth angle X and the elevation angle Y as coordinates. The azimuth angle X is defined as 0 at the starting point Xa, for example, with the starting point Xa as the origin. Regarding the panoramic coordinate system, the azimuth direction and the elevation direction are sometimes referred to as the horizontal direction and the vertical direction, respectively.

The panoramic image DOUT has a spatial resolution corresponding to the number of pixels, and therefore the coordinates (pixel position coordinates) X and Y representing the pixel positions in the panoramic coordinate system have discrete values separated from each other by an integer multiple of the pixel pitch. In the following, only the pixel position coordinate (azimuth coordinate) X in the azimuth direction will be described.

The projection position calculation unit 32 calculates relative angles (relative azimuths) ε1 to ε4 in the horizontal direction with respect to the optical axis Oa for each detection element line L1 to L4 based on the detection element line position information Gp and the field angle β _H of each detection element line, and calculates the azimuth angle X of the projection position based on the calculated relative angles ε1 to ε4 and the rotation angle θ of the imaging unit 1.

The above-mentioned relative angle of each detection element line relative to the horizontal direction of the optical axis Oa refers to the relative angle of the line of sight of the detection element line relative to the horizontal direction of the vertical plane containing the optical axis Oa, that is, the plane containing the optical axis Oa and the rotation axis Ac.

Hereinafter, relative angles ε1 to ε4 will be described with reference to FIG. 4 and FIG. 5 .

Fig. 4 and Fig. 5 show the above-mentioned four detection element lines L1 to L4 extracted from the image sensor 6 of Fig. 3. Fig. 4 is a perspective view, and Fig. 5 is a horizontal cross-sectional view.

In the examples shown in Fig. 4 and Fig. 5 , the optical axis Oa of the imaging unit 1 passes through the boundary between the second detection element line L2 and the third detection element line L3. Fig. 4 and Fig. 5 also show the principal point Q of the imaging optical system 7.

FIG5 also shows the directions E1 to E4 of the sight lines of the detection element lines L1 to L4. The direction of the sight line of each detection element line refers to the horizontal component of the direction of the sight line of the detection element belonging to the detection element line. The directions of the sight lines between adjacent detection element lines differ by the field of view angle β _H of each detection element line. |ε1| to |ε4| represent the angular differences of the directions of the sight lines E1 to E4 relative to the optical axis Oa.

As in the example shown in the figure, when the detection element lines L1 to L4 are formed in a continuous row, the viewing angle _βH of each detection element line is equal to the horizontal viewing angle _αH of each detection element.

In order to simplify the description, the field angle β _H of each detection element line is assumed to be 4 degrees. In this case, the direction E2 of the line of sight of the detection element line L2 is inclined 2 degrees to the left relative to the optical axis Oa (|ε2|=0.5×β _H ), the direction E1 of the line of sight of the detection element line L1 is inclined 6 degrees to the left relative to the optical axis Oa (|ε1|=1.5×β _H ), the direction E3 of the line of sight of the detection element line L3 is inclined 2 degrees to the right relative to the optical axis Oa (|ε3|=0.5×β _H ), and the direction E4 of the line of sight of the detection element line L4 is inclined 6 degrees to the right relative to the optical axis Oa (|ε4|=1.5×β _H ). In addition, in FIG. 5 , the angles |ε1| to |ε4| are shown as being larger than the actual angles.

The angles |ε1| to |ε4| are labeled with positive and negative signs corresponding to the tilt direction, and are called the relative angles of the detection element lines L1 to L4 with respect to the optical axis Oa, and are represented by the symbols ε1 to ε4. For example, in the case of tilting to the left, "-" is marked. Thus,

ε1＝-|ε1|＝-6 degrees Formula (2a)

ε2＝-|ε2|＝-2 degrees Formula (2b)

ε3＝|ε3|＝2 degrees Formula (2c)

ε4＝|ε4|=6 degrees Equation (2d).

Generally speaking, εI is used to represent the relative angle of the I-th (I is 1, 2, 3 or 4) detection element line LI.

Based on the relative angle εI and the rotation angle θ obtained as described above, the azimuth angle X of the projection position is calculated. The following equation (3) is used for the calculation.

X(θ, I) = θ + εI Formula (3)

In the formula (3), X(θ, I) is the azimuth angle representing the direction of the line of sight of the I-th detection element line LI when the rotation angle θ is reached.

By making the azimuth angle representing the direction of the line of sight of each detection element line the same as the azimuth angle of the pixel position in the panoramic coordinate system of the pixel projected onto the pixel line corresponding to the detection element line, it is possible to construct pixels at the same azimuth angle in the panoramic coordinate system based on the pixel values obtained using the detection element that uses a certain azimuth angle as the direction of the line of sight, thereby appropriately generating a panoramic image.

Therefore, the azimuth of the projection position of the pixel of the pixel line SI corresponding to the I-th detection element line LI at the rotation angle θ can be expressed by using X(θ, I) calculated by formula (3). Thus, the projection position calculation unit 32 calculates the azimuth of the projection position using formula (3). However, when the calculation result in formula (3) is inconsistent with the azimuth coordinate representing the pixel position, the pixel position represented by the azimuth coordinate closest to the above calculation result among the azimuth coordinates representing the pixel position is determined as the projection position.

The projection position calculated by the projection position calculation unit 32 is notified to the pixel projection unit 33 .

The pixel projection unit 33 projects each pixel of the extracted pixel lines S1 to S4 to the projection position (X, Y) calculated by the projection position calculation unit 32. Projecting a pixel means associating the pixel value of the pixel with the projection position.

When two or more pixels are projected to the same pixel position, the pixel projection unit 33 combines the pixel values of the projected pixels to calculate a combined pixel value. The combination may be an average calculation. The average may be a simple average or a weighted average.

Fig. 6 shows an example of projection to pixel positions in the panoramic coordinate system. The vertical θ represents the rotation angle of the imaging unit 1 when shooting. The horizontal X represents the azimuth angle of the panoramic coordinate system.

In the example shown in FIG. 6 , for pixels P1 to P4 located at the same vertical position in four pixel lines S1 to S4 extracted from the captured image DIN, changes in the azimuth angle X of the projection position in the panoramic coordinate system accompanying changes in the rotation angle θ are shown.

In the example shown in the figure, it is assumed that the image is captured every time the rotation angle θ changes by Δθ=1 degree. The above Δθ is called the image capturing interval. There is a relationship of the following formula (4) between the image capturing interval Δθ (degrees), the angular velocity Va (degrees/second) of the rotation of the image capturing unit 1, and the image capturing cycle Tf (seconds).

Δθ (degree) = Va (degree/second) × Tf (second) Equation (4)

For any θ, the pixel value of a pixel PI (I is 1, 2, 3, or 4) is represented by PI(θ). In FIG6 , P in PI(θ) is omitted, and only I(θ) is shown.

In FIG. 6 , the field angle β _H of each detection element line is set to 4 degrees, and the horizontal field angle γ _H of each pixel in the panoramic image DOUT is set to 1 degree.

The field angle of each pixel line in the captured image DIN is equal to the field angle β _H of each detection element line. Therefore, when the detection element lines are composed of adjacent columns, the horizontal spatial resolution of the panoramic image DOUT is four times the horizontal spatial resolution of the captured image DIN.

In the example of Figure 6, when the first shot is taken after the start of scanning, θ=1 degree. At this time, the azimuth angle X of the direction of the line of sight of the detection element lines L1, L2, L3, and L4 are -5 degrees, -1 degree, +3 degrees, and +7 degrees, respectively. These azimuth angles are determined to represent the azimuth angles of the projection positions of the pixel lines S1, S2, S3, and S4 corresponding to the detection element lines L1, L2, L3, and L4.

When the next shot is taken at θ=2 degrees, the azimuth angle X of the direction of the line of sight of the detection element lines L1, L2, L3, and L4 becomes -4 degrees, 0 degrees, +4 degrees, and +8 degrees, respectively. These azimuth angles are determined to represent the azimuth angles of the projection positions of the pixel lines S1, S2, S3, and S4 corresponding to the detection element lines L1, L2, L3, and L4.

Similarly, the azimuth angles of the directions of the lines of sight of the detection element lines L1 , L2 , L3 , and L4 change with the rotation of the imaging unit 1 , and the projection position changes accordingly.

When the rotation angle θ of the imaging unit 1 is 5 degrees, the pixel value P1(5) with the azimuth angle X=-1 degree is obtained in the pixel P1. When θ is 1 degree, the pixel value P2(1) with the azimuth angle X=-1 degree is also obtained in the pixel of the pixel line S2.

Therefore, the pixel value P1(5) of the pixel P1 when θ=5 degrees and the pixel value P2(1) of the pixel P2 when θ=1 degree are combined to obtain a combined value of the azimuth angle X=-1 degree.

The same also applies to the azimuth angle X=0 to 2 degrees.

Regarding the azimuth angle X=3 degrees, the pixel value P3(1) of the pixel P3 when θ=1 degree, the pixel value P2(5) of the pixel P2 when θ=5 degrees, and the pixel value P1(9) of the pixel P1 when θ=9 degrees are obtained at different timings, and therefore a composite value is obtained by synthesizing them.

The same also applies to the azimuth angle X=4 to 6 degrees.

Regarding the azimuth angle X=7 degrees, the pixel value P4(1) of the pixel P4 when θ=1 degree, the pixel value P3(5) of the pixel P3 when θ=5 degrees, the pixel value P2(9) of the pixel P2 when θ=9 degrees, and the pixel value P1(13) of the pixel P1 when θ=13 degrees are obtained at different timings, and therefore a composite value is obtained by synthesizing them.

The same applies to the azimuth angle X=8 degrees and above.

As described above, from the start of scanning until the fourth shooting is repeated, the number of projected pixels Np is 1, the number of projected pixels Np is 2 when the shooting is 5 or more to 8 times, the number of projected pixels Np is 3 when the shooting is 9 or more to 12 times, and when the shooting is 13 or more times, the number of projected pixels Np is 4. When the scanning is close to the end, the number of projected pixels Np gradually decreases from 3, 2, and 1.

The synthesized value obtained in the above manner is written into a panoramic image storage unit (not shown) in the pixel projection unit 33. The panoramic image storage unit in the pixel projection unit 33 has a storage area corresponding to all pixel positions in the panoramic coordinate system.

The above processing is similarly performed for all pixels arranged in the vertical direction of the pixel lines S1 to S4 and all pixel positions arranged in the vertical direction of the panoramic coordinate system. That is, the projection from each pixel line in the pixel lines S1 to S4 is performed for all pixels in the pixel line, and the calculation of the composite value at a horizontal position (a position represented by an azimuth coordinate) in the panoramic coordinate system is performed for all pixel positions arranged in the vertical direction at the horizontal position in the panoramic coordinate system.

A column image is formed by one column of pixels arranged in the vertical direction of the panoramic coordinate system.

Projection from each pixel line is performed in each imaging cycle, and synthesis at each pixel position in the panoramic coordinate system is performed after all projections to the pixel position are completed.

When the scanning of the camera unit 1 (rotation from the starting point Xa to the ending point Xb) is completed, and the projection of the pixels of each pixel line obtained by shooting to the panoramic coordinate system during the scanning and the synthesis of the projected pixels are completed, an image composed of a collection of column images written to the panoramic image storage unit in the pixel projection unit 33, that is, an image Db obtained by connecting the column images in the horizontal direction is output as a panoramic image DOUT.

In the process of generating the panoramic image DOUT described above, 4 pixels of the camera image DIN are projected to each pixel position except the end area (near the start point Xa and near the end point Xb). The end area where only less than 4 pixels are projected may be used directly, or may not be projected, or may be cropped after pixel projection.

Next, the spatial resolution and SN ratio of the generated panoramic image DOUT will be examined.

In the above example, the horizontal field angle _αH of each detection element of the image sensor and the field angle _βH of each detection element line are equal to each other, which is 4 degrees, and the horizontal field angle γH of each pixel of the panoramic image _DOUT is 1 degree. That is, in the above example, the horizontal field angle _γH of each pixel of the panoramic image DOUT is smaller than the field angle _βH of each detection element line. In addition, the imaging interval Δθ is 1 degree, which is equal to the horizontal field angle _γH of each pixel of the panoramic image DOUT.

In order to improve the spatial resolution of the panoramic image DOUT, the horizontal field angle _γH of each pixel of the panoramic image DOUT may be further reduced and the imaging interval Δθ may be further reduced.

For example, by setting _βH to 4 degrees, _γH to 0.5 degrees, and Δθ to 0.5 degrees as in the above example, the spatial resolution of the panoramic image DOUT can be doubled as in the example of FIG. 6 .

The imaging interval Δθ is given by the product of the angular velocity Va and the imaging cycle Tf. Therefore, in order to further reduce Δθ, the angular velocity Va may be reduced or the imaging cycle Tf may be further shortened.

In this case, as in the example of Fig. 6, the imaging interval Δθ may be smaller than the field angle β _H of each detection element line. To this end, the angular velocity Va and the imaging cycle Tf may be determined so as to satisfy such conditions. If the imaging cycle Tf is fixed, the angular velocity Va may be determined so as to satisfy the above conditions.

In order to improve the SN ratio of the panoramic image DOUT, the number of pixels (projected pixel number) Np of the camera image DIN projected to the same pixel position of the panoramic coordinate system can be increased. To this end, the number of pixel lines (extracted line number) N extracted from the camera image DIN can be increased. When the number of extracted lines N is increased, all columns of the image sensor 6 can also be used as detection element lines.

In order to increase the number of extraction lines N, the horizontal field angle γ _H of each pixel of the panoramic image DOUT may be set larger than the field angle β _H of each detection element line.

In short, the number N of detection element lines is not limited to 4, and may be 2 or more.

As described above, according to the first embodiment, the imaging unit is rotated to synthesize and connect pixel lines captured at different rotation angles, thereby generating a panoramic image, and achieving one or both of a high resolution and a high SN ratio of the panoramic image.

Various modifications can be made to the above-described embodiment.

For example, with reference to FIG. 3 and the like, the case where the detection element lines L1 to L4 are constituted by continuous columns of the two-dimensional image sensor is described, but the detection element lines L1 to L4 may be constituted by columns separated from each other of the two-dimensional image sensor. For example, as shown in FIG. 7 , every other column may be used as the detection element lines L1 to L4. In addition, columns further away from each other may be used as the detection element lines L1 to L4.

In the above example, which column of the image sensor 6 is used as the detection element line is predetermined, but it may be configured so that when the image generation system starts operating or when the imaging unit 1 starts scanning, the operation control unit 4 specifies the column used as the detection element line. In this case, the operation control unit 4 may generate and output the detection element line position information Gp and the corresponding pixel line position information Gs based on the information indicating the position of the specified column. In addition, when the field angle β _H of each detection element line is changed by specifying the detection element line, the changed β _H may be output.

The viewing angle _βH of each detection element line is calculated based on the relationship between the columns constituting the detection element lines L1 to L4 (for example, the distance with the column pitch as 1 unit). For example, as shown in FIG7 , when every other column is used as the detection element lines L1 to L4, the viewing angle _βH of each detection element is twice the horizontal viewing angle _αH .

In the example shown in FIG3 , the detection element lines L1 to L4 are arranged symmetrically with the plane (center plane) containing the optical axis Oa and the rotation axis Ac of the imaging unit 1 as the center. However, this is not essential. For example, the detection element lines L1 to L4 may all be arranged on one side of the above-mentioned center plane.

It is desirable to select a detection element line in such a manner that the pixels corresponding to the detection elements of the detection element line are located in an area close to the optical axis Oa where lens distortion is small, i.e., in the central area, in the captured image DIN. When the above-mentioned pixels (i.e., the pixels corresponding to the detection elements of the detection element line) are located in an area where the influence of lens distortion is assumed, it is desirable to perform lens distortion correction by preprocessing.

In the configuration described with reference to FIG. 2 and the like, the imaging unit 1 is supported so that the principal point Q of the imaging optical system 7 is located on the rotation axis Ac. However, this is not essential.

For example, as shown in FIG. 8 , the imaging unit 1 may be supported so that the principal point Q of the imaging optical system 7 is located at a position separated from the rotation axis Ac by a distance Rs.

However, the larger the distance Rs is, the larger the difference between the direction, such as the azimuth, of the line of sight of the subject from the rotation axis Ac and the direction, such as the azimuth, of the line of sight of the same subject from the principal point Q of the imaging optical system 7 becomes. In order to prevent the above azimuth difference from increasing, the distance Rs is preferably small.

In the above-described example, the rotation drive system 2 causes the imaging unit 1 to move at a constant angular velocity.

In this case, the camera unit 1 continues to rotate during the exposure for shooting. Since the exposure is performed while rotating, blur may occur in the image. However, if the exposure time during shooting is sufficiently short, the image degradation caused by blur can be suppressed to a small extent. In addition, blur compensation may be applied to the image obtained by shooting.

When the rotation drive system moves at a constant angular velocity, a panoramic image can be generated in a relatively short time.

Alternatively, the rotation drive system 2 may cause the imaging unit 1 to perform a step rotation motion in which rotation and stop are alternately repeated.

The stepping rotational motion can be realized by using a stepping motor. In this case, it is desirable to perform exposure during the stop period and to perform rotation when not performing exposure.

If the rotation drive system 2 performs step-by-step rotational motion and performs exposure during the stop period, a panoramic image with less motion blur can be generated.

Implementation method 2.

FIG. 9 shows an image generation system according to Embodiment 2 of the present invention.

The image generation system shown in FIG. 9 is substantially the same as the image generation system of FIG. 1 , but includes an image processing device 3 b , an operation control unit 4 b , and an image sensor 6 b instead of the image processing device 3 , the operation control unit 4 , and the image sensor 6 of FIG. 1 .

The image sensor 6b of Fig. 9 is substantially the same as the image sensor 6 of Fig. 1, but optical filters are provided on the detection element lines L1 to L4. For example, as shown in Fig. 10, optical filters F1 to F4 are provided to cover the detection element lines L1 to L4. The transmittances of the optical filters F1 to F4 are different from each other.

As a result of providing the optical filters F1 to F4 having different transmittances, images are captured at different exposures in the detection element lines L1 to L4 , and pixel values corresponding to the different exposures are obtained.

The action control unit 4b of Figure 9 is substantially the same as the action control unit 4 of Figure 1 , but for each pixel line corresponding to the detection element line, a weight wI (I is 1, 2, 3 or 4) corresponding to the transmittance of the optical filter FI (I is 1, 2, 3 or 4) provided on the detection element line is stored in an internal parameter storage unit.

The weight wI corresponding to the transmittance is, for example, a value that increases with the increase of (1-μ) when the transmittance of the optical filter is μ, such as a value proportional to (1-μ). Alternatively, it may be a value that increases with the increase of the inverse of the transmittance μ, such as a value proportional to the inverse of μ.

The image processing device 3 b in FIG. 9 is substantially the same as the image processing device 3 in FIG. 9 , but may include a pixel projection unit 33 b instead of the pixel projection unit 33 .

The pixel projection unit 33b is substantially the same as the pixel projection unit 33 of FIG. 1 , but when synthesizing pixels of a plurality of pixel lines projected to the same pixel position in the panoramic coordinate system, a weighted average is calculated using the weights wI stored in the parameter storage unit of the motion control unit 4b.

According to Embodiment 2, by providing optical filters with different transmittances in a plurality of detection element lines of an image sensor, pixel values corresponding to different exposure amounts can be obtained. Therefore, by synthesizing the pixel values of pixels corresponding to the detection elements of a plurality of detection element lines, a panoramic image with a wider dynamic range can be generated.

Implementation method 3.

FIG. 11 shows an image generation system according to Embodiment 3 of the present invention.

The image generation system shown in FIG. 11 is substantially the same as the image generation system of FIG. 1 , but includes an image processing device 3 c , an operation control unit 4 c , and an image sensor 6 c instead of the image processing device 3 , the operation control unit 4 , and the image sensor 6 of FIG. 1 .

The image sensor 6c of FIG. 11 can change the exposure time TeI for each detection element line.

The exposure time TeI of each detection element line is controlled by the operation control unit 4c. The operation control unit 4c also generates and outputs a weight wI (I is 1, 2, 3 or 4) corresponding to the exposure time TeI.

The shorter the exposure time TeI is, the larger the weight wI corresponding to the exposure time is. For example, the weight wI is a value proportional to the inverse of the exposure time TeI.

The image processing device 3 c is substantially the same as the image processing device 3 in FIG. 1 , but includes a pixel projecting unit 33 c instead of the pixel projecting unit 33 .

The pixel projection unit 33c is substantially the same as the pixel projection unit 33 of FIG. 1 , but when synthesizing pixels of a plurality of pixel lines projected to the same pixel position in the panoramic coordinate system, a weighted average is calculated using the weight wI supplied from the operation control unit 4c.

According to Embodiment 3, pixel values corresponding to different exposure times can be obtained from a plurality of detection element lines of the image sensor. Therefore, by synthesizing the pixel values of pixels corresponding to the detection elements of the plurality of detection element lines, a panoramic image with a wider dynamic range can be generated.

Implementation method 4.

FIG. 12 shows an image generation system according to Embodiment 4 of the present invention.

The image generation system shown in FIG. 12 is substantially the same as the image generation system of FIG. 1 , but includes an image processing device 3 d and an operation control unit 4 d instead of the image processing device 3 and the operation control unit 4 of FIG. 1 .

The image processing device 3 d is substantially the same as the image processing device 3 , but is additionally provided with a reference image storage unit 36 and an image quality improvement processing unit 37 .

The image generation system shown in Fig. 12 operates in a reference image generation mode or a real-time image generation mode. In the reference image generation mode, a high-sensitivity panoramic image is generated with high resolution and stored in the reference image storage unit 36 as a reference image Dr. In the real-time image generation mode, a high-quality image is processed using the reference image Dr on a normal panoramic image Db acquired in real time.

The operation in the reference image generation mode is performed, for example, periodically or in response to a user's request.

When generating the reference image Dr, the operation control unit 4d causes the imaging unit 1, the rotation drive system 2, and the pixel line extraction unit 31, the projection position calculation unit 32, and the pixel projection unit 33 of the image processing device 3d to operate in the same manner as in the generation of a normal panoramic image. However, the operating conditions of the imaging unit 1, the rotation drive system 2, the pixel line extraction unit 31, and the projection position calculation unit 32 are different as follows.

First, the operation control unit 4d sets the angular velocity of the rotation of the rotation drive system 2 to a lower value. That is, the imaging unit 1 is rotated at a lower angular velocity Va than when generating a normal panoramic image. As a result of reducing the angular velocity Va, the imaging interval Δθ can be reduced.

The operation control unit 4 d may change the rotation angular velocity and increase the number of extraction lines N as described above.

In order to increase the number of extraction lines N, the field angle β _H of each detection element line of the image sensor may be made smaller than the horizontal field angle γ _H of each pixel of the reference image Dr.

The change of the field angle _βH of each detection element line is achieved by the operation control unit 4d changing the optical magnification of the imaging optical system 7. For example, _βH can be reduced by increasing the optical magnification. Alternatively, in order to change the field angle _βH , the operation control unit 4d may change the extracted pixel lines. For example, βH can be reduced by changing from a state in which every other column in the imaging DIN is extracted as pixel lines S1 to S4 to a state in which continuous _columns are extracted as pixel lines S1 to S4.

When the extracted pixel line is changed, the action control unit 4d may also transmit information Gs indicating the position of the changed pixel line to the pixel line extraction unit 31, and convey information Gp indicating the position of the detection element line corresponding to the changed pixel line and the changed field of view angle _βH to the projection position calculation unit 32.

The panoramic image generated by the pixel projection unit 33 is stored in the reference image storage unit 36 as a reference image Dr.

In the real-time image generation mode, the camera unit 1, the rotation drive system 2, and the pixel line extraction unit 31, the projection position calculation unit 32 and the pixel projection unit 33 of the image processing device 3d operate in the same manner as the respective parts described in Implementation Example 1, and the panoramic image Db generated by the pixel projection unit 33 is supplied to the high image quality processing unit 37.

The image quality enhancement processing unit 37 enhances the image quality of the panoramic image Db using the reference image Dr stored in the reference image storage unit 36. The image Dc obtained as a result of the image quality enhancement is output as a real-time panoramic image DOUT.

As the image quality enhancement processing using the reference image Dr, for example, edge-preserving smoothing filter processing using a joint bilateral filter may be performed, or guided high-resolution enhancement processing using a guided filter may be performed.

By performing the high-quality image processing as described above, a panoramic image with a higher resolution or higher sensitivity than before can be generated. For example, in the application of storing the unchanged part of the subject as the background image and obtaining the camera image of the changed part in real time, the overall image quality can be improved.

All or part of the image processing devices and the operation control unit in Embodiments 1 to 4 can be constituted by a processing circuit.

For example, the functions of each part of the image processing device 3, 3b, 3c or 3d and the action control unit 4, 4b, 4c or 4d may be implemented by separate processing circuits, or the functions of multiple parts may be implemented by one processing circuit.

The processing circuit may be constituted by hardware or by software, that is, a programmed computer.

Some of the functions of the image processing device 3, 3b, 3c or 3d and the operation control unit 4, 4b, 4c or 4d may be implemented by hardware, and the other parts may be implemented by software.

FIG. 13 shows a configuration example of a computer 9 that realizes all functions of the image processing device 3 , 3 b , 3 c , or 3 d and the operation control unit 4 , 4 b , 4 c , or 4 d .

In the example shown in the figure, the computer 9 includes a processor 91 and a memory 92 .

The memory 92 stores a program for realizing the functions of each section of the image processing device 3 , 3 b , 3 c , or 3 d and the operation control section 4 , 4 b , 4 c , or 4 d .

As the processor 91 , for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, a DSP (Digital Signal Processor), or the like is used.

The memory 92 uses, for example, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM) or EEPROM (Electrically Erasable Programmable ROM) or other semiconductor memories, magnetic disks, optical disks, or magneto-optical disks.

The processor 91 implements the functions of the image processing device 3 , 3 b , 3 c , or 3 d and the operation control unit 4 , 4 b , 4 c , or 4 d by executing the program stored in the memory 92 .

The functions of the image processing device 3, 3b, 3c or 3d include generation of a panoramic image.

The functions of the operation control unit 4, 4b, 4c, or 4d include control of the rotation drive system and control of the imaging optical system.

The computer in FIG. 13 includes a single processor, but may include two or more processors.

The above-mentioned processing circuit may also be attached to the image sensor 6. That is, the image processing device 3, 3b, 3c or 3d and the motion control unit 4, 4b, 4c or 4d may be installed in the processing circuit attached to the image sensor 6. Alternatively, the image processing device 3, 3b, 3c or 3d and the motion control unit 4, 4b, 4c or 4d may be installed on a cloud server that can be connected to the image sensor 6 via a communication network.

The image processing device 3, 3b, 3c or 3d and the action control unit 4, 4b, 4c or 4d may also be installed in a communication portable terminal such as a smart phone or a remote controller.

The image generation system may also be applied to home appliances. In this case, the image processing device 3, 3b, 3c or 3d and the motion control unit 4, 4b, 4c or 4d may also be installed in a HEMS (Home Energy Management System) controller.

Hereinafter, a processing procedure in a processor when the functions of the image processing device 3 d and the operation control unit 4 d of the image generation system of FIG. 12 are realized by the computer of FIG. 13 will be described with reference to FIGS. 14 to 16 .

FIG14 shows an overview of the processing steps.

In the steps shown in FIG. 14 , first in step ST1 , a determination is made as to whether the image generation system is to be operated in the reference image generation mode or the real-time image generation mode.

The operation in the reference image generation mode may be performed, for example, periodically or in response to a user's request.

When the operation is to be performed in the real-time image generation mode, real-time image generation is performed (ST2).

When the operation is to be performed in the reference image generation mode, reference image generation is performed (ST3).

FIG. 15 shows the steps of the operation ( ST2 ) in the real-time image generation mode.

In the following, it is assumed that the imaging unit 1 rotates from the starting point Xa to the ending point Xb to generate a panoramic image. Which column in the detection element array of the image sensor 6 is set as the detection element line is predetermined, and the detection element line position information Gp and the pixel line position information Gs are stored in advance in the parameter storage unit of the motion control unit 4d. In addition, the horizontal field angle _γH of each pixel of the panoramic image is also predetermined.

In step ST11, the operation control unit 4d determines the operation conditions. The operation conditions here include the angular velocity Va of the rotation of the imaging unit 1, the imaging cycle Tf, and the field angle _βH of each detection element line.

In step ST12, the operation control unit 4d controls the optical magnification of the imaging optical system 7. The optical magnification is controlled so that the field angle _βH of each detection element line becomes the value determined in step ST11.

In step ST13, the operation control unit 4d notifies the projection position calculation unit 32 of the field angle _βH of each detection element line.

In step ST14 , the operation control unit 4 d notifies the projection position calculation unit 32 of the detection element line position information Gp, and notifies the pixel line extraction unit 31 of the pixel line position information Gs.

The process of step ST13 can be performed in parallel with the process of step ST12 , and the process of step ST14 can be performed in parallel with the processes of steps ST11 to ST13 .

In step ST15 , the operation control unit 4 d starts scanning by the imaging unit 1 .

In step ST16 , the image sensor 6 performs imaging (acquires one frame of image). For example, imaging is performed at each imaging cycle Tf determined in step ST11 . The captured image DIN is output to the pixel line extraction unit 31 .

In step ST17 , the pixel line extraction unit 31 extracts a plurality of pixel lines S1 to S4 from the captured image DIN based on the pixel line position information Gs acquired in step ST14 .

In step ST18 , the operation control unit 4 d acquires the rotation angle θ when the imaging in step ST16 is performed, and notifies the projection position calculation unit 32 of the acquisition angle θ.

In step ST19 , the projection position calculation unit 32 calculates the projection position using the field angle β _H of each detection element line notified in step ST13 , the detection element line position information Gp notified in step ST14 , and the rotation angle θ notified in step ST18 .

The processing of steps ST18 and ST19 can be performed in parallel with the processing of step ST17.

In step ST20 , the pixel projection unit 33 performs projection and synthesis, and writes the image to the internal panoramic image storage unit. In projection, each pixel of the pixel line extracted in step ST17 is associated with the projection position calculated in step ST19 .

In step ST21 , the operation control unit 4 d determines whether the imaging direction of the imaging unit 1 has reached the end point Xb.

If it has not reached Xb, the process returns to step ST16. In this case, the image is taken in the next image-taking cycle.

If it reaches Xb, it proceeds to steps ST22 and ST23.

In step ST22 , the scanning by the imaging unit 1 is terminated.

In step ST23 , the image quality improvement processing unit 37 improves the image quality of the panoramic image of one frame written into the panoramic image storage unit in the pixel projection unit 33 using the reference image Dr stored in the reference image storage unit 36 .

After steps ST22 and ST23 are completed, the processing is terminated.

FIG. 16 shows the procedure of the operation (ST3) in the reference image generation mode.

Among the steps shown in Fig. 16, steps ST11 to ST22 are the same as the steps shown in Fig. 15. However, the operating conditions are different.

For example, in step ST11, the angular velocity Va is set to be lower. Also, _βH is set to be smaller than _γH .

In FIG. 16 , step ST23 in FIG. 15 is replaced by step ST24 .

In step ST24 , the generated panoramic image is written into the reference image storage unit 36 as the reference image Dr.

In addition, when any of the operating conditions determined in step ST11 is predetermined, its value may be used. In addition, when it is not predetermined which column in the detection element array of the image sensor 6 is set as the detection element line, it may be determined which column is set as the detection element line, and the detection element line position information Gp and the pixel line position information Gs are generated. In addition, the horizontal field angle γ _H of each pixel of the panoramic image may also be determined in step ST11.

When the functions of the image processing device 3, 3b or 3c or 3d and the action control unit 4, 4b or 4c of the image generating system of Figure 1, Figure 9 or Figure 11 are implemented by the computer of Figure 13, the processing steps in the processor are the same as the processing of steps ST11 to ST22 of Figure 15.

As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment.

For example, although Embodiment 4 has been described as a modification of Embodiment 1, similar modifications can be applied to Embodiments 2 and 3. Furthermore, the modifications described with respect to Embodiment 1 can also be applied to Embodiments 2 to 4.

In the first to fourth embodiments, the pixel line extraction unit 31 extracts pixel lines from the captured image DIN output from the imaging unit 1 , but only pixel lines to be extracted may be output from the imaging unit 1 . In this case, the pixel line extraction unit 31 is unnecessary.

In Embodiments 1 to 4, a two-dimensional image sensor is used as the image sensor, but instead, the image sensor 6 may be composed of a plurality of one-dimensional image sensors arranged in parallel with each other. In this case, the pixel line extraction unit 31 is also unnecessary.

In the first to fourth embodiments, the image sensor is a thermal image sensor, but this is not essential, and the image sensor may be an image sensor other than a thermal image sensor. For example, it may be an image sensor that receives visible light and performs imaging.

As described above, according to the present invention, by connecting images captured while rotating the imaging unit, it is possible to achieve at least one of a higher resolution and a higher SN ratio of the panoramic image.

The image generation system of the present invention has been described above, but the image processing device constituting the image generation system, the image generation method implemented by the image generation system, and the image processing method implemented by the image processing device also constitute a part of the present invention. In addition, a program that causes a computer to execute the processing in the above-mentioned device or method, and a recording medium such as a non-transitory recording medium that can be read by a computer recording the program also constitute a part of the present invention.

Description of Reference Numerals

1 camera unit, 2 rotation drive system, 3, 3b, 3c, 3d image processing device, 4, 4b, 4c, 4d action control unit, 6, 6b, 6c image sensor, 7 camera optical system, 31 pixel line extraction unit, 32 projection position calculation unit, 33, 33b, 33c pixel projection unit, 36 reference image storage unit, 37 high image quality processing unit.

Claims (16)

1. An image generation system, wherein: The image generation system comprises: an imaging unit that is rotatable about a rotation axis and has a plurality of detection element lines extending parallel to the rotation axis and each consisting of a plurality of detection elements, the imaging unit generating a captured image including pixels corresponding to the detection elements of the detection element lines by imaging; and An image processing device calculates corresponding pixel positions in a panoramic coordinate system as projection positions for pixels respectively corresponding to the detection elements, projects the pixels to the projection positions, synthesizes the pixel values of the projected two or more pixels at the pixel positions in the panoramic coordinate system where two or more pixels are projected, calculates a synthesized value, and generates a panoramic image including a plurality of pixels each having the synthesized value as a pixel value. 2. The image generation system according to claim 1, wherein: The image processing device calculates the projection position based on position information indicating the position of each of the plurality of detection element lines, the field angle of each of the detection element lines, and information indicating the imaging direction of the imaging unit. 3. The image generation system according to claim 1, wherein: The panoramic coordinate system uses the direction of the rotation axis and the direction of rotation around the rotation axis as coordinates. 4. The image generation system according to claim 2, wherein: The panoramic coordinate system uses the direction of the rotation axis and the direction of rotation around the rotation axis as coordinates. 5. The image generation system according to any one of claims 3 to 4, wherein: The angle of view of the rotation direction of each pixel of the panoramic image is set to be smaller than the angle of view of each of the detection element lines. 6. The image generation system according to any one of claims 3 to 4, wherein: The angle of view in the direction of the rotation of each pixel of the panoramic image is set to be larger than the angle of view of each of the detection element lines. 7. The image generation system according to any one of claims 1 to 4, wherein: The plurality of detection element lines are selected in such a manner that pixels corresponding to the detection elements of the detection element lines are located in a portion of the captured image where lens distortion is small. 8. The image generation system according to any one of claims 1 to 4, wherein: The plurality of detection element lines are provided with optical filters having different transmittances. When synthesizing pixel values of two or more pixels, the image processing device performs weighted averaging using a weight corresponding to the transmittance of an optical filter provided on a detection element line including a detection element corresponding to the pixel. 9. The image generation system according to any one of claims 1 to 4, wherein: The imaging unit is controlled so that the exposure time differs for each of the detection element lines. When synthesizing pixel values of two or more pixels, the image processing device performs weighted averaging using the reciprocal of the exposure time for a detection element line including the detection element corresponding to the pixel as a weight. 10. The image generation system according to any one of claims 1 to 4, wherein: The camera unit includes a two-dimensional image sensor. The plurality of detection element lines are respectively formed by detection elements arranged in a straight line among the detection elements of the two-dimensional image sensor. 11. The image generation system according to any one of claims 1 to 4, wherein: The camera unit includes a plurality of one-dimensional image sensors. The plurality of detection element lines are respectively constituted by the plurality of one-dimensional image sensors. 12. The image generation system according to any one of claims 1 to 4, wherein: The image generation system further includes a rotation drive system that moves the imaging unit at a constant angular velocity. 13. The image generation system according to claim 12, wherein: The angular velocity or the imaging period is determined so that the product of the angular velocity of the rotation of the imaging unit and the imaging period of the imaging unit is smaller than the field angle of each of the detection element lines. 14. The image generation system according to any one of claims 1 to 4, wherein: The image generation system further includes a rotation drive system that causes the imaging unit to perform step rotational motion. 15. An image processing device, which generates a panoramic image based on a camera image generated by shooting by a camera unit, the camera unit being rotatable around a rotation axis and having a plurality of detection element lines extending parallel to the rotation axis and each consisting of a plurality of detection elements, the camera image including pixels corresponding to the detection elements of the detection element lines, wherein: The image processing device comprises: a projection position calculation unit for calculating corresponding pixel positions in a panoramic coordinate system as projection positions for pixels corresponding to the detection elements respectively; and a pixel projection unit that projects the pixel to the projection position, and synthesizes the pixel values of the projected two or more pixels at a pixel position in the panoramic coordinate system where two or more pixels are projected to calculate a synthesized value, The image processing device generates a panoramic image including a plurality of pixels each having the composite value as a pixel value. 16. A computer-readable recording medium, wherein: The recording medium records a program for causing a computer to execute the processing in the image processing apparatus according to claim 15 .