JP5952766B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method.

  In order to generate a panoramic image, a user moves an imaging region little by little to perform imaging, and a plurality of image data obtained thereby are connected. At this time, an appropriate panoramic image cannot be generated unless a predetermined amount of overlapping area between a plurality of image data is secured. Therefore, in order to secure a predetermined amount of an area where a plurality of image data overlap, a technique is disclosed that guides and displays the direction and position where the user should move the camera during imaging (for example, Patent Document 1).

JP 2009-60278 A

  However, in the technique described in Patent Document 1, the movement amount of the guide display is a fixed amount. For this reason, for example, when there are few clear feature points in the portion of the image that is a joint between a plurality of image data and it is difficult to join, it is necessary for joining if the amount of movement of the guide display is set too large in advance. The number of feature points may be insufficient. On the other hand, if there are many feature points in the image of the portion that is a joint between a plurality of image data and it is easy to connect, if the amount of movement of the guide display is set too small in advance, a wasteful imaging operation is performed for the user. There is a risk of being forced.

  The present invention has been made in view of the above circumstances, and an imaging apparatus and an imaging system that enable a wasteful imaging operation while securing the number of image feature points necessary for joining a plurality of image data It aims to provide a method.

  An imaging apparatus according to the present invention is an imaging apparatus that acquires a plurality of image data in order to generate panoramic image data by combining a plurality of image data, and the plurality of image data is captured by an imaging frame indicating an imaging region. An imaging unit that is acquired by capturing a clipped image, a movement information acquisition unit that acquires movement direction information indicating a direction in which the imaging frame moves, and an imaging frame in a direction indicated by the movement direction information acquired by the movement information acquisition unit Image feature point detecting means for detecting, from the first image data, image feature points in a region where the imaging frame and the first image data acquired by the imaging means overlap according to the amount of movement of the image, To obtain second image data adjacent to the first image data in accordance with the number of feature points of the image detected by the feature point detection means Comprising a moving distance determining means for image frames to determine the distance to be moved, and a guide means for performing an output corresponding to the distance the imaging frame should move as determined by the moving distance determining means.

  An imaging method according to the present invention is an imaging method for acquiring a plurality of image data in order to generate panoramic image data by synthesizing a plurality of image data. The plurality of image data is acquired by an imaging frame indicating an imaging region. An imaging step that is acquired by capturing an image to be cut out, a movement information acquisition step that acquires movement direction information that indicates the direction in which the imaging frame moves, and an imaging frame that is in the direction indicated by the movement direction information acquired in the movement information acquisition step An image feature point detecting step for detecting, from the first image data, a feature point of the image in a region where the imaging frame and the first image data acquired in the imaging step overlap according to the amount of movement of the image; The second adjacent to the first image data according to the number of feature points of the image detected in the feature point detection step. Comprising a moving distance determining step of imaging frames to determine the distance to be moved in order to acquire the image data, and a guide step of performing an output corresponding to the distance the imaging frame should move determined in travel distance decision step.

  According to these inventions, when capturing a plurality of image data, movement direction information indicating the direction in which the imaging frame indicating the imaging region moves is acquired. The feature point of the image in the region where the imaging frame and the first image data acquired by the imaging means overlap according to the amount of movement of the imaging frame in the direction indicated by the acquired movement direction information is the first image. Detected from data. In accordance with the number of feature points of the detected image, a distance that the imaging frame should move to determine the second image data adjacent to the first image data is determined. And the output according to the distance which an imaging frame should move is performed. Thus, the imaging operation can be performed based on the distance to which the imaging frame grasped by the output should move, and the number of feature points of the portion of the image serving as a joint between the plurality of image data can be calculated as the panorama image data. It is possible to secure as much as necessary for the generation of the image data (connection of a plurality of image data). Therefore, it is possible to provide an imaging apparatus and an imaging method that enable an imaging operation without waste while ensuring the number of image feature points necessary for joining a plurality of image data.

  In the imaging apparatus according to the present invention, the movement information acquisition unit may detect the direction in which the imaging frame has moved, and acquire the movement direction information using the moved direction. In this case, it is possible to appropriately acquire the moving direction information of the imaging frame while detecting the direction in which the imaging frame has moved.

  In the imaging apparatus according to the present invention, the guide unit may perform output according to the movement direction information acquired by the movement information acquisition unit. In this case, a more appropriate imaging operation can be performed based on the movement direction information grasped by the output.

  In the imaging apparatus according to the present invention, the movement information acquisition unit detects the distance moved by the imaging frame, and the guide unit performs output according to the distance moved by the imaging frame detected by the movement information acquisition unit. May be. In this case, a more appropriate imaging operation can be performed based on the distance moved by the imaging frame grasped by the output.

  In the imaging apparatus according to the present invention, the difference between the distance to which the imaging frame determined by the movement distance determination unit should move and the distance to which the imaging frame detected by the movement information acquisition unit has moved has reached a predetermined threshold value. A frame arrival determination means for determining whether or not the guide means outputs an output indicating that the movement of the imaging frame should be stopped when the frame arrival determination means determines that the difference has reached a predetermined threshold value. You may go. In this case, a more appropriate imaging operation can be performed based on the timing at which the movement of the imaging frame grasped by the output should be stopped. In addition, it is possible to perform an imaging operation without blurring.

  In the imaging apparatus according to the present invention, the guide unit may perform output by screen display. In this case, by performing the above-described output by screen display, the user can visually obtain information on how to move the imaging frame. Therefore, the user can perform a more appropriate imaging operation.

  In the imaging apparatus according to the present invention, the guide unit may display a frame or an arrow as the screen display. In this case, the user can perform a more appropriate imaging operation according to a visually comprehensible display such as the frame or the arrow.

  According to the present invention, it is possible to provide an image pickup apparatus and an image pickup method that enable an image pickup operation without waste while ensuring the number of image feature points necessary for connecting a plurality of pieces of image data.

1 is an external view illustrating an external configuration of an imaging apparatus according to an embodiment of the present invention. It is a block diagram which shows the hardware constitutions of the imaging device shown in FIG. It is a block diagram which shows the functional structure of the imaging device shown in FIG. It is a figure explaining the judgment logic by the image feature point detection part and movement distance determination part of FIG. It is a figure which shows the aspect of a display of the imaging guide by the guide part of FIG. It is a figure explaining that the display of the imaging guide in FIG. 5 is according to the distance which the imaging frame should move. 4 is a flowchart illustrating an operation of controlling an imaging guide by a guide unit in FIG. 3. FIG. 8 is a flowchart showing in detail the operations of image feature point detection and movement distance determination in FIG. 7. FIG. It is a figure which shows another aspect of a display of the imaging guide of FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is an external view showing an external configuration of an imaging apparatus according to an embodiment of the present invention. As illustrated in FIG. 1, the imaging device 1 is a mobile terminal configured by a housing 2, for example, and includes a touch panel display 4 and a camera 6 disposed on one surface of the housing 2. When the touch panel display 4 is touched (operated by a finger, a touch pen, or the like) by the user, the imaging apparatus 1 executes an operation of the camera 6 or operations in various applications.

  The camera 6 includes a front-side camera 6a disposed on the front surface 2a that is a surface on which the touch panel display 4 of the housing 2 is disposed, and a back-side camera 6b disposed on the back surface 2b that is a surface facing the front surface 2a. including. The camera 6 has an imaging function of a camera provided in a normal mobile terminal, and can capture an image of a predetermined area to which the camera 6 is directed. A shutter button 12 is displayed in the touch panel display 4. When the imaging device 1 detects that the shutter button 12 is pressed by a user's touch operation, the captured image is acquired as data. Note that the imaging device 1 acquires the captured image as data not only at the timing of detecting the pressing of the shutter button 12 but also at the timing according to the movement of the imaging frame described later.

  The imaging device 1 has a panoramic image imaging mode for generating a panoramic image by combining a plurality of image data. The imaging device 1 may be included in a panoramic photo generation system that generates a panoramic image (panoramic photo) from a plurality of image data acquired by the imaging device 1. Note that the same method as the conventional method can be used to generate the panoramic image. The imaging device 1 continuously captures images with the camera 6 in a state in which the panorama image capturing mode is set, thereby acquiring an image cut out by an imaging frame indicating an imaging region as screen display data. The acquired screen display data is displayed on the touch panel display 4. In addition, when detecting the pressing of the shutter button 12 by the user, the imaging device 1 acquires a plurality of pieces of image data necessary for generating a panoramic image among images cut out by an imaging frame indicating an imaging region. Hereinafter, one of the plurality of image data necessary for generating the panoramic image is simply referred to as “first image data”, and is adjacent to and overlaps the first image data among the plurality of image data. Image data having a region is simply referred to as “second image data”. The second image data is acquired after the first image data is acquired. By repeatedly acquiring the first image data and the second image data, all of the plurality of image data necessary for generating the panoramic image is acquired. In other words, the second image data is the first image data for the next acquired image data.

  When the imaging device 1 acquires the first image data 40, the imaging device 1 displays the first image data 40 and an imaging guide including a frame 46a and an arrow 46b on the touch panel display 4. The imaging guide frame 46a guides the position of the imaging frame 42 to the position where the second image data should be cut out in order to acquire a plurality of image data necessary for generating a panoramic image. Various shapes corresponding to the shape of 42 are exhibited. Hereinafter, the configuration of the imaging apparatus 1 in a state where the imaging apparatus 1 is set to the panoramic image imaging mode will be described in detail with reference to FIGS.

  FIG. 2 is a block diagram showing a hardware configuration of the imaging apparatus 1 shown in FIG. As shown in FIG. 2, the imaging apparatus 1 includes a camera 6, a CPU 101 that executes camera control processing, an operating system, an application program, and the like, a main storage unit 102 that includes a ROM and a RAM, a hard disk, a memory, and the like. An auxiliary storage unit 103, an input unit 104 such as a touch panel display 4, and an output unit 105 such as a touch panel display 4. Each function in an imaging unit, a movement information acquisition unit, an image feature point detection unit, a movement distance determination unit, a frame arrival determination unit, and a guide unit, which will be described later, causes the CPU 101 and the main storage unit 102 to read predetermined software and control the CPU 101. Will be executed.

  FIG. 3 is a block diagram illustrating a functional configuration of the imaging apparatus illustrated in FIG. 1. As shown in FIG. 3, the imaging device 1 functionally includes an imaging unit 20, a movement information acquisition unit 22, an image feature point detection unit 24, a movement distance determination unit 26, a frame arrival determination unit 28, and a guide unit 30. I have. Each configuration will be described below.

  The imaging unit 20 is an imaging unit that captures an image cut out by an imaging frame and acquires a plurality of image data necessary for generating a panoramic image. The imaging frame indicates an area cut out as image data in the subject area, that is, an imaging area. The imaging frame indicates, for example, a predetermined area centered in the direction in which the camera 6 is directed by a rectangular frame. The imaging unit 20 acquires an image cut out by the imaging frame as screen display data. When the imaging unit 20 detects that the user has pressed the shutter button 12, the imaging unit 20 acquires first image data 40 that is one of a plurality of pieces of image data necessary for generating a panoramic image. After acquiring the first image data 40, the imaging unit 20 outputs that the imaging frame has reached the position where the second image data should be cut out by a frame arrival determination unit 28, which will be described later. When the fact that the imaging frame is stationary is output by 22, second image data that is one of a plurality of image data necessary for generating a panoramic image is acquired. The time from when the frame arrival determination unit 28 outputs that the imaging frame has reached the position where the second image data should be cut out until the movement information acquisition unit 22 outputs that the imaging frame is stationary. Is not particularly defined, but may be within a preset time. Accordingly, the user operates the shutter button 12 only when acquiring the first first image data, and after the acquisition of the first first image data, the image capturing unit 20 automatically performs the operation after the acquisition of the first image data. A plurality of pieces of image data necessary for generating a panoramic image can be acquired. Note that the plurality of pieces of image data may be acquired by operating the shutter button 12 by the user even after the acquisition of the first first image data.

  Hereinafter, the distance that the imaging frame should move from the position where the first image data 40 is cut out to the position where the second image data should be cut out is simply referred to as “distance Y where the imaging frame should move”. Note that the distance Y to which the imaging frame should move is calculated by the movement distance determination unit 26 described later. The imaging unit 20 acquires all of the plurality of image data necessary for generating the panoramic image by repeating the acquisition of the first image data 40 and the second image data a plurality of times. The screen display data, the first image data 40, and the second image data cut out by the imaging frame acquired by the imaging unit 20 are output to the movement information acquisition unit 22, the image feature point detection unit 24, and the guide unit 30, which will be described later. Is done.

  The movement information acquisition unit 22 is movement information acquisition means for acquiring movement direction information indicating the direction in which the imaging frame moves. When the imaging unit 20 outputs the first image data 40, the movement information acquisition unit 22 displays four directions (up, down, left, and right) corresponding to the four sides indicated by the first image data 40 cut out by the rectangular imaging frame. Then, it is obtained as movement direction information from information stored in advance. The movement information acquisition unit 22 detects the moving direction of the imaging frame when the imaging frame is moved by moving the imaging device 1 in order for the user to acquire the second image data, and the detected direction is detected. Is acquired as movement direction information. More specifically, the movement information acquisition unit 22 detects the position coordinates of the feature points of the image from the screen display data output by the imaging unit 20, and the imaging frame has moved based on the change in the position coordinates. Detect direction. An image feature point is a point that becomes a feature such as a border or corner of an image, and is, for example, a location where the color of the image changes greatly, a dark portion, a contour, or the like. The position coordinates of the feature points of the image may be detected by an image feature point detection unit 24 described later and output to the movement information acquisition unit 22. The movement information acquisition unit 22 may acquire the movement direction information by detecting the acceleration in the vertical and horizontal directions of the camera 6. In this case, for example, the imaging unit 20 may have a function of detecting the acceleration of the camera 6 using a triaxial acceleration sensor or the like, and the detected acceleration of the camera 6 may be output to the movement information acquisition unit 22. In addition to the acceleration or instead of the acceleration, the moving direction information may be acquired by detecting the angular velocity. The movement direction information acquired by the movement information acquisition unit 22 is output to an image feature point detection unit 24 and a guide unit 30 described later.

  The movement information acquisition unit 22 is movement information acquisition means for detecting the distance that the imaging frame has moved. The movement information acquisition unit 22 detects the position coordinates of the feature points of the image from the screen display data output by the imaging unit 20, and the position where the first image data 40 is cut out based on the change of the position coordinates. , The distance the imaging frame has moved (hereinafter simply referred to as “distance X the imaging frame has moved”) is detected. In addition, you may perform the detection of the distance X to which the imaging frame moved according to the direction to which the imaging frame moved. The distance X by which the imaging frame detected by the movement information acquisition unit 22 has moved is output to a frame arrival determination unit 28 and a guide unit 30 described later. Further, the movement information acquisition unit 22 detects the position coordinates of the feature points of the image from the screen display data output by the imaging unit 20, and detects that the imaging frame is stationary based on the change of the position coordinates. May be. Note that the movement information acquisition unit 22 may detect that the imaging frame is stationary by detecting at least one of the acceleration and the angular velocity of the camera 6. When the movement information acquisition unit 22 detects that the imaging frame is stationary, the fact is output to the imaging unit 20.

  The image feature point detection unit 24 detects an image feature point in the region where the first image data 40 output from the imaging unit 20 and the virtual imaging frame overlap from the first image data 40. It is a means of getting out. FIG. 4 is a diagram for explaining determination logic by the image feature point detection unit and the movement distance determination unit in FIG. As shown in FIG. 4, the virtual imaging frame 41 has the same shape as the actual imaging frame 42 indicating a predetermined imaging area centered on the direction in which the camera 6 is directed, but the image feature points are It differs from the actual imaging frame 42 in that it is virtually set to indicate the area to be detected. The virtual imaging frame 41 is calculated based on the movement direction information output by the movement information acquisition unit 22. Hereinafter, the virtual imaging frame 41 is simply referred to as “virtual frame” 41 and is distinguished from the above-described actual imaging frame 42.

  The image feature point detector 24 detects the feature point A of the image of the first image data 40 in a region 44 where the first image data 40 and the virtual frame 41 overlap. The feature point A of the image is detected by, for example, generating luminance image data having a luminance component value as a pixel value from the first image data 40 and detecting an edge or the like in the luminance image data. The image feature point detection unit 24 detects, for example, the number of pixels having a luminance component value exceeding a predetermined threshold value, whereby the image feature point A in the region 44 where the first image data 40 and the virtual frame 41 overlap is detected. Detect the number of It should be noted that image feature points in the entire first image data 40 are detected in advance, and the feature points of the image in the region 44 where the first image data 40 and the virtual frame 41 overlap later are used. A may be detected.

  The region 44 where the first image data 40 and the virtual frame 41 overlap corresponds to the amount of movement of the virtual frame 41 in the direction indicated by the movement direction information output by the movement information acquisition unit 22. The amount of movement of the virtual frame 41 refers to the amount of translation of the virtual frame 41 with respect to the first image data 40. As the movement amount of the virtual frame 41 increases, the area of the region 44 where the first image data 40 and the virtual frame 41 overlap decreases. Here, the area of the region 44 where the first image data 40 and the virtual frame 41 overlap is the first threshold value (for example, 25% of the total area of the virtual frame 41) that is the minimum necessary for generating a panoramic image. And can be set in advance. Further, the amount of movement itself by which the virtual frame 41 moves can be set in advance. The number of feature points A of the image in the region 44 where the first image data 40 and the virtual frame 41 detected by the image feature point detection unit 24 are output to the movement distance determination unit 26 described later.

  The moving distance determination unit 26 outputs the second image data according to the number of image feature points A in the region 44 where the first image data 40 output from the image feature point detection unit 24 and the virtual frame 41 overlap. This is a moving distance determining means for determining the distance Y that the actual imaging frame 42 should move to capture an image. Specifically, the moving distance determination unit 26 determines that the number of image feature points A in the region 44 where the first image data 40 output from the image feature point detection unit 24 and the virtual frame 41 overlap is a predetermined threshold value β. The distance Y to which the actual imaging frame 42 should move is determined so as to be (for example, 8) or more.

  4A shows a case where the distance Y that the imaging frame 42 should move is determined as a long distance, and FIG. 4B shows a case where the distance Y that the imaging frame 42 should move is determined as a short distance. Show. Before the image feature point detection unit 24 detects the number of image feature points A in the region 44 where the first image data 40 and the virtual frame 41 overlap, the movement distance determination unit 26 moves the imaging frame 42. The distance Y to be set is set to a long distance as a default value. Here, the long distance refers to the distance from the position where the first image data 40 is cut out to the position of the virtual frame 41 set as described above. That is, when the distance Y to which the imaging frame 42 should move is a long distance, the area of the region 48 where the first image data 40 and the actual imaging frame 42 overlap is such that the first image data 40 and the virtual frame 41 are the same. Similar to the area of the overlapping region 44, this is the first minimum threshold necessary for generating a panoramic image (for example, 25% of the total area of the imaging frame 42).

  As shown in FIG. 4A, the moving distance determination unit 26 determines the feature point A of the image in the region 44 where the first image data 40 output from the image feature point detection unit 24 and the virtual frame 41 overlap. When the number is equal to or greater than a predetermined threshold β (for example, 8), the distance Y that the actual imaging frame 42 should move is set to a long distance. On the other hand, as shown in FIG. 4B, the number of image feature points A in a region 44 where the first image data 40 output by the image feature point detection unit 24 and the virtual frame 41 overlap is a predetermined threshold value. When the number is less than (for example, 8), the imaging frame is set such that the number of image feature points A in the region 48 where the first image data 40 and the actual imaging frame 42 overlap is equal to or greater than a predetermined threshold β. The distance Y to be moved is changed from a long distance to a short distance. Here, the short distance is a short distance with respect to the long distance, and is set so that the area of the region 48 where the first image data 40 and the actual imaging frame 42 overlap is larger than that in the case of the long distance. Distance. That is, when the distance Y to which the imaging frame 42 should move is a short distance, the second threshold value in which the area 48 of the region 48 where the first image data 40 and the actual imaging frame 42 overlap is larger than the first threshold value. (For example, 75% of the total area of the imaging frame 42). However, even if the distance Y to which the imaging frame 42 should move is short, depending on the setting of the second threshold and the number of feature points in the first image data, the first image data 40 and the actual imaging frame 42 are used. There may be a case where the number of feature points A of the image in the region 48 where and overlap does not exceed the predetermined threshold value β.

  As shown in the right ends of FIGS. 4A and 4B, the area 52 where the first image data 40 and the second image data 50 overlap is such that the distance Y that the imaging frame 42 should move is long. The distance is smaller than the short distance. As described above, before the image feature point detection unit 24 detects the feature point A of the image in the region 44 where the first image data 40 and the virtual frame 41 overlap, the moving distance determination unit 26 performs the imaging frame 42. Since the distance Y to be moved is set as a default value as a long distance, the distance that the imaging frame 42 moves from the acquisition of the first image data 40 to the acquisition of the second image data 50 is maximized. be able to. That is, it is possible to reduce imaging operations for acquiring a plurality of image data for generating a panoramic image as much as possible. For example, it is possible to reduce an operation for the user to stop the imaging apparatus 1 at the imaging position. Further, when the user acquires image data by operating the shutter button 12, the number of times of operating the shutter button 12 of the camera 6 can be reduced. The distance Y to be moved by the imaging frame 42 determined by the movement distance determination unit 26 is output to a frame arrival determination unit 28 and a guide unit 30 described later.

  The frame arrival determination unit 28 stores in advance a difference between the distance Y to which the imaging frame 42 output by the movement distance determination unit 26 should move and the distance X to which the imaging frame 42 output by the movement information acquisition unit 22 has moved. Frame arrival determination means for determining whether or not the predetermined threshold value α has been reached (below α or less). The predetermined threshold value α can be set to a value in units of the number of pixels that is so small that, for example, the distance Y that the imaging frame 42 should move and the distance X that the imaging frame 42 moved can be regarded as substantially equal. That is, the frame arrival determination unit 28 determines whether or not the imaging frame 42 has reached a position where the second image data 50 should be cut out. When it is determined by the frame arrival determination unit 28 that the imaging frame 42 has reached the position where the second image data 50 should be cut out, this is output to the imaging unit 20 and a guide unit 30 described later. Note that the frame arrival determination unit 28 may determine whether or not the predetermined threshold value α is less than the predetermined threshold value α.

  The guide unit 30 is a guide unit that performs various outputs according to information output by the imaging unit 20, the movement information acquisition unit 22, the movement distance determination unit 26, and the frame arrival determination unit 28. The guide unit 30 performs the output by displaying, for example, a graphic indicating an imaging guide including a frame 46a and an arrow 46b on the touch panel display 4 (see FIG. 1). The imaging guide guides how to move the imaging frame 42 from the position where the first image data 40 is cut out to the position where the second image data is cut out. FIG. 5 is a diagram showing a display mode of the imaging guide by the guide unit of FIG. When the first image data 40 is output by the imaging unit 20, the guide unit 30 corresponds to the first image data 40 and the movement direction information output by the movement information acquisition unit 22 as illustrated in FIG. 5. The frame 46a and the arrow 46b are displayed. The frame 46a and the arrow 46b are located in the direction in which the imaging frame 42 moves. The frame 46a has a shape along one side of the direction in which the imaging frame 42 moves in the rectangle indicated by the imaging frame 42 and two corners at both ends thereof. The frame 46 a guides the position where the imaging frame 42 should cut out the second image data 50. When acquiring the second image data 50, the frame 46a should be positioned so that one side of the direction in which the imaging frame 42 moves and two corners at both ends thereof coincide with the corresponding frame 46a. It shows that. The arrow 46b has a shape extending in the direction in which the imaging frame 42 moves. The arrow 46 b guides the direction in which the imaging frame 42 should go in order to cut out the second image data 50.

  When the movement information acquisition unit 22 outputs four directions (up, down, left and right) corresponding to the four sides indicated by the first image data 40 acquired from information stored in advance as the movement direction information, the guide unit 30 outputs As shown in FIG. 5A, four frames 46a and four arrows 46b corresponding to the four directions are displayed. The positions of these four frames 46a are determined by the distance Y to which the imaging frame 42 output by the movement distance determination unit 26 should move. The positions of the four arrows 46b correspond to the positions of the four frames 46a determined in this way, and are determined so as to be positioned between the first image data 40 and the four frames 46a. The positions of the four frames 46a and the four arrows 46b may be set in advance. When the movement information acquisition unit 22 outputs the direction in which the imaging frame 42 has moved, the guide unit 30 displays the frame 46a only in the direction in which the imaging frame 42 has moved, as shown in FIG. And an arrow 46b. That is, the guide unit 30 hides the frame 46a and the arrow 46b other than the direction in which the imaging frame 42 has moved.

  For example, the guide unit 30 displays the frame 46a at a position corresponding to the distance Y to which the imaging frame 42 output by the movement distance determination unit 26 should move as follows. FIG. 6 is a diagram for explaining that the display of the imaging guide in FIG. 5 corresponds to the distance to which the imaging frame should move. In FIG. 6, the frame 46 a is simply shown as a one-dot chain line rectangle for explanation. 6A shows the display of the imaging guide when the distance Y that the imaging frame 42 should move is a long distance, and FIG. 6B shows the imaging when the distance Y that the imaging frame 42 needs to move is a short distance. The guide display is shown. When the distance Y to be moved by the imaging frame 42 output by the movement distance determining unit 26 is a long distance, the area of the region 48 where the first image data 40 and the imaging frame 42 overlap is the first threshold value. A frame 46a is displayed as shown in FIG. 6A in accordance with the position of the imaging frame 42 (for example, 25% of the total area of the imaging frame 42) (see FIG. 4A). . On the other hand, when the distance to which the imaging frame 42 output by the movement distance determination unit 26 should move is short, the area 48 of the region 48 where the first image data 40 and the imaging frame 42 overlap is the second area. A frame 46a is displayed as shown in FIG. 6B in accordance with the position of the imaging frame 42 (see FIG. 4B) such that the threshold value (for example, 75% of the area of the imaging frame 42) is reached. To do.

  In addition, the guide unit 30 has an arrow 46b depending on the distance X that the imaging frame 42 output by the movement information acquisition unit 22 has moved and the distance Y that the imaging frame 42 output by the movement distance determination unit 26 should move. Change the displayed size. Specifically, in correspondence with the difference between the distance Y to which the imaging frame 42 output by the movement distance determination unit 26 should move and the distance X to which the imaging frame 42 output by the movement information acquisition unit 22 has moved, The arrow 46b is displayed so that the size of the arrow 46b decreases as the difference decreases. That is, as shown in FIGS. 5B and 5C, the size of the arrow 46b decreases as the position of the imaging frame 42 approaches the position of the frame 46a.

  Further, when the frame arrival determination unit 28 determines that the imaging frame 42 has reached the position where the second image data 50 should be extracted, the guide unit 30 indicates that the movement of the imaging frame 42 should be stopped. Output. Specifically, as shown in FIG. 5D, a stationary mark 46c is displayed near the center of the area indicated by the imaging frame 42.

  Next, the operation for controlling the imaging guide in the imaging method using the imaging apparatus 1 will be described with reference to FIG. FIG. 7 is a flowchart showing an operation of controlling the imaging guide by the guide unit of FIG.

  As illustrated in FIG. 7, in the imaging apparatus 1, for example, when an application for imaging is started by a user, a panoramic image imaging mode is set, and the shutter button 12 is pressed, the imaging unit 20 is cut out by the imaging frame 42. First image data 40 to be acquired is acquired (S1: imaging step). Subsequently, the movement information acquisition unit 22 acquires information indicating four directions corresponding to the four sides of the first image data 40 (S2: movement information acquisition step).

  Subsequently, the image feature point detection unit 24 detects the feature point A of the image in the region 44 where the first image data 40 and the virtual frame 41 overlap (S3: image feature point detection step). Details of this detection method will be described later.

  Subsequently, the movement distance determination unit 26 determines the distance Y to which the imaging frame 42 should move (S4: movement distance determination step). Details of this determination method will be described later.

  Subsequently, the guide unit 30 displays the first image data 40 and the imaging guide on the touch panel display 4 (S5: guide step). The imaging guide includes four frames 46a and four arrows 46b, and corresponds to the distance Y to which the imaging frame 42 determined by the movement distance determination unit 26 should move in the four directions acquired by the movement information acquisition unit 22, respectively. The position is displayed (see FIG. 5A).

  Subsequently, the movement information acquisition unit 22 detects the movement of the imaging frame 42 and determines whether the imaging frame has moved (S6). When the movement information acquisition unit 22 determines that the imaging frame 42 has not moved (S6: NO), the guide unit 30 displays an imaging guide including four frames 46a and four arrows 46b on the touch panel display 4. Continue (see FIG. 5A).

  On the other hand, if the movement information acquisition unit 22 determines that the imaging frame 42 has moved (S6: YES), the movement information acquisition unit 22 acquires movement direction information indicating the direction in which the imaging frame 42 has moved ( S7: Movement information acquisition step). Then, the guide unit 30 sets the imaging frame 42 determined by the movement distance determination unit 26 to the distance Y to be moved only in the direction in which the imaging frame 42 indicated by the movement direction information acquired by the movement information acquisition unit 22 has moved. A corresponding imaging guide is displayed on the touch panel display 4 (S8: guide step). That is, on the touch panel display 4, the frame 46a and the arrow 46b other than the direction in which the imaging frame 42 has moved are not displayed, and the frame 46a in the direction in which the imaging frame 42 has moved is the position where the imaging frame 42 should be moved. (See (b) of FIG. 5).

  Subsequently, the movement information acquisition unit 22 detects the distance X that the imaging frame 42 has moved (S9: movement information acquisition step). Then, the guide unit 30 changes the size of the arrow 46b displayed on the touch panel display 4 in proportion to an amount obtained by subtracting the distance X that the imaging frame 42 has moved from the distance Y that the imaging frame 42 should move (S10). : Guide step). That is, on the touch panel display 4, the arrow 46b is displayed smaller as the imaging frame 42 approaches the rectangular frame 46a (see FIGS. 5B and 5C). If the distance X that the imaging frame 42 has moved is greater than the distance Y that the imaging frame 42 should move, that is, if the imaging frame 42 has exceeded the position to which the imaging frame 42 should move, the imaging frame 42 has been moved. You may perform the guide to the effect of moving to the opposite side to a direction.

  Subsequently, the frame arrival determination unit 28 determines whether or not an amount obtained by subtracting the distance X that the imaging frame 42 has moved from the distance Y that the imaging frame 42 should move is equal to or less than a predetermined threshold value α (S11: frame arrival determination). Step). The predetermined threshold α is, for example, a value in units of pixels that is so small that the distance Y that the imaging frame 42 should move and the distance X that the imaging frame 42 moved can be regarded as substantially the same. When the amount obtained by subtracting the distance X traveled by the imaging frame 42 from the distance Y that the imaging frame 42 should move is not less than or equal to the predetermined threshold value α (S11: NO), the guide unit 30 moves the imaging frame 42. The size of the arrow 46b displayed on the touch panel display 4 is continuously changed in proportion to an amount obtained by subtracting the distance X that the imaging frame 42 has moved from the power distance Y (see FIGS. 5B and 5C).

  On the other hand, when the amount obtained by subtracting the distance X to which the imaging frame 42 has moved from the distance Y to which the imaging frame 42 should move is equal to or less than the predetermined threshold value α (S11: YES), the guide unit 30 The stationary mark 46c is displayed as an output indicating that 42 should be stationary (S12: guide step). That is, on the touch panel display 4, for example, a stationary mark 46c is displayed near the center of the area indicated by the imaging frame 42 (see FIG. 5D).

  The stationary mark 46c is displayed by the guide unit 30, and when the user stops the operation of the imaging apparatus 1 according to the display, the imaging frame 42 is stationary. When the movement information acquisition unit 22 detects that the imaging frame 42 is stationary, the imaging unit 20 acquires the second image data 50 (S13: imaging step). In this way, the operation for controlling the imaging guide from the acquisition of the first image data 40 to the acquisition of the second image data 50 is completed.

  Next, operations of image feature point detection (S5) and movement distance determination (S6) in the operation of controlling the imaging guide will be described in detail with reference to FIG. FIG. 8 is a flowchart showing in detail the operations of image feature point detection and movement distance determination in FIG.

  As shown in FIG. 8, first, when the image feature point detection operation starts, the image feature point detection unit 24 detects the feature point A of the image in the region 44 where the first image data 40 and the virtual frame 41 overlap. (S31). Subsequently, the image feature point detection unit 24 calculates the number of feature points A of the detected image (S32), and ends the image feature point detection operation.

  Subsequently, when the movement distance determination operation is started, the movement distance determination unit 26 determines whether or not the calculated number of feature points A of the image is equal to or greater than a predetermined threshold β (S41). If it is determined that the number of feature points A in the image is not equal to or greater than the predetermined threshold β (S41: NO), the distance Y that the imaging frame 42 should move is determined as a short distance (S43), and the movement distance is determined. End the operation. On the other hand, when it is determined that the number of feature points A of the image is equal to or greater than the predetermined threshold β (S41: YES), the distance Y to which the imaging frame 42 should move is determined as a long distance (S42) and moved. The distance determination operation is terminated.

  As described above, according to the imaging apparatus 1 of the embodiment, when the movement information acquisition unit 22 captures a plurality of pieces of image data, movement direction information indicating the direction in which the imaging frame 42 indicating the imaging region moves is acquired. An area in which the imaging frame 42 and the first image data 40 acquired by the imaging unit overlap according to the amount of movement of the imaging frame 42 in the direction indicated by the acquired movement direction information by the image feature point detection unit 24. A feature point A of the image at 44 is detected from the first image data 40. The distance Y by which the imaging frame 42 should move in order to obtain the second image data 50 adjacent to the first image data 40 according to the number A of feature points of the detected image by the movement distance determination unit 26. Is determined. Then, the guide unit 30 displays the imaging guide including the frame 46 a and the arrow 46 b on the touch panel display 4 according to the distance Y that the imaging frame 42 should move. Thereby, the user can perform an appropriate imaging operation based on the distance to which the imaging frame grasped by the display should move, and the number of feature points A of the image of the portion that becomes a joint between a plurality of image data It is possible to secure the necessary amount for generating panoramic image data. Therefore, it is possible to perform a wasteful imaging operation while securing the number of image feature points A necessary for joining a plurality of image data.

  Further, according to the imaging apparatus 1 of the embodiment, the movement information acquisition unit 22 detects the direction in which the imaging frame 42 has moved, and the movement direction information is acquired using the moved direction. Further, the guide unit 30 displays the frame 46 a and the arrow 46 b on the touch panel display 4 only in the direction in which the imaging frame 42 acquired by the movement information acquisition unit 22 has moved. In accordance with the display of the imaging guide corresponding to the direction in which the imaging frame 42 has moved, the user can perform an imaging operation more appropriately.

  Further, according to the imaging apparatus 1 of the embodiment, the movement information acquisition unit 22 detects the distance X that the imaging frame 42 has moved. Also, the guide unit 30 displays the size of the arrow 46b on the touch panel display 4 as the difference between the distance Y that the imaging frame 42 should move and the distance X that the imaging frame 42 moved decreases. . Therefore, the user can intuitively determine how much the imaging frame 42 should be moved according to the display size of the arrow 46b, and can perform a more appropriate imaging operation.

  Further, according to the imaging apparatus 1 of the embodiment, the frame arrival determination unit 28 moves the imaging frame 42 determined by the movement distance determination unit 26 and the imaging frame detected by the movement information acquisition unit 22. It is determined whether or not the difference (Y−X) from the distance X traveled by 42 has reached a predetermined threshold value α or less. In addition, when it is determined that the difference (Y−X) has reached a predetermined threshold value α or less, the stationary mark 46c for notifying that the movement of the imaging frame 42 should be stopped is displayed on the touch panel display by the guide unit 30. 4 is displayed. Therefore, the user can stop the imaging operation in accordance with the display of the stationary mark 46c, and the imaging frame 42 can be stopped at an appropriate timing. In addition, it is possible to perform an imaging operation without blurring. For example, when the user holds the imaging device 1 and performs imaging as in the present embodiment, camera shake can be prevented.

  The preferred embodiments of the present invention have been described above. However, the present invention is not necessarily limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  For example, the display mode of the imaging guide including the frame 46a, the arrow 46b, and the stationary mark 46c is not limited to that in the above embodiment, and the frame 46a, the arrow 46b, and the stationary mark 46c may have various shapes. FIG. 9 is a diagram showing another aspect of the display of the imaging guide of FIG. As shown in FIG. 9, for example, four bracketed frames 46a may be displayed by arranging them corresponding to the four corners of the first rectangular image data 40 (imaging frame 42). The arrow 46 b may be displayed on the touch panel display 4. The frame 46a may be a rectangle having the same shape as the imaging frame 42. When the frame 46a has such a rectangular shape, the user performs an operation so that the image data is not tilted with respect to the frame 46a while viewing continuous image data cut out by the imaging frame 42 on the touch panel display 4. Therefore, a more appropriate imaging operation can be performed.

  The display mode of the imaging guide is not limited to the display by the frame 46a, the arrow 46b, and the stationary mark 46c, and other figures, symbols, and the like may be displayed on the touch panel display 4. In addition, the display mode of the imaging guide is not limited to a still image, and the imaging guide may be performed in various modes that can be visually recognized on the screen display, such as animation display or blinking light on the touch panel display 4.

  The imaging guide is not limited to being displayed on the touch panel display 4, and may be performed by voice, vibration, or the like.

  In the above embodiment, the distance Y that the imaging frame 42 should move is set as a long distance as a default value. However, the present invention is not limited to this. For example, the distance Y to which the imaging frame 42 should move is set as a short distance as a default value, and imaging is performed according to the number of feature points A of the image in the region 44 where the first image data 40 and the virtual frame 41 overlap. The distance Y to which the frame 42 should move may be close to a long distance.

  In the above embodiment, the distance Y that the imaging frame 42 should move is determined to be either a long distance or a short distance, but is not limited thereto. For example, the distance Y to be moved by the imaging frame 42 is set so that the region 48 where the first image data 40 and the actual imaging frame 42 overlap has the number of image feature points A as many as necessary for panoramic image generation. It may be variable steplessly. For example, the area of the region 44 where the first image data 40 and the virtual frame 41 overlap is set to a minimum necessary size for panorama generation, and the number of feature points A of the image in the region 44 is detected. When the number of detected feature points A of the image in the region 44 is insufficient for panorama generation, the area of the region 44 is increased so as to be a sufficient number. Conversely, the area 44 where the first image data 40 and the virtual frame 41 overlap is maximized, that is, the virtual frame 41 is positioned so as to overlap the first image data 40, and from there The area of the region 44 may be reduced.

  The setting of each threshold value is not limited to the default setting stored in advance by the manufacturer or the like, and the setting may be appropriately changed by the user.

  Further, when the first image data 40 is acquired, the imaging guide is displayed in all directions as shown in FIG. 5, but the present invention is not limited to this, and the imaging frame 42 is previously prepared so that a panoramic image is appropriately generated. It is also possible to set a direction in which the camera should move and display the imaging guide only in that direction.

  Further, the imaging apparatus of the present invention is not limited to a portable terminal, and may be various imaging devices such as a digital camera. In the case of a digital camera or the like, since it is an imaging device, it is not necessary to start an application in order to perform imaging like a mobile terminal such as a mobile phone.

DESCRIPTION OF SYMBOLS 1 ... Imaging device, 20 ... Imaging part (imaging means), 22 ... Movement information acquisition part (movement information acquisition means), 24 ... Image feature point detection part (image feature point detection means), 26 ... Movement distance determination part (movement) (Distance determination means), 28 ... frame arrival determination section (frame arrival determination means), 30 ... guide section (guide means), 40 ... first image data, 42 ... imaging frame, 46a ... frame, 46b ... arrow, 50 ... Second image data, A: Image feature points.

Claims (8)

In order to generate panoramic image data by combining a plurality of image data, the imaging device acquires the plurality of image data,
Imaging means for acquiring the plurality of image data by capturing an image cut out by an imaging frame indicating an imaging region;
Movement information acquisition means for acquiring movement direction information indicating a direction in which the imaging frame moves;
The image in the region where the imaging frame and the first image data acquired by the imaging unit overlap according to the amount of movement of the imaging frame in the direction indicated by the movement direction information acquired by the movement information acquisition unit. Image feature point detection means for detecting a feature point from the first image data;
In accordance with the number of feature points of the image detected by the image feature point detection means, a distance that the imaging frame should move in order to acquire second image data adjacent to the first image data is determined. A moving distance determining means;
Guide means for performing output according to the distance to which the imaging frame determined by the movement distance determination means should move;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the movement information acquisition unit detects a direction in which the imaging frame has moved, and acquires the movement direction information using the moved direction.   The imaging apparatus according to claim 1, wherein the guide unit performs output according to the movement direction information acquired by the movement information acquisition unit. The movement information acquisition means detects the distance that the imaging frame has moved,
The imaging apparatus according to any one of claims 1 to 3, wherein the guide unit performs output in accordance with a distance traveled by the imaging frame detected by the movement information acquisition unit. It is determined whether or not the difference between the distance to which the imaging frame determined by the moving distance determination unit should move and the distance to which the imaging frame detected by the movement information acquisition unit has reached a predetermined threshold value. A frame arrival determination unit;
The imaging apparatus according to claim 4, wherein the guide unit performs an output indicating that the movement of the imaging frame should be stopped when the frame arrival determination unit determines that the difference has reached a predetermined threshold value.   The imaging apparatus according to claim 1, wherein the guide unit performs the output by screen display.   The imaging apparatus according to claim 6, wherein the guide unit displays a frame or an arrow as the screen display. In order to generate panoramic image data by combining a plurality of image data, an imaging method for acquiring the plurality of image data,
An imaging step of acquiring the plurality of image data by capturing an image cut out by an imaging frame indicating an imaging region;
A movement information acquisition step of acquiring movement direction information indicating a direction in which the imaging frame moves;
The image in the region where the imaging frame and the first image data acquired in the imaging step overlap according to the amount of movement of the imaging frame in the direction indicated by the movement direction information acquired in the movement information acquisition step. An image feature point detection step of detecting a feature point from the first image data;
In accordance with the number of feature points of the image detected in the image feature point detection step, a distance that the imaging frame should move to acquire second image data adjacent to the first image data is determined. A moving distance determining step;
A guide step for performing output in accordance with the distance to which the imaging frame determined in the moving distance determining step should move;
An imaging method including:

Images