JP2001119625A - Image processing method and apparatus - Google Patents

Abstract

(57) [Summary] [Problem] When creating a panoramic image by synthesizing images, since it is not known how much the image overlaps with another image, it is necessary to specify an overlapping area of a plurality of images. , The information on the overlapping position and the shape is stored, and a plurality of images are synthesized and displayed based on the information. SOLUTION: A new image picked up by an image pickup unit 100 is designated to overlap with another image to acquire a new image (ROI setting unit 102), and an overlap position between the other image and the new image is obtained. And information on the shape are stored in the image storage unit 103 together with the new image to be captured, and based on the information on the overlapping position and the shape of the other image and the new image, the other image and the new image are And display them.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for synthesizing a plurality of images and an apparatus therefor.

[0002]

2. Description of the Related Art Heretofore, it has been practiced to divide a landscape or the like with a digital camera and shoot the images, combine the images with a personal computer (hereinafter, PC), and create a panoramic image with a wide angle of view. I have. When synthesizing such a panoramic image, it is necessary for the images to have a certain degree of overlap. Therefore, a dedicated mode (hereinafter, panoramic mode) for easily capturing such an image is required. A digital camera having a digital camera has been proposed.

[0003]

Heretofore, when a panoramic image is created by synthesizing an image, it is not known how much the image overlaps with another image. Needed to search the entire image to find the overlap position. Because of that, it takes time to combine images and the search range is wide,
In some cases, an incorrect overlapping position was detected. Also, the area where the images overlap is compressed at the same compression rate as the other areas. Therefore, when the compression rate is high, noise is generated, and the accuracy of the synthesis processing is reduced.

The present invention has been made in view of the above-mentioned conventional example, and specifies an overlapping area of a plurality of images, stores information on the overlapping position and shape, and based on the information,
It is an object of the present invention to provide an image processing method and apparatus for combining and displaying a plurality of images.

It is another object of the present invention to provide an image processing method and apparatus for improving the creation of a composite image by storing the image at a reduced compression rate of an overlapping area between the images.

[0006]

In order to achieve the above object, an image processing apparatus according to the present invention has the following arrangement.
That is, an acquisition unit that designates an overlap with another image to acquire a new image, and a storage unit that stores information on an overlapping position and a shape of the another image and the new image together with the new image. And display control means for combining and displaying the other image and the new image based on information on the overlapping position and shape of the other image and the new image. I do.

[0007] To achieve the above object, the image processing method of the present invention comprises the following steps. That is, an acquisition step of acquiring a new image by specifying an overlap with another image,
A storage step of storing information on an overlapping position and a shape of the other image and the new image in a memory together with the new image, and information on an overlapping position and a shape of the other image and the new image. And a display control step of combining and displaying the other image and the new image.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a configuration example of an image processing apparatus (for example, a digital camera) according to Embodiment 1 of the present invention.

In FIG. 1, reference numeral 100 denotes an image pickup unit which includes a lens, a CCD and the like, and outputs an electric signal corresponding to a picked-up image. An image forming unit 101 forms an image based on an electric signal output from the imaging unit 100.
An ROI setting unit 102 sets an ROI (Region Of Interest) for an image created by the image forming unit 101. This ROI represents a certain area in the image, and in an image processing apparatus such as a digital camera according to the present embodiment, as described later, it represents an image area of a main subject or overlaps with an adjacent image. Used to represent an image area. 103, an image storage unit;
Is stored in a storage medium such as a memory card.

Next, ROI in ROI setting section 102
Will be described.

As described above, the ROI indicates a partial image area of image data of a captured image, and the ROI is described in information on the captured image, for example, as shown in FIG. You.

In FIG. 2, reference numeral 201 denotes image data stored in the digital camera or the like according to the present embodiment. Reference numeral 202 denotes a header, which stores supplementary information of the image. In the header 202, an ROI 203 is stored. There are several types of ROI 203,
For example, reference numeral 204 denotes ROI information indicating the type of the main subject. The ROI information 204 representing the main subject is set by the user when photographing with a digital camera or the like. For example, in the case of an image of a person, the user sets the person as the ROI representing the type of the main subject. can do. Specifically, in the present embodiment, when the user indicates a range where the person is captured using an operation unit or the like of a digital camera (not shown), the ROI setting unit 102
The area is set in the image.

Reference numerals 205 and 208 denote information indicating the shape of the image portion of the ROI. For example, a rectangle, a circle, a closed curve, or the like can be designated. When a person is specified as in the above-described example, the ROI setting unit 102 describes the specified image region with a closed curve. 206 and 209 are information indicating the range of the ROI. In the case of 206, since the shape of the ROI is a “closed curve”, a point at coordinates (100, 100) and a closed curve surrounded by points such as coordinates (101, 102) and coordinates (102, 104) are set as the ROI range. When the main subject has a complicated shape as described above, the outer periphery of the shape may be represented by listing coordinates. Another example is
For example, if the main subject is “circular”, the center and radius of the circle may be described. If the shape of the main subject is “rectangular”, the upper left coordinate and lower right You only have to describe the coordinates.

Reference numeral 207 denotes ROI information indicating an overlap with the image located on the right side of the image. This RO
The I information 207 is the aforementioned ROI information 20 representing the main subject.
4 are different from each other, and different names (“overlap with the right image” in the figure) are given as “type” of the ROI information 207 in FIG. 2 so that they can be distinguished from each other. The shape of the image of the ROI information 207 is the ROI information 20
As shown by 8, it is a "rectangle", and indicates that a rectangular portion surrounded by the coordinates (560,0) at the upper left and the coordinates (800,600) at the lower right of this image overlaps the image on the right. Similarly, ROI information 210 indicates an overlap with an image located below (“overlap with lower image”), and ROI information 211 indicates an overlap with an image located on the left side (“overlap with left image”). Overlap "), 212
Indicates an overlap with the image located above (“overlap with the upper image”). However, if there is no image corresponding to each position, there is no ROI information corresponding to that image. These ROI information
As will be described later, the image area is stored as an image captured in the panorama mode, and represents an area of an overlapped image on a liquid crystal screen of a digital camera or the like.

The image position 213 indicates the position of this image (main subject).
This is data representing the positional relationship between the image and another image by coordinates. Based on the coordinate values, it is possible to recognize an image existing above, below, right and left of a certain image. For example, when the images A to I are arranged as shown in FIG. 3, the x coordinate of the image E is set to “0”,
Assuming that the y coordinate is “0”, the x coordinate of the image F is “1”, and y
The coordinates are "0". In the example of the image position 213, X
Since the coordinate is “1” and the y coordinate is “1”, FIG.
It can be seen that the image C corresponds to this. Also,
In order to search for an image on the right side of the image C, an image whose x coordinate is "2" and whose y coordinate is "1" may be searched for. The image main body 216 is an area in which data such as the color of each pixel of the image is stored, and is an image data area used for displaying a corresponding image. However, here, it is not relevant to the gist of the present invention. So don't touch it.

FIG. 4 is a block diagram showing the configuration of the digital camera according to the present embodiment.

Referring to FIG. 1, reference numeral 301 denotes a CPU, which operates in accordance with a control program stored in a program memory 302.
The operation of the entire camera is controlled. Reference numeral 303 denotes an imaging unit corresponding to the imaging unit 100 in FIG. 1 described above, which includes a lens unit and a photoelectric conversion unit such as a CCD, and outputs a video signal corresponding to a captured image. 304 is RAM
The CPU 301 is used as a work area when the operation of the CPU 301 is executed, and temporarily stores various data. 305 is a storage medium such as a memory card. 30
Reference numeral 6 denotes an interface unit which controls an interface for exchanging data with a personal computer (PC) 307, and is capable of bidirectional communication. An operation unit 308 includes various operation switches such as a shutter switch and an aperture. Reference numeral 309 denotes a display unit which is formed of, for example, liquid crystal or the like, functions as a view window, and can also display an image or the like captured and stored in the storage medium 305. Reference numeral 310 denotes a bus for connecting these components.

Next, the function of the panorama mode of the digital camera according to the present embodiment will be described.

First, the contents displayed on the screen of the display unit 309 of the digital camera when the panorama mode is set will be described.

FIG. 5 shows a display example on the display unit 309 of the digital camera when the panorama mode is set.

In FIG. 5, on the screen of the display unit 309, reference numeral 1001 denotes an image taken immediately before.
When the first image is photographed, the immediately preceding image 1001 is not displayed because there is no image photographed immediately before. 10
02 is EVF (Electric View Finder)
The image you are about to shoot is displayed live. 1003 is the previous image 100
1 and the image 1002 to be photographed are overlapping portions, and the two images are transparently displayed. The amount of overlap in this part 1003 is
For example, it is fixed at about 30% of the entire original image. Therefore, the user moves the direction of the camera so that the two images are properly overlapped with each other in the overlapping portion 1003, and captures the next image. This allows the user to create an approximately 30% overlapped image with the previous image,
You can easily shoot.

In FIG. 5 of this embodiment, a case where the next image is photographed by superimposing the next image on the right of the immediately preceding image has been described, but this is merely an example.
The user can use the operation unit of the camera or the like to move the area of the image 1002 to be photographed in any of the vertical and horizontal directions of the immediately preceding image 1001. As a result, it is possible to shoot a superimposed image in any of the up, down, left, and right directions of the image 1001 taken immediately before.

Next, the photographing operation of the digital camera according to the embodiment of the present invention will be described with reference to the flowchart of FIG. A control program for executing the processing shown in this flowchart is stored in the program memory 302.

First, in step S 1, when the release switch of the operation unit 308 of the digital camera according to the embodiment is pressed by the user, the image forming unit 101 (here realized by a program) reads out from the imaging unit 100. The converted electric signal is converted into image information to form an image. Next, the process proceeds to step S2, where it is determined whether or not the image was captured in the panorama mode (designated by the switch of the operation unit 308).
02 (implemented by a program) checks whether there is an already photographed image in the vertical and horizontal directions of the photographed image. This can be identified by examining the image position indicated by image position 213 in FIG. If there is no image at any position, the image is taken first, and the process proceeds to step S5, where the ROI setting unit 102 sets the x coordinate “0” to indicate the reference position, The y coordinate “0” is written in the image at the image position 213. Thereafter, the process proceeds to step S7, where the image storage unit 103 stores the captured image in a JPEG format.
And store it in a storage medium such as a memory card.

On the other hand, in step S3, if there is an image already taken at any of the upper, lower, left and right positions of the newly taken image, the process proceeds to step S4, where the ROI setting unit 10
2 is the RO of FIG. 2 in the header 202 of the photographed image.
ROI information such as I203 is written in association with the image.

For example, if there is already an image on the left side, ROI information indicating "overlap with the left side image" is written in the image as shown at 211 in FIG. At that time,
For example, in the example of FIG. 5, the rectangle of the overlapping portion 1003, which is a 30% area of the image, is set as the ROI range. In addition, for the image on the left side of the photographed image, FIG.
Then, an ROI indicating “overlap with the right image” indicated by reference numeral 207 is written in the image. Thereafter, the process proceeds to step S5, where the ROI setting unit 102 writes the image position 213 in FIG. 2 into the image. The position of this image is determined by the user while looking at the screen of the display unit 309 on which these images are displayed. Therefore, the ROI setting unit 102 acquires the position and writes it in the image. Finally, the process proceeds to step S7, where the captured image is compressed by the image storage unit 103 by a method such as JPEG and the storage medium 305 such as a memory card is compressed.
To memorize. At that time, the image storage unit 103 compresses the ROI of the overlap area designated in step S4 at a lower compression ratio than the other areas so that the image quality does not deteriorate. This is because when combining images, an overlapping portion is searched for a corresponding portion of the image, and if the image quality of the portion is deteriorated, the accuracy of the search is deteriorated.
Therefore, the image is stored in such a manner that the compression ratio of the image is reduced so that noise is not mixed.

If it is determined in step S2 that the mode is not the panorama mode, the process proceeds to step S6, where the ROI setting unit 102
The area of the main subject is included in the header 202 of the image and indicated by reference numeral 204 in FIG.
Is written. The area of the main subject is specified by the user using the operation unit 303 of the digital camera as described above, or the ROI setting unit 10
2, the main subject may be automatically determined based on the edge or the histogram of the image. Then, the process proceeds to step S7, where the image storage unit 103 compresses the captured image by a method such as JPEG and stores the compressed image in a storage medium 305 such as a memory card. At that time, the image storage unit 1
In step 03, the ROI information specified in step S6 is compressed at a lower compression ratio than the other areas so that the image quality of the main subject does not deteriorate. This is because it is desirable to store the main subject with higher image quality than other areas.

If the PC 307 is connected via the interface unit 306, these various operations may be instructed by a command from the PC 307 instead of the instruction from the operation unit 308.

Next, an operation of synthesizing an image photographed in the panorama mode by processing by software of the PC 307 connected to the digital camera will be described.

FIG. 7 is a flowchart for explaining the operation of software of the PC 307 for executing the panorama synthesizing process in the digital camera according to the present embodiment.

First, in step S11, an ROI indicating an overlap with an adjacent image and an image position are read from the headers of all images to be synthesized. This is an ROI that indicates an overlap between adjacent images in FIG.
Are ROI information 207, 210, 211, 2 shown in FIG.
12, and the image position is 213. These ROs
By examining I and its image position, the arrangement of all images and their overlap can be known. At this time, since the ROI information indicating the main subject described in 204 in FIG. 2 is described as a different type from the ROI information indicating the overlap, the ROI information is unnecessary information for synthesizing the panoramic image. , You can skip over.

Next, the process proceeds to step S12, in which the range of the overlap area described in the header 202 of the image is searched, and how the images overlap each other is calculated accurately. The overlap described in this image is the one specified at the time of shooting and is not accurate. Therefore, it is necessary to accurately check how the images actually overlap.

Then, the process proceeds to a step S13, and a step S1
The images are synthesized based on the overlapping position calculated in step 2 to generate one image. Here, since the information indicating the overlap area is described in the image, the search range is limited, so that the image can be synthesized more efficiently and more accurately than when searching the entire image.

Even if the present invention is applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, and the like), a single device (for example, a copying machine, a facsimile machine, etc.) ) May be applied.

An object of the present invention is to supply a storage medium (or a recording medium) storing program codes of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. Alternatively, the present invention is also achieved when a CPU or an MPU reads and executes a program code stored in a storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. By executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. This also includes a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the program code is read based on the instruction of the program code. This also includes the case where the CPU provided in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

As described above, according to the present embodiment, it is possible to specify an overlapping area of an image at the time of photographing, and to further store the overlapping area in the image. Accordingly, when creating a panoramic image by synthesizing images, the processing time is shorter than searching the entire image by searching for an overlapping area between them, and the image can be synthesized with high accuracy.

Further, since the image quality of the overlapping area is compressed so as to be higher than that of the other image areas, the influence of the compression noise on the image synthesizing process is reduced, and the accuracy of the image synthesizing can be further improved. .

Further, since an area where images overlap each other is assigned a different identifier from other areas indicating the main subject or the like, when composing images to create a panoramic image, only the area where these images overlap is extracted. Can be processed.

[0041]

As described above, according to the present invention, an overlapping area of a plurality of images is designated, information on the overlapping position and shape is stored, and a plurality of images are synthesized based on the information. Can be displayed.

Further, according to the present invention, there is an effect that the creation of a composite image can be further improved by storing the image at a reduced compression rate of the overlapping area between the images.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of an image processing device (digital camera) according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating information when an image is stored in the present embodiment.

FIG. 3 is a diagram illustrating an example of image arrangement.

FIG. 4 is a block diagram illustrating a hardware configuration example of the digital camera according to the embodiment of the present invention.

FIG. 5 is a diagram illustrating a display example of a liquid crystal screen when shooting is performed in a panoramic mode in the digital camera according to the present embodiment.

FIG. 6 is a flowchart illustrating a manner in which an image is photographed in the image processing apparatus (digital camera) according to the embodiment of the present invention.

FIG. 7 is a flowchart illustrating an operation when combining images in the image processing device (digital camera) according to the embodiment of the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 imaging unit 101 image forming unit 102 ROI setting unit 103 image storage unit

Claims (10)

[Claims] An acquisition unit that acquires a new image by designating an overlap with another image; and stores information on an overlap position and a shape of the another image with the new image together with the new image. Storage means for performing, and display control means for synthesizing and displaying the other image and the new image based on information on an overlapping position and a shape of the other image and the new image. An image processing apparatus characterized by the above-mentioned. 2. A display means for displaying the other image and the new image side by side; a changing means for changing a relative display position of the new image displayed on the display means with respect to the other image; 2. The method according to claim 1, further comprising: determining means for determining information on an overlapping position with the new image based on the relative display position of the new image changed by the changing means. 3. Image processing device. 3. The image according to claim 1, further comprising a compression unit configured to compress an overlapping area between the other image and the new image at a lower compression ratio than the other area. Processing equipment. 4. The image processing apparatus according to claim 1, wherein the storage unit further stores information on the other image and the shape of the new image. 5. The image processing apparatus according to claim 1, wherein the image processing apparatus is a digital camera. 6. An acquiring step of acquiring a new image by designating an overlap with another image, and storing information on an overlapping position and a shape of the other image and the new image together with the new image. And a display control step of combining and displaying the other image and the new image based on information on an overlapping position and a shape of the other image and the new image. An image processing method comprising: 7. A display step of displaying the other image and the new image side by side; a changing step of changing a relative display position of the new image displayed in the display step with respect to the other image; 7. The determining step according to claim 6, further comprising: determining information on an overlapping position with the new image based on the relative display position of the new image changed in the changing step. Image processing method. 8. The image according to claim 6, further comprising a compression step of compressing an overlapping area between the other image and the new image at a lower compression ratio than the other area. Processing method. 9. The image processing method according to claim 6, wherein in the storing step, information on the shape of the another image and the shape of the new image is further stored. 10. A computer-readable storage medium storing a program for executing the image processing method according to claim 5. Description: