KR20210086129A - Method and apparatus for registration of object information - Google Patents

Abstract

Provided are a method and a device for registering object information. The method for registering an image of an object includes the steps of: determining a guide for photographing an object; photographing an image of the object and storing the photographed image, when the object is located in the guide as an object photographing device moves according to the guide; extracting features of the object from the stored image; and determining the similarity between the features by comparing the feature with the previously stored feature of the object, and registering the feature as object information of the object based on the similarity between the features.

Description

Object information registration method and device {METHOD AND APPARATUS FOR REGISTRATION OF OBJECT INFORMATION}

The present description relates to an apparatus and method for registering object information.

Conventionally, when detecting and recognizing an object using a camera, many images related to a recognition target need to be machine-learned in advance. In order for an object not previously learned to be recognized, an image of the object may be registered immediately, and a large amount of images need to be registered for stable and high recognition performance. Also, even if a large amount of images are registered, if the registered images do not contain effective data for machine learning for recognition, additional time and effort are required to extract effective data from the images.

One embodiment provides an apparatus for acquiring and registering object information from an image captured based on a guide.

Another embodiment provides a method of obtaining and registering information of an object from an image taken based on a guide.

According to one embodiment, an object information registration apparatus for registering object information is provided. The object information registration device includes a guide determining unit that determines a guide for photographing an object, and when the object is located in the guide by moving the object photographing device according to the guide, photographing an image of the object and storing the photographed image unit, a feature extractor that extracts features of an object from a stored image, and a feature register that compares the features with pre-stored features of the object to determine the degree of similarity between features, and registers the feature as object information of the object based on the degree of similarity between the features includes

In the object information registration apparatus, the guide determiner may determine a guide based on object information of a previous guide used when photographing an object.

When determining a guide based on object information of a previous guide used when photographing an object, the guide determining unit in the object information registration device determines a histogram of the information element included in the previous guide information among the object information, and By normalizing the inverse histogram of the histogram, the probability distribution of the section of the histogram is calculated, and a guide can be determined based on the probability distribution.

In the object information registration apparatus, the image storage unit determines whether the object photographing apparatus is stopped based on a comparison between the image of the current frame and the image of the previous frame, and when it is determined that the object photographing apparatus is stopped, photographing an image and storing the photographed image can be saved

In the object information registration apparatus, the image storage unit increments the count if the difference value between the image of the current frame and the image of the previous frame is less than the threshold difference when determining whether the object photographing apparatus is stopped, and the increased count is the threshold If it becomes larger than the count, an image can be taken.

In the object information registration apparatus, the feature extraction unit may extract color information and edge information of an image as features.

In the object information registration apparatus, the feature registration unit may store the feature together with the image and guide when the similarity between features is lower than a predetermined reference value, and discard the feature when the similarity between the features is higher than the predetermined reference value.

According to another embodiment, an object information registration method for registering object information is provided. The object information registration method includes the steps of: determining a guide for photographing an object; when an object is located in the guide by moving the object photographing device according to the guide, photographing an image of the object and storing the photographed image; extracting the feature of the object from the , and determining the similarity between the features by comparing the feature with the previously stored features of the object, and registering the feature as object information of the object based on the similarity between the features.

In the object information registration method, the determining of a guide for photographing an object may include determining a guide based on object information of a previous guide used when photographing an object.

In the object information registration method, the step of determining a guide based on object information of a previous guide used when photographing an object includes determining a histogram of an information element included in the previous guide information among object information, The method may include calculating a probability distribution of a section of the histogram by normalizing the inverse histogram of the histogram, and determining a guide based on the probability distribution.

In the object information registration method, when the object is located in the guide by moving the object photographing device according to the guide, the step of photographing an image of the object and storing the photographed image is based on a comparison between the image of the current frame and the image of the previous frame. It may include the steps of determining whether the object photographing apparatus is stopped, and if it is determined that the object photographing apparatus is stopped, photographing an image and storing the photographed image.

In the object information registration method, the step of determining whether the object photographing apparatus is stopped based on the comparison between the image of the current frame and the image of the previous frame may include, if the difference value between the image of the current frame and the image of the previous frame is less than the threshold difference , incrementing the count, and taking an image when the incremented count is greater than the threshold count.

The step of extracting the feature of the object from the stored image in the object information registration method may include extracting color information and edge information of the image as features.

In the object information registration method, determining the similarity between the features by comparing the feature with the pre-stored features of the object, and registering the feature as object information of the object based on the similarity between the features includes the similarity between features being lower than a predetermined reference value. storing the feature together with the image and the guide, and discarding the feature when the similarity between the features is higher than a predetermined reference value.

According to another embodiment, a method of providing a guide for taking an image of an object is provided. The guide providing method includes determining a histogram regarding an information element included in previous guide information of a previous guide used when photographing an object, and calculating a probability distribution of a section of the histogram by normalizing an inverse histogram of the histogram regarding the information element and determining a guide based on the probability distribution.

In the guide providing method, the guide and the previous guide include a plurality of information elements, and the step of determining a histogram for the information element included in the previous guide information of the previous guide used when photographing the object includes: individually determining the histograms for the plurality of information elements.

In the guide providing method, the step of determining the guide based on the probability distribution includes determining the information element of the guide based on a probability density function (PDF)-conversion histogram of a normalized inverse histogram. may include

In the guide providing method, the step of determining the guide based on the probability distribution includes determining the information element of the guide based on a cumulative distribution function (CDF)-transformed histogram of a normalized inverse histogram. may include

Through a guide that can effectively extract the characteristics of an object, an image of the object is taken, and object information can be quickly and appropriately registered therefrom.

1 is a block diagram illustrating an object information registration apparatus for registering object information of an object according to an exemplary embodiment.
2A and 2B are conceptual views illustrating a method of determining guide information according to an exemplary embodiment.
3 is a conceptual diagram illustrating a process in which an object is guided into a guide according to an exemplary embodiment.
4 is a flowchart illustrating a method of photographing and storing an image of an object according to an exemplary embodiment.
5 is a block diagram illustrating an apparatus for registering object information according to another embodiment.

Hereinafter, with reference to the accompanying drawings, it will be described in detail for those of ordinary skill in the art to which the present invention pertains to easily implement the embodiments of the present disclosure. However, the present description may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present description in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a block diagram illustrating an object information registration apparatus for registering object information of an object according to an embodiment, FIGS. 2A and 2B are conceptual diagrams illustrating a method of determining guide information according to an embodiment, and FIG. 3 is It is a conceptual diagram illustrating a process in which an object is guided into a guide according to an embodiment.

The object information registration apparatus 100 according to an embodiment includes a guide determiner 110 , an image storage unit 120 , a feature extraction unit 130 , and a feature registration unit 140 . The object information registration apparatus 100 is connected to the characteristic database 200 in which characteristic information of the object and guide information of the object 11 are stored. The guide information stored in the feature database 200 is guide information of a previous guide used when photographing an object or acquiring an image of the object.

The guide determiner 110 may determine the guide 12 for photographing the object 11 . The object photographing apparatus may photograph the object 11 by automatically moving according to the guide 12 displayed on the display unit (viewfinder or monitor, etc.) 10 of the object photographing apparatus. Alternatively, a person photographing the object 11 may photograph the object 11 by moving a camera or the like according to the guide 12 . The guide determining unit 110 according to an embodiment may determine a polygonal guide or a circular guide, and a rectangular guide 12 is illustrated in FIG. 3 . The guide information about the guide 12 may include information elements such as size (w, h), position (x, y), and direction (a) of the guide 12 . For example, the guide information may be a numeric string (w, h, x, y, a) indicating the size, position, and direction of the guide 12 . The guide 12 according to an embodiment is not registered in the feature database 200, but for obtaining an image of the object 11 necessary for extracting the features of the object 11 necessary for machine learning for object identification It may be a shooting guide 12 . The guide determiner 110 according to an embodiment may display the guide 12 on the display unit 10 of the object photographing apparatus.

The guide determiner 110 according to an embodiment may determine one information element from among guide information stored in the feature database 200 , and calculate a histogram regarding the determined information element. Table 1 below shows guide information of object A stored in the feature database 200 .

In Table 1 above, feature information 1 may be, for example, a vector such as [0.007, 1.273, 2.312, -0.123, ??]. The guide information a, the guide information b, and the guide information e may be information about a guide used for capturing an image from which the characteristic information 1 is obtained. For example, the guide information c is information about a guide used for shooting an image in which the feature information 2 is acquired, and the guide information d and the guide information f are guides used for shooting an image in which the feature information 3 is acquired, respectively. is information about

Referring to FIG. 2A , the guide determiner 110 may determine a histogram from direction information in each guide information to determine the direction of the guide 12 . In FIG. 2A , direction information has a value between 0 and 1, and the number n of each bin of the histogram is 10. The direction information included in the guide information a is 0.09, the direction information included in the guide information b is 0.08, the direction information included in the guide information c is 0.35, the direction information included in the guide information e is 0.03, and the guide information The direction information included in f is 0.72. Therefore, the bin length of the histogram for n=10 is h _k is 3, 1, 1. h _k may be determined as in Equation 1 below.

Thereafter, the guide determiner 110 determines an inverse histogram of the histogram. The inverse histogram may be determined as in Equation 2 below.

Thereafter, the guide determiner 110 may calculate a probability distribution for each section by normalizing the inverse histogram, and determine the direction of the guide 12 through random extraction based on the probability distribution. According to one embodiment, the guide determiner 110 selects one section according to the probability of each section in the probability density function (PDF)-conversion histogram, and randomly selects one value within the selected section. It can be determined as the information element of the guide. Here, the probability of each section may be a probability of being selected by the guide determiner 110 . For example, the probability that the interval 0 to 0.1 is selected is 0%, the probability that the interval 0.1 to 0.2 is selected is 12%, and the probability that the interval 0.3 to 0.4 is selected is 8%. Referring to FIG. 2A , for example, when a section 0.4 to 0.5 is selected, the guide determiner 110 may randomly determine one value within 0.4 to 0.5 as an information element. Alternatively, when the section 0.4 to 0.5 is selected, the guide determiner 110 may determine a representative value of 0.45 of the section 0.4 to 0.5 as the information element.

According to another embodiment, the guide determiner 110 may randomly determine one value as the information element of the guide in the cumulative distribution function (CDF)-transformed histogram of the normalized inverse histogram. Referring to FIG. 2B , for example, the guide determiner 110 may determine 0.65 as an information element.

Normalization may be performed according to Equation 3 below.

The guide determining unit 110 may determine other information elements necessary for determining the guide 12 as in the method of FIGS. 2A and 2B . The guide determined by the guide determiner 110 may be output in the display unit 10 of the object photographing apparatus. The information element of the guide determined by the guide determiner 110 may be output in the display unit 10 after being converted according to the unit of each information element. For example, the size information element and the position information element may be converted in units of pixels and the direction information element may be converted in units of degrees and then output in the display unit 10 . A unit conversion method of each information element may be according to Equation 4 below.

The image storage unit 120 may store an image of the object 11 located in the guide 12 when the object is located in the guide as the object photographing apparatus moves according to the guide. For example, when a person photographs the object 11 with a camera, the image storage unit 120 determines whether an image of the still object 11 can be obtained by comparing the image of the current frame and the image of the previous frame. can determine whether Referring to FIG. 3 , the image storage unit 120 may store the image when it is determined that the object photographing apparatus has been properly moved so that the object 11 is located in the guide 12 . An image capturing and storing method of the image storage unit 120 will be described in detail below with reference to FIG. 4 .

The feature extraction unit 130 may extract features of the object 11 from the image of the object 11 in the guide 12 . The feature extraction unit 130 may remove a portion other than the object 11 in the guide 12 using a background removal algorithm, and extract the features of the object 11 from the image of the object 11 . The feature extractor 130 may extract color information, edge information, and the like of the image as features of the object 11 . The feature extraction unit 130 uses an algorithm such as Histograms of Oriented Gradients (HOG) or Scale-Invariant Feature Transform (SIFT), or a deep learning-based feature extraction method (eg, , a feature extraction method using a specific layer of a pre-trained artificial neural network) may be used to obtain the features of the object 11 from the image of the object 11 .

The feature registration unit 140 compares the acquired features with features previously stored in the feature database 200 to determine the degree of similarity between the features, and based on the similarity between the features, the characteristics of the object, a guide used for photographing the object, and Accordingly, the image of the object can be registered as object information. If feature information about the photographed object is not stored in the feature database 200 , the feature registration unit 140 may determine the similarity between features to be 0 .

The feature registration unit 140 according to an embodiment compares the degree of similarity between features with a reference value, and when the degree of similarity between features is lower than a predetermined reference value, the acquired feature of the object, along with an image of the object and a guide used for photographing the image, is included in the feature database. It can be stored in (200). This is because a feature with a low similarity may be determined as a new feature that is not stored in the feature database 200 . The feature registration unit 140 according to an embodiment discards the acquired feature of the object when the similarity between features is higher than a predetermined reference value. This is because a feature with a high degree of similarity may be determined to overlap with a feature already stored in the feature database. In this case, with respect to a feature in which the similarity between features is higher than the reference value, guide information of a guide used for image capturing may be stored in the feature database 200 .

The similarity between features may be based on Equation 5 below, and as the individual comparison function, a similarity determining function between vectors using a cosine function, a Euclidean distance function, or the like may be used.

4 is a flowchart illustrating a method of photographing and storing an image of an object according to an exemplary embodiment.

Referring to FIG. 4 , the image storage unit 120 of the object information registration apparatus 100 compares the image of the current frame of the object with the image of the previous frame ( S110 ). And if the difference value between the image of the current frame and the image of the previous frame is greater than the threshold difference (S120), the image storage unit 120 maintains the count as 0 (S130). That is, if the difference between the image of the current frame and the image of the previous frame is large, it is determined that the object photographing apparatus has not yet stopped, and the image of the current frame is not stored. However, if the difference value between the image of the current frame and the image of the previous frame is less than the threshold difference, the image storage unit 120 increments the count by 1 ( S140 ). Here, the difference value between the image of the current frame and the image of the previous frame may be a sum of absolute values of differences between RGB values for each pixel or a value obtained by dividing the sum of absolute values by the total number of pixels in the guide. Thereafter, when the count increased through step S140 becomes greater than the threshold count (S150), the image storage unit 120 may store the image (S160).

As described above, according to the embodiment, an image of an object may be captured through a guide capable of effectively extracting features of the object, and features may be registered therefrom.

5 is a block diagram illustrating an apparatus for registering object information according to another embodiment.

The object information registration apparatus according to an embodiment may be implemented in a computer system, for example, a computer-readable medium. Referring to FIG. 5 , a computer system 500 includes a processor 510 , a memory 530 , an input interface device 550 , an output interface device 560 , and a storage device 540 that communicate via a bus 570 . ) may include at least one of. Computer system 500 may also include a communication device 520 coupled to a network. The processor 510 may be a central processing unit (CPU) or a semiconductor device that executes instructions stored in the memory 530 or the storage device 540 . The memory 530 and the storage device 540 may include various types of volatile or nonvolatile storage media. For example, the memory may include read only memory (ROM) and random access memory (RAM). In the embodiment of the present disclosure, the memory may be located inside or outside the processor, and the memory may be connected to the processor through various known means. The memory is a volatile or non-volatile storage medium of various types. For example, the memory may include a read-only memory (ROM) or a random access memory (RAM).

Accordingly, an embodiment of the present invention may be implemented as a computer-implemented method, or as a non-transitory computer-readable medium having computer-executable instructions stored thereon. In one embodiment, when executed by a processor, the computer readable instructions may perform a method according to at least one aspect of the present disclosure.

The communication device 520 may transmit or receive a wired signal or a wireless signal.

On the other hand, the embodiment of the present invention is not implemented only through the apparatus and/or method described so far, and a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded may be implemented. And, such an implementation can be easily implemented by those skilled in the art from the description of the above-described embodiments. Specifically, the method (eg, network management method, data transmission method, transmission schedule generation method, etc.) according to an embodiment of the present invention is implemented in the form of a program command that can be executed through various computer means, and is stored in a computer-readable medium. can be recorded. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable medium may be specially designed and configured for the embodiment of the present invention, or may be known and used by those skilled in the art of computer software. The computer-readable recording medium may include a hardware device configured to store and execute program instructions. For example, the computer-readable recording medium includes magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and floppy disks. Such as magneto-optical media, ROM, RAM, flash memory, or the like. The program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer through an interpreter or the like.

Although the embodiments have been described in detail above, the scope of the rights is not limited thereto, and various modifications and improved forms of those skilled in the art using the basic concepts defined in the following claims also belong to the scope of the rights.

Claims (18)

An object information registration device for registering object information, comprising:
A guide determining unit that determines a guide for shooting an object,
When an object is located in the guide by moving the object photographing device according to the guide, an image storage unit for photographing an image of the object and storing the photographed image;
A feature extraction unit for extracting features of the object from the stored image, and
A feature registration unit that compares the feature with a feature stored in advance of the object to determine a degree of similarity between features, and registers the feature as object information of the object based on the degree of similarity between the features
Object information registration device comprising a. In claim 1,
The guide determining unit,
An object information registration apparatus for determining the guide based on object information of a previous guide used when photographing the object. In claim 2,
When the guide determining unit determines the guide based on object information of a previous guide used when photographing the object,
Determine a histogram of an information element included in previous guide information among the object information, and calculate a probability distribution of a section of the histogram by normalizing an inverse histogram of the histogram on the information element, and the guide based on the probability distribution To determine, object information registration device. In claim 1,
The image storage unit,
Object information registration apparatus for determining whether the object photographing apparatus is stopped based on a comparison between the image of the current frame and the image of the previous frame, and photographing the image and storing the photographed image when it is determined that the object photographing apparatus is stopped . In claim 4,
The image storage unit, when determining whether the object photographing apparatus is stopped,
If the difference value between the image of the current frame and the image of the previous frame is less than a threshold difference, the count is incremented, and when the increased count is greater than the threshold count, the image is photographed. In claim 1,
The feature extraction unit,
and extracting color information and edge information of the image as the features. In claim 1,
The feature registration unit,
If the similarity between the features is lower than a predetermined reference value, the feature is stored together with the image and the guide, and if the similarity between the features is higher than the predetermined reference value, the feature is discarded. An object information registration method for registering object information, comprising:
determining a guide for photographing the object;
When the object is located in the guide by moving the object photographing device according to the guide, photographing an image of the object and storing the photographed image;
extracting features of the object from the stored image; and
determining a similarity between features by comparing the feature with a previously stored feature of the object, and registering the feature as object information of the object based on the similarity between the features
A method of registering object information, including. In claim 8,
The step of determining a guide for photographing an object is,
determining the guide based on object information of a previous guide used when photographing the object
Including, object information registration method. In claim 9,
The step of determining the guide based on the object information of the previous guide used when photographing the object,
determining a histogram of information elements included in previous guide information among the object information;
calculating a probability distribution of an interval of the histogram by normalizing the inverse histogram of the histogram for the information element, and
determining the guide based on the probability distribution
Including, object information registration method. In claim 8,
When an object is located in the guide by moving the object photographing device according to the guide, photographing an image of the object and storing the photographed image includes:
determining whether the object photographing apparatus is stopped based on a comparison between the image of the current frame and the image of the previous frame; and
When it is determined that the object photographing apparatus is stopped, photographing the image and storing the photographed image
Including, object information registration method. In claim 11,
The step of determining whether the object photographing apparatus is stopped based on a comparison between the image of the current frame and the image of the previous frame comprises:
If the difference value between the image of the current frame and the image of the previous frame is less than a threshold difference, increasing the count, and capturing the image when the increased count is greater than the threshold count
Including, object information registration method. In claim 8,
The step of extracting the feature of the object from the stored image,
extracting color information and edge information of the image as the feature
Including, object information registration method. In claim 8,
determining the similarity between the features by comparing the feature with the pre-stored feature of the object, and registering the feature as object information of the object based on the similarity between the features,
storing the feature together with the image and the guide if the similarity between the features is lower than a predetermined reference value, and discarding the feature if the similarity between the features is higher than the predetermined reference value
Including, object information registration method. A method of providing a guide for taking an image of an object, comprising:
determining a histogram of information elements included in previous guide information of a previous guide used when photographing the object;
calculating a probability distribution of an interval of the histogram by normalizing the inverse histogram of the histogram for the information element;
determining the guide based on the probability distribution
How to provide a guide that includes. In claim 15,
wherein the guide and the previous guide comprise a plurality of information elements;
The step of determining the histogram of the information element included in the previous guide information of the previous guide used when photographing the object,
individually determining a histogram for a plurality of information elements included in the previous guide information
A method of providing a guide, including. In claim 15,
The step of determining the guide based on the probability distribution comprises:
Determining information elements of the guide based on a probability density function (PDF)-transformed histogram of a normalized inverse histogram;
A method of providing a guide, including. In claim 15,
The step of determining the guide based on the probability distribution comprises:
Determining information elements of the guide based on a cumulative distribution function (CDF)-transformed histogram of a normalized inverse histogram
A method of providing a guide, including.

Images