JP2009129269A - Information reader and information reading method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information reader that is capable of reading information even when a reading means and a reading object are not in a specific positional relationship, and that is miniaturizable, and to provide an information reading method. <P>SOLUTION: The information reader includes: a feature point extraction unit 111 for extracting a feature point from an input image representing an information display part for displaying information in an optically readable manner and feature points arranged in the vicinity; a surface detection unit 112 for detecting a shape of the surface where the information display part is arranged on the basis of extraction results of the feature points; and an information analysis unit 12 for acquiring the information, which is displayed by the information display part, after correcting the information display part in the input image on the basis of the detection results of the surface detection unit 112. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information reading apparatus and method that can optically read information and can be downsized even when a reading unit and a reading object are not in a specific positional relationship.

  In the field of article distribution and inventory management, an RFID (Radio Frequency IDentification) tag in which information indicating the type of product, delivery destination, storage position, etc. is recorded is attached to the article, and the RFID tag is used during actual work. By using a reader to read information, information is used so as not to cause erroneous delivery or management failure.

  However, if there are many items that are subject to distribution and inventory management, when trying to read RFID tags with a hand-held RFID reader, the RFID tags must be read one by one for each item. Will be forced.

  If a stationary RFID reader is used, a plurality of RFID tags can be read at the same time, which reduces the burden on the administrator. However, in this case, since the positional relationship of the plurality of RFID tags read at the same time becomes unclear, when trying to extract a part of the plurality of articles, it depends on visual observation or a handheld RFID reader is used. You will have to search for the item to be extracted.

  Moreover, even if a plurality of RFID tags can be read at the same time by a stationary RFID reader, the RFID tag must be brought close to the RFID reader to some extent, and is usually limited to a few meters at most. RFID tags can be read from a longer distance by using a high-power RFID reader, but in addition to the reader device becoming expensive, it consumes more power because it always emits a high-power radio signal, and running costs are increased. Also gets higher.

  Furthermore, RFID tags are generally expensive, and among them, those that can be used even when attached to metal products are very expensive, so it is difficult to use them for the management and distribution of low-priced products. there were.

  In addition, a method of managing articles using a bar code or a two-dimensional code instead of an expensive RFID tag is generally used. Since barcodes or two-dimensional codes can be placed by sticking or printing a seal-like member on the surface of goods or packaging media, they are less expensive than RFID tags, and management of low-priced products・ Applicable to distribution.

  However, since the reader cannot read the barcode or the two-dimensional code unless the barcode or the two-dimensional code faces the reading surface of the barcode reader or the two-dimensional code reader, the manager can change the direction of the article or the reader one by one. In other words, the work of aligning the barcode on the article side or the two-dimensional code with the reading surface of the reader is forced.

  Although an apparatus for automatically reading a barcode or a two-dimensional code arranged on an article or the like has been realized, for that purpose, a part of the article where the barcode or the two-dimensional code is arranged is detected and its surface is detected. It is necessary to provide a mechanism on the reader side for directing the reader to the reader side, which makes it difficult to reduce the size and price of the reader.

  That is, in general, in order to read a barcode or a two-dimensional code with a barcode reader or a two-dimensional code reader, a specific positional relationship in which the barcode or the two-dimensional code and the reading surface of the reader are opposed to each other. There is a problem that a barcode or a two-dimensional code cannot be read unless such a positional relationship can be established. Furthermore, if the surface on which the bar code or two-dimensional code is displayed is curved, the bar constituting the code and the small square block itself will be deformed into a curved figure that is not a curve or a square. Even if the code or the two-dimensional code is directly faced, there is a case where it cannot be read without substantially facing the face. In addition, since there is only one barcode or two-dimensional code that can be read in one operation, there is a complexity that the reading operation must be repeated when managing a plurality of articles.

  There is an OCR (Optical Character Reader) as a device that optically reads characters rather than encrypted information such as barcodes or two-dimensional codes. However, if the reader and the information display surface do not face each other, the information is displayed. There is no change in reading correctly.

As a technique related to enabling the barcode to be read without being directed in a specific direction, there is a “collective barcode symbol recognition system” disclosed in Patent Document 1.
In the invention disclosed in Patent Document 1, a barcode symbol is created by using special ink, and the barcode symbol is extracted at once by a stereoscopic scanner. Thus, a plurality of articles can be managed collectively.
Japanese Patent Laid-Open No. 10-302021

  However, the invention disclosed in Patent Document 1 must generate a three-dimensional transmission image in order to extract barcode symbols. For this purpose, it is necessary to three-dimensionally irradiate a plurality of articles arranged in the space with transmitted waves that can pass through them, but the place where such an apparatus can be installed is limited. Of course, the reader cannot have a portable device configuration.

  Thus, an apparatus that can read information and can be miniaturized even when the reading means and the reading object are not in a specific positional relationship has not been realized.

  The present invention has been made in view of such problems, and provides an information reading apparatus and method that can read information and can be miniaturized even when the reading means and the reading object are not in a specific positional relationship. For the purpose.

  In order to achieve the above object, according to the first aspect of the present invention, a feature point is obtained from an input image in which an information display unit for displaying information optically readable and a feature point arranged in the vicinity thereof are copied. Feature point extraction means to extract, surface shape detection means for detecting the shape of the surface on which the information display unit is arranged based on the extraction result of the feature points, and input image based on the detection result of the surface shape detection means The present invention provides an information reading apparatus characterized by comprising an information analysis means for acquiring information displayed by the information display section after correcting the information display section.

  In order to achieve the above object, as a second aspect, the present invention provides a feature point extraction means for extracting a feature point from an input image in which the feature point is copied, and a feature point based on the feature point extraction result. The surface shape detecting means for detecting the shape of the surface on which the lens is arranged and the input image are corrected based on the detection result of the surface shape detecting means and then compared with the reference image recorded in the reference state information recording means Thus, an information reading device having an information analysis means for acquiring difference information is provided.

  In order to achieve the above object, as a third aspect, the present invention provides a feature from an input image in which an information display unit that displays information in an optically readable manner and feature points arranged in the vicinity thereof are copied. Input based on the feature point extraction process for extracting points, the surface shape detection process for detecting the shape of the surface on which the information display unit is arranged based on the extraction result of the feature points, and the detection result in the surface shape detection process The present invention provides an information reading method characterized by comprising an information analysis step of obtaining information displayed by the information display unit after correcting the information display unit in the image.

  In order to achieve the above object, as a fourth aspect, the present invention provides a feature point extraction step for extracting a feature point from an input image on which the feature point is copied, and a feature point based on the feature point extraction result. The surface shape detection step for detecting the shape of the surface on which the lens is arranged, and the input image is corrected based on the detection result of the surface shape detection means, and then compared with the reference image recorded in the reference state information recording means And providing a difference information acquisition step of acquiring difference information.

  According to the present invention, it is possible to provide an information reading apparatus and method that can read information and can be downsized even when the reading unit and the reading object are not in a specific positional relationship.

[First Embodiment]
A first embodiment in which the present invention is suitably implemented will be described. As shown in FIG. 1, the information reading apparatus according to the present embodiment obtains a feature point from an input image in which an information display unit that displays information in an optically readable manner and a feature point arranged in the vicinity thereof are copied. A feature point extraction unit 111 to be extracted, a surface detection unit 112 that detects the shape of the surface on which the information display unit is arranged based on the extraction result of the feature points, and an input image based on the detection result of the surface detection unit 112 As shown in FIG. 2, the feature point extraction unit 111 and the surface detection unit 112 are provided with a basic configuration of an information analysis unit 12 that acquires information displayed by the information display unit. The space recognition unit 11 and the information analysis unit 12 including the image processing unit 10 further include the camera 2.

The feature point extraction unit 111 extracts feature points existing in the input image input from the camera 2, and specifies a location where information is displayed and a distance to the location. The surface detection unit 112 detects the surface shape of the information display surface from the position of the extracted feature point.
The information analysis unit 12 corrects and reproduces the distortion of the information captured in the image based on the surface shape information of the information display surface acquired from the surface detection unit 112.

  For an article to be managed, an information display part and a feature point are formed at an arbitrary position on the surface. The information display unit and the feature points may be formed by any method as long as they can be read optically. For example, a seal-like material may be attached to the surface of the article, printed on the surface of the article, or unevenness may be formed on the surface of the article by laser processing or the like. The feature point may be optically detectable by reflecting light of a specific wavelength or forming it in a specific shape.

The operation of the information reading apparatus 1 will be described.
When an image is input from the camera 2, the feature point extraction unit 111 extracts a feature point that is copied in the input image. The surface detection unit 112 detects the surface shape of the information display surface based on the positions and shapes of three or more feature points extracted by the feature point extraction unit 111.

  As an example, as shown in FIG. 3A, four feature points are arranged around a barcode serving as an information display unit. If the feature points are copied in the input image as shown in FIG. 3B, the surface detection unit 112 determines the surface shape of the surface on which the information display unit exists based on the position and shape of the feature points in FIG. c) It is estimated that In this case, the barcode that is originally a set of straight lines is reflected in the input image as a set of curves. Therefore, the information analysis unit 12 performs the correction process for converting the set of curves shown in FIG. 3C to the set of straight lines shown in FIG. 3A and extracts information (decodes the barcode). To do.

  In this example, the information display unit is a barcode. However, even if the information is a character string or a symbol, the input image is displayed on the information display surface by correcting it according to the surface shape of the information display surface. Information is read from the information display surface even if it is not photographed at a position directly opposite to the image, the information display surface is not flat, or the input image is distorted due to the influence of aberration. It is possible. Similarly, when a two-dimensional code is used instead of a bar code as the information display unit, the input image is not photographed at a position facing the information display surface, or the information display surface is not flat. Even when the input image is distorted due to the influence of aberration, information can be read from the information display surface. The same applies to other embodiments.

  By processing using a plurality of input images taken at different angles, the information on the information display unit can be reliably restored by reducing the number of blind spots.

  When a plurality of information display units are detected in one image, the positional relationship of the information display unit in the input image corresponds to the positional relationship of the articles, so even if a plurality of articles are mixed, The position can be easily identified. For example, if an image of a shelf on which a product is placed is taken, it is possible to easily specify at which step and in which position a desired article is stored.

  As described above, the information reading apparatus according to the present embodiment displays the information on the information display unit even when the camera is not directly facing the object to be read or the part where the information display unit is arranged is deformed into a curved surface. Information can be read. In addition, when there are a plurality of information display units in the imaging range of the camera, it is possible to read the information collectively. In addition, since the information displayed on the information display unit can be read within the imaging range of the camera, it is not necessary to bring the article close to the camera. Further, since a mechanism for changing the direction of the article is not necessary, the apparatus can be easily downsized.

[Second Embodiment]
The second embodiment in which the present invention is preferably implemented will be described. FIG. 4 shows the configuration of the information reading apparatus according to the present embodiment. The information reading apparatus according to the first embodiment includes an article database 3 (hereinafter referred to as an article). The information DB) is further added.

  The article information DB 3 stores article appearance information (color, box shape, logo mark, etc.) and information (display information) displayed as an information display section in association with each other.

In this embodiment, the information analysis part 12 complements using the article information in article information DB3, when the display information acquired after correcting based on the surface shape of an information display surface is incomplete. Process.
For example, as shown in FIG. 5 (a), the information display unit is a barcode, and a part of the information display unit displayed on the article is defaced so that the number corresponding to the first two digits is completely restored. When the information cannot be obtained, the information analysis unit 12 restores information for a portion other than the first two digits to obtain a numeric string. In addition, the information analysis unit 12 detects appearance information copied in the input image (FIG. 5B). Then, data matching the combination of the restored number string and the extracted appearance information is extracted from the article information DB 3 and output as a complement result (FIG. 5C).

  As described above, the information reading apparatus according to the present embodiment can be supplemented by using the article information stored in the article information DB even when part of the display information cannot be read from the information display unit displayed on the article. It is. Therefore, errors in reading information can be reduced.

  Even when all the information displayed on the article as the information display unit can be read, the read information may be corrected using data stored in the article information DB. In this way, misrecognition during information reading is reduced, and reading accuracy is improved.

  Other points are the same as those in the first embodiment, and the same effects can be obtained.

[Third Embodiment]
A third embodiment in which the present invention is preferably implemented will be described. FIG. 6 shows the configuration of the information reading apparatus according to this embodiment. The information reading device 51 includes a feature point extraction unit 611 that extracts feature points from an input image in which the feature points are copied, and surface detection that detects the shape of the surface on which the feature points are arranged based on the feature point extraction results. And an information analysis unit 12 that obtains difference information by correcting the input image based on the detection result of the surface detection unit 612 and comparing it with a reference image recorded in the reference state information DB 4. As shown in FIG. 7, the space recognition unit 61 including the feature point extraction unit 611 and the surface detection unit 612 and the information analysis unit 62 form an image processing unit 60, and further includes a camera 52 as shown in FIG. In addition, a reference state information DB 4 is provided.

  In the present embodiment, feature points are formed not on the article itself to be managed but on a shelf board on which the article is placed as shown in FIG. Further, the reference state information DB 4 stores image data of the surface state of the shelf when no article is placed.

The operation of the information reading apparatus according to this embodiment will be described.
Similar to the first embodiment, the feature point extraction unit 111 and the surface detection unit 112 perform feature point extraction and surface detection on the input image.
The information analysis unit 12 corrects the input image based on the surface detection result, compares it with the information stored in the reference state information DB 4, and outputs difference information. For example, the presence or number of articles placed on the shelf can be grasped by comparing the state shown in FIG. 8B with the captured input image, with the shelf being empty. At this time, since the input image is corrected based on the surface detection result, comparison is possible even if the shooting position of the input image does not completely match the shooting position of the reference state information. In addition, since the process which detects an article in an image can be performed using well-known processes, such as an outline detection, description is abbreviate | omitted.

  When the information reading apparatus according to the present embodiment is applied to inventory management, if it is determined that there is a missing item as a result of comparison, a warning is given to the mobile terminal owned by the store clerk about replenishment of the item. It is also possible to use a system that automatically orders missing items.

  As described above, the entire input image is regarded as an information display unit, corrected based on the feature point and the extraction result, and compared with the reference information, thereby detecting a difference from the reference state. In a site such as inventory management, it is not always easy to photograph a shelf or the like from the same position every time. However, the information reading apparatus according to the present embodiment does not have to be completely coincident with each other. It can be configured.

[Fourth Embodiment]
A fourth embodiment in which the present invention is preferably implemented will be described.
FIG. 9 shows the configuration of the information reading apparatus according to this embodiment. The information reading apparatus according to the present embodiment has a configuration in which a display device 5 for displaying information is added to the information reading apparatus according to the first embodiment. In the present embodiment, the information reading device 1 is mounted on a movable object such as a cart.
The function of each part of the information reading apparatus 1 is the same as that of the first embodiment.

  As shown in FIG. 10, the information reading apparatus according to the present embodiment is used in an environment where a plurality of feature points and information display units are formed on the ceiling, and the imaging range of the camera 2 faces the ceiling. The information reading device 1 detects the surface state of the part where the information display unit is formed based on the detection result of the feature point, and reads the information display unit while correcting the result according to the detection result. Or information can be acquired from the information display unit even if the cart is tilted.

  As the cart moves, the imaging range of the camera 2 also moves. Therefore, the information reading device can acquire different information from the information display unit according to the position of the card and display the information on the display device. For example, as shown in FIG. 11A, when the cart is located immediately below the point C and the information display unit arranged on the ceiling at each point A to E is within the imaging range of the camera 2. Then, as shown in FIG. 11B, the display device 5 can display the current position and the surrounding position information.

  The information displayed on the ceiling as the information display section may be position information (such as a sales floor name) for specifying the position where the information display section is formed, or other information related to the position (the product of the sales floor) Or other bargain information).

  In a system in which multiple infrared or wireless transmitters are installed and a receiver is mounted on a cart or the like to display information on a display device installed on the cart, it is necessary to route the wiring to the ceiling. In addition, a huge amount of capital investment is required. Furthermore, since it is necessary to always emit infrared rays or radio signals, the running cost is high. Since the information reading apparatus according to the present embodiment only needs to operate the camera and input an image (take an image of the information display unit) only when necessary, energy saving and low running cost are achieved.

  Here, the case where information acquired from the information display unit is displayed on the display device has been described as an example. However, as shown in FIG. 12, a related information storage unit 6 for storing information related to position information is provided, Information related to the position information acquired from the display unit may be read from the related information storage unit 6 and displayed on the display device 5. Moreover, as shown in FIG. 13, the communication part 7 which implement | achieves a communication function is provided, the positional information acquired from the information display part is transmitted to the related information server 8 provided with the related information storage part 82 which stored the related information, The related information according to the position information may be received from the information distribution unit 81 of the related information server 8 and the information may be displayed on the display device 5.

In addition, the said embodiment is an example of suitable implementation of this invention, and this invention is not limited to this.
For example, in the above-described embodiment, processing based on still images has been described as an example. However, when processing based on moving images is performed, it is regarded as a set of a plurality of still images and a predetermined interval (for example, 1 The same process as described above may be performed on the frame image every second).
Further, the camera and the display device may be configured separately from the information reading device.
Further, the article information DB and the reference state information DB are not necessarily provided inside the information reading apparatus, and the article information and the reference state information are acquired from an external apparatus based on the information acquired by the information analysis unit. Also good.
As described above, the present invention can be variously modified.

It is a figure which shows the basic composition of the information reading apparatus which concerns on 1st Embodiment which implemented this invention suitably. It is a figure which shows the structure of the information reader which concerns on 1st Embodiment. It is a figure which shows the relationship between a feature point and an information display part, and a surface state. It is a figure which shows the structure of the information reader which concerns on 2nd Embodiment which implemented this invention suitably. It is a figure which shows the information complementation operation | movement of the information reader which concerns on 2nd Embodiment. It is a figure which shows the basic composition of the information reader which concerns on 3rd Embodiment which implemented this invention suitably. It is a figure which shows the structure of the information reader which concerns on 3rd Embodiment. It is a figure which shows reference | standard state information and the information which the information analysis part acquired. It is a figure which shows the structure of the information reader which concerns on 4th Embodiment which implemented this invention suitably. It is a figure which shows the use condition of the information reader which concerns on 4th Embodiment. It is a figure which shows an example which displays the information acquired by reading the information display part on a display apparatus. It is a figure which shows the structural example of the information reading apparatus which displays the information relevant to the information acquired by reading the information display part on a display apparatus. It is a figure which shows the structural example of the information reading apparatus which displays the information relevant to the information acquired by reading the information display part on a display apparatus.

Explanation of symbols

1, 51 Information reader 2, 52 Camera 3 Item information DB
4 Standard state information DB
DESCRIPTION OF SYMBOLS 5 Display apparatus 6,82 Related information storage part 7 Communication part 8 Related information server 10, 60 Information processing part 11, 61 Spatial recognition part 12, 62 Information analysis part 81 Information distribution part 111,611 Feature point extraction part 112,612 Surface Detection unit

Claims (15)

A feature point extracting means for extracting the feature point from an input image in which an information display unit for displaying information optically readable and a feature point arranged in the vicinity thereof are copied;
Surface shape detecting means for detecting the shape of the surface on which the information display unit is arranged based on the extraction result of the feature points;
An information reading unit comprising: an information analysis unit configured to acquire information displayed by the information display unit after correcting the information display unit in the input image based on a detection result of the surface shape detection unit. apparatus.   2. The information according to claim 1, wherein, for each of a plurality of information display units copied in the input image, the feature points are individually extracted to detect a surface shape, and information displayed by the surface is acquired. Reader.   The said surface shape detection means detects the shape of the surface where the said information display part is arrange | positioned based on the position and shape of the said feature point copied to the said input image. Information reader. Comprising article information storage means for storing article information in which display information displayed by the information display section and appearance information of an article on which the information display section is arranged are associated;
If the information display unit cannot read a part of the information display unit copied in the input image, the information analysis unit uses the product information stored in the product information storage unit for the information acquired from the information display unit. The information reading apparatus according to claim 1, wherein the information reading apparatus complements the information reading apparatus.   The information reading apparatus according to claim 1, further comprising an imaging unit that captures the input image.   The information reading apparatus according to claim 5, wherein the information reading apparatus is mounted on a movable object, and a shooting direction of the imaging unit is fixed in a specific direction.   The information reading apparatus according to claim 1, further comprising a display unit configured to display information acquired by the information analysis unit.   It has an accumulation means for accumulating information related to the information displayed by the information display section, and reads information related to the information acquired by the information analysis means from the accumulation means and displays the information on the display means. The information reading apparatus according to claim 7. Means for transmitting information acquired by the information analysis means to a server;
Means for receiving information related to the information acquired by the information analysis means from the server;
8. The information reading device according to claim 7, further comprising means for displaying information received from the server on the display device.   The information reading apparatus according to claim 1, wherein the information display unit is a bar code or a two-dimensional code. Feature point extraction means for extracting the feature points from an input image in which the feature points are copied;
Surface shape detection means for detecting the shape of the surface on which the feature points are arranged based on the extraction results of the feature points;
An information analysis unit that corrects the input image based on the detection result of the surface shape detection unit, and obtains difference information by comparing with the reference image recorded in the reference state information recording unit; An information reader characterized by the above. A feature point extracting step of extracting the feature point from an input image in which an information display unit that displays information optically readable and a feature point arranged in the vicinity thereof are copied;
A surface shape detection step of detecting the shape of the surface on which the information display unit is arranged based on the extraction result of the feature points;
And an information analysis step of obtaining information displayed by the information display unit after correcting the information display unit in the input image based on a detection result in the surface shape detection step. Reading method.   13. The information according to claim 12, wherein, for each of a plurality of information display sections copied in the input image, the feature points are individually extracted to detect a surface shape, and information displayed by them is acquired. Reading method.   The shape of the surface on which the information display unit is arranged is detected based on the position and shape of the feature point imaged in the input image in the surface shape detection step. Information reading method. A feature point extraction step of extracting the feature points from an input image in which the feature points are copied;
A surface shape detection step for detecting the shape of the surface on which the feature points are arranged based on the extraction results of the feature points;
A difference information acquisition step of correcting the input image based on the detection result of the surface shape detection unit and acquiring difference information by comparing with the reference image recorded in the reference state information recording unit; An information reading method.

Images