JP2016099760A - Information providing system and printed matter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information providing system that provides a content in accordance with a user's way of viewing a printing surface, and a printed matter.MEANS FOR SOLVING THE PROBLEM: An information providing system 1 includes a printed matter 3 and an information providing device 5. The printed matter 3 has a printing surface 11 having a plurality of markers 7 printed uniformly over the entire surface. The markers 7 each have marker identification information 25. The information providing device 5 includes storage means 15 that has content information 21 related to the printing surface 11 stored therein, and stores respective pieces of marker print position information 23 and the marker identification information 25 on the printing surface 11 in association with each other. The information providing device 5 includes imaging means 17 that creates information on a photographed image of the printing surface 11. The information providing device 5 includes detection means that detects marker images from the photographed image to read the identification information and searches for the marker print position information 23 corresponding to the identification information in the storage means 15. The information providing device 5 includes application means that displays, on the display means, the content information 21 in accordance with the marker print position information 23.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information providing system that provides information related to a printing surface, and a printed matter.

  Patent Document 1 describes an information providing system including a printed matter, a pen-type reading device, and an information device.

  The printed matter has a printing surface on which a plurality of markers are printed with infrared absorbing ink. The plurality of markers are arranged in a grid pattern in the vertical and horizontal directions of the printing surface without gaps, and have dot codes for identifying each other. The dot code is formed from a group of small dots that are not easily visible to the human eye. Visible information is further printed on the plurality of markers.

  The pen-type reading device is used with its front end in contact with the printing surface. The reading device reads the dot code of the marker printed on the contact portion with the printing surface, detects the unique code information given to the dot code, and transmits this to the information device.

  In the information device, content information related to the visible information is stored in association with the code information. When the code information is input from the reading device, the information device searches the content information using the code information as a search key, and displays the content information.

  Such an information providing system is configured to focus only on the marker that the user is touching with the reading device and output content information related to the marker. Therefore, it is impossible to realize an information provision form in which the user is aware of which part of the entire printing surface is focused and content information is output in accordance with the user's attention position on the printing surface.

JP 2007-213352 A

  It is an object of the present invention to provide an information providing system and a printed matter that can output content information in correspondence with a user's attention position with respect to a printing surface.

(1) In the information providing system of the present invention, a plurality of markers having different information are printed over the entire printed surface, and visible information is printed on the printed surface, and the visible information is related to the visible information. An information providing device that outputs content information to be
The information providing apparatus includes: a storage unit that stores the content information; an imaging unit that captures an image of the print surface and generates captured image information including a marker image; Reading the marker information from the marker image included in the image, detecting the position of the imaged part on the print surface, and displaying the content information in correspondence with the position of the imaged part on the display means Application means.
(2) The storage unit stores marker print position information indicating the print position of the marker on the printing surface in association with the marker information, and the detection unit searches for the read information of the marker. The marker print position information is searched from the storage means as a key, and the position of the imaged portion is determined based on the searched marker print position information.
(3) Further, the printed matter has a plurality of the printing surfaces, and each marker is printed so that the information of the marker does not overlap between the printing surfaces. The content information is stored in association with the print surface identification information for identifying the print surface, and the print surface identification information of the print surface on which the marker is printed is stored in association with the marker information. The printed surface identification information is searched from the storage means using the read marker information as a search key, and the application means acquires content information using the searched print surface identification information as a search key. To do.
(4) The storage means stores a reference value of the size of the marker image, and the detection means detects the size of the marker image in the captured image, and the detected marker image with respect to the reference value. It is characterized by obtaining a size ratio.
(5) The detection unit detects a direction of the marker image in the captured image, and the application unit causes the display unit to display the content information in accordance with the detected direction of the marker image. And
(6) The content information includes substitute image information that substitutes all or part of the visible information.
(7) The content information includes animation image information for displaying an animation related to an element constituting the visible information.
(8) The application means moves the content information displayed on the display means.
(9) In the printed matter described above, the plurality of markers are printed with invisible ink.
(10) The invisible ink is an ink that attenuates reflection of invisible wavelength rays on the printing surface.
(11) The plurality of markers are unevenly oriented.

  According to the information providing system of the present invention, a plurality of markers are printed over the entire printing surface, the imaging unit images the printing surface to generate captured image information, and the detection unit includes the captured image. By reading the information of the marker image, the position of the imaged part on the printing surface is detected. Then, the application means displays the content information on the display means in correspondence with the position of the imaged part. Therefore, it is possible to detect which position on the printing surface the user is paying attention to and provide content information corresponding to the position.

It is a block diagram which shows the information provision system which concerns on embodiment. It is a flowchart which shows operation | movement of the information provision apparatus which concerns on embodiment. It is a figure which shows an example of the use condition of the information provision system which concerns on embodiment. It is a block diagram which shows the modification of embodiment. It is a block diagram which shows the other modification of embodiment. It is a figure which shows an example of the use condition of another modification. (A) The top view which shows the arrangement | positioning mode of the marker which concerns on an Example, (b) It is a figure which shows an example of a marker.

  Embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol shown throughout each figure shows the same or the same thing.

  As shown in FIG. 1, the information providing system 1 includes a printed material 3 and an information providing device 5.

  The printed matter 3 has a printing surface 11 on which a plurality of markers 7 and visible information 9 are printed. The plurality of markers 7 are arranged across the entire surface of the printing surface 11 with a space between each other, and have different information. This information is typically identification information that is individually assigned to each of the markers 7 and identifies each other (hereinafter referred to as marker identification information 25). As such a marker 7, a two-dimensional code symbol or a one-dimensional code symbol that can be read by a known machine-readable technique represented by a QR code (registered trademark) symbol is used. A symbol that can be read by an original reading method may be used. The visible information 9 is information that can be recognized by human eyes. In the present embodiment, the world map as the visible information 9 is printed on the entire printing surface 11, but the present invention is not limited to this, and other visible information 9 may be printed.

  Such markers 7 and visible information 9 are typically printed on the printing surface 11 by offset printing. The marker 7 is preferably printed with invisible ink. This invisible ink is ink that is invisible to the human eye under visible light and absorbs light in an invisible wavelength region such as near infrared rays (attenuates reflection). On the other hand, the visible information 9 is printed with ink (basic inks such as cyan, magenta, and yellow) that are visible to human eyes under visible light.

  In the present embodiment, as shown in FIG. 1, the information providing apparatus 5 is realized using a computer including a central processing unit 13, a storage unit 15, an imaging unit 17, and a display unit 19 such as a display. The operation flow of the information providing apparatus 5 will be described with reference to FIGS.

  In advance, content information 21 related to the visible information 9 on the printing surface 11 is stored in the storage unit 15, and marker printing indicating the printing position of the marker 7 on the printing surface 11 in association with the marker identification information 25 of each marker 7. The position information 23 is stored. The storage means 15 may be a semiconductor storage means such as random-access memory or read-only memory, a recording medium such as a magnetic disk / CD-ROM, or a storage device in a microcomputer. The typical content information 21 is image information such as a still image or a moving image displayed on the display unit 19, and is still image information of a world map displayed on the printing surface in the present embodiment. The marker printing position information 23 is XY coordinate information of the marker 7 with the center of the printing surface 11 as the origin of the XY plane.

  First, information of a captured image including a marker image is input to the central processing unit 13 from the imaging unit 17 that images the printing surface 11 of the printed matter 3 (s11). The imaging means 17 receives light in the near-infrared region and generates image information to be imaged, and is disposed in the vicinity of the image sensor to irradiate the printing surface 11 with invisible rays such as infrared rays. It comprises a light emitting means, a lens for condensing the reflected light reflected by the printing surface 11 on the image sensor, and a filter that covers the light receiving surface of the image sensor and attenuates visible light.

  When information on the captured image is input, the central processing unit 13 detects a marker image included in the input captured image (s12), reads the marker identification information 25 from the marker image (s13), and detects the marker. The marker print position information 23 stored in the storage unit 15 is searched using the identification information 25 as a search key, and this is set as the position of the imaged portion on the print surface 11 (s14). Note that as preprocessing for detecting a marker image (s12), it is preferable to perform binarization processing on the input captured image to generate binary image information. When a known symbol such as a QR code symbol or a barcode is used as the marker 7, the marker image is detected by using a known machine-readable technique defined for each symbol and the marker identification information 25 is read. (S13). Thus, the central processing unit 13 of the computer functions as a detection unit that detects the marker image from the information of the captured image, reads the marker image, and detects the position of the imaged portion of the printing surface 11. The function as the detecting means is not limited to only the detection of the position of the imaged part, and may have a function of detecting the direction and size of the marker image as will be described later.

  When a plurality of marker images are detected from the input captured image, the central processing unit 13 performs processing up to the search for the marker print position information 23 for each marker image. Then, an average value of the marker print position information 23, preferably a weighted average value giving the reciprocal of the distance from the center of the captured image to each marker as a weight for each marker print position information 23 ( Hereinafter, this is referred to as average marker printing position information.) This is the position of the imaged portion.

  When the position of the part to be imaged is detected, the central processing unit 13 acquires the content information 21 stored in the storage unit 15 (s21). Then, as shown in FIG. 3, the content information 21 is displayed on the display means 19 in correspondence with the position of the imaged part (s22). Specifically, the display coordinates of the content information 21 are shifted so that the portion corresponding to the position of the imaging target in the content information 21 is displayed at the center of the display. Thus, the central processing unit 13 of the computer also functions as application means for displaying the content information 21 based on the information detected by the detection means.

  According to the information providing system 1 according to the present embodiment, the markers 7 are scattered all over the printing surface 11, and the position of the imaging target is detected by reading the marker identification information 25 therein. And the content information 21 is displayed according to the position of the to-be-imaged part. That is, it is possible to detect which position on the printing surface 11 the user is paying attention to and display the content information 21 in accordance with this position.

  Moreover, in order to provide the content information 21 suitable for the user's attention position in real time, the above-described operation flow is preferably performed for each frame.

The present invention is not limited to the above embodiment.
For example, as illustrated in FIG. 4, a reference value 29 of the size of the marker image in the captured image may be stored in the storage unit 15 in advance. The reference value 29 is a value serving as a reference for measuring the size of the marker image in the captured image, and is determined according to the size of each marker 7 printed on the printing surface 11. When the marker image is detected (s12), the central processing unit 13 detects the size of the marker image in the captured image and refers to the reference value 29 of the storage unit 15 to determine the size of the marker image with respect to the reference value 29. The ratio may be obtained. The obtained ratio is stored in the memory in association with the marker identification information 25 as necessary. In addition, for example, when a QR code symbol is used, the size of the marker image in the captured image is obtained by detecting the position detection pattern from the captured image and obtaining the maximum length between the position detection patterns. It is detected by determining the size of the marker image.

  When a plurality of marker images are detected in the captured image, the central processing unit 13 detects the size of each marker image, obtains an average value of these sizes, and calculates this as a marker in the captured image. The image size (average size) is used. Further, it is preferable to obtain a ratio (average ratio) to the reference value 29 using the average size of the marker image.

  Furthermore, the central processing unit 13 may enlarge or reduce the content information 21 according to the obtained ratio and display it on the display unit 19 (s22). According to the information providing system 1 as described above, it is possible to display the content information 21 according to the degree of attention of the user.

  Furthermore, the central processing unit 13 may correct the position of the imaging target using the obtained ratio. In this correction, when the marker image is detected (s12), the central processing unit 13 detects the position (coordinates) of the marker image with the center of the captured image as the origin. The central processing unit 13 multiplies the detected position of the marker image by the reciprocal of the ratio, and uses the obtained value as correction information. When searching for the marker print position information 23, the central processing unit 13 adds the correction information to the marker print position information 23 to determine the position of the imaged portion (s14). According to such an information providing system 1, it is possible to improve the detection accuracy of a user's attention part.

  In addition, when a plurality of marker images are detected in the captured image, the central processing unit 13 detects the position of each marker image in the captured image, obtains the average value of these positions, A value obtained by multiplying the reciprocal of the average ratio is used as correction information, and this correction information is added to the average marker printing position information to determine the position of the imaged portion.

  Further, when detecting the marker image (s12), the central processing unit 13 may detect the direction of the marker image in the captured image, and further rotate and display the content information 21 in accordance with this direction (s22). . Direction detection is appropriately performed in accordance with the type and shape of the marker 7 being printed. For example, when a QR code symbol is used as the marker 7, a position detection pattern is detected from the captured image, and the direction of the marker image is detected based on the diagonal angle of the position detection pattern in the captured image. When a one-dimensional code symbol typified by a bar code symbol is used, the direction of the marker image is detected based on the bar inclination angle in the captured image. According to such an information providing system 1, the content information 21 can be displayed in accordance with the user's attention direction with respect to the printing surface 11.

  In addition, when a some marker image is detected in a captured image, it is preferable that the central processing unit 13 detects the direction of each marker 7, and calculates | requires the average value of these directions.

  The printed matter 3 of the present embodiment may include a plurality of printing surfaces 11a, 11b, and 11c on which the markers 7 are printed so that the marker identification information 25 does not overlap each other. In this case, the storage unit 15 stores content information 21 related to the visible information 9 of each printing surface 11a, 11b, 11c in association with the printing surface identification information 31 for identifying each printing surface 11a, 11b, 11c, Furthermore, the print surface identification information 31 of the print surfaces 11a, 11b, and 11c on which the marker 7 is printed is stored in association with the marker identification information 25. The central processing unit 13 searches the printing surface identification information 31 from the storage unit 15 using the detected marker identification information 25 as a search key, and the content information 21 from the storage unit 15 using the printing surface identification information 31 as a search key. Search is performed (s21). According to such an information providing system 1, it is possible to specify the printing surfaces 11 a, 11 b, 11 c on which the user is paying attention and to provide the content information 21 related to the visible information 9.

In addition, the content information of the present embodiment is not limited to only image information related to the entire visible information 9 printed on the printing surface 11 (hereinafter referred to as overall image information).
For example, as shown in FIG. 5, the content information stored in the storage means 15 includes animation image information 51a and 51b (hereinafter referred to as element image information) related to the element 49 constituting the visible information 9a. May be included. This element image information is stored in association with element print position information 53 indicating the print position of the element 49 on the printing surface 11. The element printing position information 53 is typically XY coordinate information of the element 49 having the center of the printing surface 11 as the origin of the XY plane. It should be noted that the part corresponding to the element image information is excluded from the entire image information 55 shown in FIG.

  The central processing unit 13 acquires the entire image information 55 (s21), and further executes the next process when displaying the entire image information 55 (s22). First, when the entire image information 55 is acquired (s21), the element print position information 53 is read from the storage unit 15 together with the element image information. When the entire image information 55 is displayed (s22), the relative position of the element 49 with respect to the position of the imaged part is obtained from the difference between the read element print position information 53 and the position of the imaged part, and the relative position is obtained. In addition, the display position of the element image information is set, and this is superimposed on the entire image information 55 and displayed. Further, the central processing unit 13 moves or rotates the display position after superimposing and displaying the element image information according to the relative position.

  According to such an information providing system 1, as shown in FIG. 6, element image information (animation image information 51 a) of the element 49 in the vicinity of the position of the imaging target is displayed together with the entire image information 55. Then, the element image information (animation image information 51b) is displayed while moving. In this way, the content displayed on the display means 19 can be changed or moved.

  Further, the content information is related to the visible information 9, and may be substitute image information that substitutes the whole or a part thereof. The entire substitute image information is, for example, image information obtained by changing the color arrangement pattern of the entire landscape in the entire image information 55 of FIG. 5 in accordance with the images of spring, summer, autumn and winter. Further, the partial substitute image information is, for example, image information obtained by changing the color or model of the vehicle, which is the element image information in FIG.

  Moreover, what is printed on one printing surface 11 is not limited to one visible information 9, and a plurality of visible information may be printed. For example, one printing surface 11 can be divided into two areas, and visible information of a car traveling on a road and an airplane flying between clouds can be assigned to each area and printed. In this case, the storage unit 15 stores the content information related to the visible information of the area in association with the area identification information for identifying each area, and further stores the area identification information in association with the marker identification information 25. Remember. The central processing unit 13 searches the storage means 15 for area identification information using the detected marker identification information 25 as a search key, and stores content information (for example, still image information of a car) using the area identification information as a search key. A search is made from the means 15 (s21). According to such an information providing system 1, it is possible to provide content information corresponding to the area of the printing surface that the user is paying attention to.

  Further, the markers 7 according to the present embodiment do not need to be regularly arranged, and if there is a place where the markers 7 cannot be arranged due to the specifications of the printed matter 3 or printing convenience, the markers 7 may be appropriately shifted in position. . Further, the degree of density of the markers 7 printed on the printing surface 11 is not particularly limited, and is appropriately determined according to the angle of view of the imaging unit 17. Furthermore, it is preferable to arrange the markers 7 densely in a portion where the detection accuracy of the position of the imaged portion is particularly desired. The plurality of markers 7 may be arranged vertically and horizontally on the printing surface 11, but are preferably arranged in a staggered manner as shown in FIG. 1 in order to increase the detection accuracy of the position of the imaged portion. . Furthermore, it is preferable that the plurality of markers 7 be arranged so that equilateral triangles are formed when the neighboring markers 7 are connected to each other by a straight line. According to such an arrangement mode, it is possible to improve the detection accuracy of the position of the above-described imaging target portion regardless of the imaging direction.

  Hereinafter, an embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 7A, two-dimensional code symbols of the same size as the marker 35 are printed on the printing surface 33 evenly arranged at intervals. In addition, in order to prevent a decrease in the detection rate of the marker image or the detection rate of the size with respect to the captured image generated by imaging from a specific direction, the directions of the markers 35 are printed unevenly. As shown in FIG. 7B, the marker 35 includes a function pattern 37 and an encoding area 39.

  The function pattern 37 is used for detection of a marker image in a captured image, detection of its direction, and detection of its size, and is composed of a frame pattern 41, a blank area 43, and a directional dot 45. . The frame pattern 41 is a pattern indicating the outer shape of the marker 35, and is a square frame having a predetermined thickness. The blank area 43 is a blank area (upper right, lower right, and lower left blank areas in FIG. 7B) provided at the three corners inside the frame pattern 41. The directional dot 45 is a dot that indicates the directionality of the marker 35, is a dot provided in a corner other than the three corners (upper left area in FIG. 7B), and is a corner of the frame pattern 41. It is provided with a gap between.

  The encoded area 39 is an area other than the function pattern 37, and an encoded dot 47 obtained by encoding the marker identification information is assigned to the encoded area 39. The encoded dot 47 may include a dot encoded with error detection and error correction information.

  The information providing apparatus in the present example has the following configuration and flow in addition to the configuration and operation flow of the above embodiment.

  In the present embodiment, the storage unit 15 stores in advance the direction information of each marker 35 with respect to the printing surface 33 in association with each marker identification information. This direction information is, for example, rotation angle information of the markers 35b and 35c with respect to the marker 35a.

  When detecting the marker image (s12), the central processing unit 13 detects an image of the frame pattern 41 (hereinafter referred to as a frame image) from the captured image. This frame image can be detected by a known image processing technique. For example, after performing a labeling process on the captured image, a line image having a predetermined thickness is closed for each region using contour tracking or the like. It is detected by judging whether it is. Next, the central processing unit 13 scans the inside of each frame image, detects whether there is a dot at one corner, and detects that there is no dot at the other corner. Thereby, a marker image included in the captured image is detected.

  When reading the marker identification information (s13), the central processing unit 13 scans the inside of the frame image to detect the encoded dot 47. Then, the marker identification information is obtained by analyzing the arrangement pattern of the encoded dots 47. In analyzing the encoded dot 47, perspective transformation or affine transformation may be performed.

  Further, when detecting the size of each marker image, the central processing unit 13 obtains a virtual rectangle circumscribing the marker image so that at least the short side passes through the vertex of the marker image, and between the vertices on the short side The distance is obtained, and this distance is set as the marker size. This virtual rectangle is obtained, for example, by the following process. First, the xy coordinates of each vertex of the marker image are detected, and the values of the xy coordinates of each vertex are compared to identify the maximum value and minimum value of the x coordinate, and the maximum value and minimum value of the y coordinate. Then, a rectangle whose vertical side passes through the maximum value and minimum value of the x coordinate and whose horizontal side passes through the maximum value and minimum value of the y coordinate is obtained.

  Although the present invention has been described above, the present invention can be implemented in variously modified, modified, and modified embodiments based on the knowledge of those skilled in the art without departing from the spirit thereof. Are also within the scope of the present invention.

  For example, the central processing unit 13 of the present invention may display additional information on the display unit 19 in accordance with the position of the imaged part. This additional information is information that is not included in the visible information 9 on the printing surface 11. For example, in FIG. 1, the additional information is image information of buildings, special products, and the like representing each region. The storage unit 15 stores additional information in association with information indicating the position in the printing surface 11 on which the additional information is displayed. The central processing unit 13 acquires the content information 21 from the storage unit 15, searches for additional information using the position of the imaged part as a search key (s 21), and superimposes the additional information on the content information 21 to display the display unit 19. Is displayed (s22). According to such an information providing system 1, the content information 21 (for example, map information) is displayed in accordance with the portion of interest of the user, and additional information (for example, an image of a building or the like) that is particularly related to the portion of interest is displayed. 19 can be displayed. Further, the additional information is not limited to image information of a moving image or a still image, and may be sound information, and the information providing apparatus 5 may include output means such as a speaker.

  Further, the filter of the imaging means 17 may be slidable to a position that does not cover the light receiving surface of the image sensor. As a result, the image sensor can receive visible light, and generates information of a captured image using visible light. In such an information providing system, the central processing unit 13 can cause the display unit 19 to display a captured image using visible light.

  The information that each marker has is not limited to the marker identification information 25, and may be XY coordinate information of each marker 7 on the printing surface 11. In this case, the central processing unit 13 detects the marker image included in the input captured image, detects the printing position information of the marker directly from the marker image, and uses this to capture the image on the printing surface. The position of the part. Therefore, the marker printing position information 23 may not be stored in the storage unit 15.

DESCRIPTION OF SYMBOLS 1 ... Information provision system 3 ... Printed matter 5 ... Information provision apparatus 7 ... Marker 9 ... Visible information 11 ... Printing surface 15 ... Storage means 17 ... Imaging means 19 ... Display means 21 ... Content information 23 ... Marker printing position information 49 ... Element 51a , 51b ... Animation image information

Claims (11)

A plurality of markers having different information from each other are printed over the entire printed surface, and further visible information is printed on the printed surface;
An information providing device for outputting content information related to the visible information;
Is an information providing system with
The information providing apparatus includes:
Storage means for storing the content information;
Imaging means for imaging the printing surface and generating information of a captured image including a marker image;
Detecting means for acquiring information of the captured image, reading information of the marker from the marker image included in the captured image, and detecting a position of the imaged portion on the print surface;
Application means for causing the display means to display the content information in correspondence with the position of the imaged part;
Information provision system with The storage means stores marker printing position information indicating a printing position of the marker on the printing surface in association with the marker information,
The detecting unit searches the marker printing position information from the storage unit using the read marker information as a search key, and determines the position of the image pickup unit based on the searched marker printing position information. The information providing system according to claim 1, wherein the system is an information providing system. The printed matter has a plurality of printing surfaces, and each marker is printed so that the information of the marker does not overlap between the printing surfaces,
The storage means stores the content information in association with printing surface identification information for identifying each of the printing surfaces, and prints the printing surface on which the marker is printed in association with the marker information. Remember identification information,
The detection means searches the printing surface identification information from the storage means using the read marker information as a search key,
The information providing system according to claim 1, wherein the application unit acquires content information using the searched printing surface identification information as a search key. The storage means stores a reference value for the size of the marker image,
The said detection means detects the size of the said marker image in the said captured image, and calculates | requires the ratio of the size of this detected marker image with respect to the said reference value. Information providing system described in 1. The detection means detects a direction of the marker image in the captured image,
5. The information providing system according to claim 1, wherein the application unit displays the content information on the display unit in accordance with the direction of the detected marker image. 6.   6. The information providing system according to claim 1, wherein the content information includes alternative image information that replaces all or part of the visible information.   The information providing system according to claim 1, wherein the content information includes animation image information for displaying an animation related to an element constituting the visible information.   The information providing system according to claim 7, wherein the application unit moves the content information displayed on the display unit. The printed matter according to claim 1 or claim 3,
The printed matter, wherein the plurality of markers are printed with invisible ink.   The printed matter according to claim 9, wherein the invisible ink is an ink that attenuates reflection of invisible wavelength rays on the printing surface.   The printed matter according to claim 9 or 10, wherein directions of the plurality of markers are not uniform.

Images