JP2004302768A - Input/output device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an input/output device which easily outputs the sound and image, etc., of a musical instrument or the like at a low cost. <P>SOLUTION: A sheet 21 where a keyboard 51 is drawn is placed on a sheet mounting base 11 where a virtual screen S is set, the sheet 21 is photographed in an image pickup part 13, a coordinate area on the virtual screen S on a sheet surface is generated for each key 51a and the sound of a frequency corresponding to the key 51a is related to each coordinate area. The state of indicating the key 51a of the keyboard 51 by a light emitting part 41 is photographed in the image pickup part 13 and the sound related to the coordinate area on the virtual screen S indicated by the light emitting part 41 is generated as the sound corresponding to the key 51a. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an input / output device capable of recognizing input information such as images and outputting desired information such as images and sounds easily.
[0002]
[Prior art]
The applicant has taken a document, and in a presentation device for presentation that displays the photographed image on a display device such as a screen, generates a virtual screen on a placement table on which the material is placed or a monitor for a presenter. Japanese Patent Application No. 2002-216470 and Japanese Patent Application No. 2003-52384 have proposed a technique of pointing and displaying a point image on a display device by instructing light emission on a virtual screen with a light pen. The virtual screen in this case is a two-dimensional space in which XY coordinates are generated by image processing. The light emission of the light pen on the virtual screen is captured by the image capturing unit, and the control unit that captures the captured image provides a virtual screen. The light emission position on the screen is obtained as coordinates, and a predetermined point image is displayed at the coordinates of the screen image corresponding to the coordinates. According to this method, there is no danger like a laser pointer that projects a laser beam, and it is possible to safely point a displayed image.
[0003]
[Problems to be solved by the invention]
By the way, the present applicant has applied a method of outputting not only a point image as information to be output but also various information, for example, a sound such as a sound of a musical instrument, by applying the pointing method to the virtual screen as described above. We studied diligently. In the background, for example, to play or practice an instrument, it is necessary to use a real instrument, and in recent years, musical scores have been converted to electronic scores. Uses a dedicated input device with a keyboard. However, when an instrument is expensive or large, such as a piano or a drum, it is difficult to play the real thing. In addition, in the case of a child, for example, playing a piano may be difficult to play physically or physically, such as a key having a large width or a weak pressing force. Further, in order to create an electronic musical score, a dedicated input device is required as described above. As described above, it has sometimes been difficult to play a real musical instrument or create an electronic musical score in the past, and therefore, it has been desired to develop a device that can easily realize such a desire. .
[0004]
Therefore, an object of the present invention is to provide an input / output device capable of outputting voices and images of musical instruments and the like easily and cheaply.
[0005]
[Means for Solving the Problems]
According to the present invention, a sheet holding an output information specifying image for specifying output information, a sheet mounting table on which the sheet is mounted, and a virtual screen set on the surface thereof, and a sheet mounting table mounted on the sheet mounting table are provided. An image capturing unit that captures the placed sheet, a memory that stores output information, and a control unit that receives an image captured by the image capturing unit; and an output unit that outputs output information. Comparing the image captured by the imaging unit with the coordinates of the virtual screen and calculating the coordinates of the pointing mark on the virtual screen, and when the coordinates are in a coordinate area corresponding to the output information specifying image, the output information specification is performed. It is characterized in that output information corresponding to an image is taken out of a memory and transmitted to an output unit. The output information of the present invention includes an image and a sound.
[0006]
According to the present invention, by placing a sheet on the sheet placement table, the virtual screen overlaps the surface of the sheet, and when the output information designation image held on the sheet is designated by the designation mark, the virtual screen corresponds to the output information designation image. Output information is output from the output unit. The output information designation image may be held in a certain area with respect to the sheet, and the output information may be output when an instruction mark is designated in that area. For example, it is possible to take a picture of a sheet on which a piano keyboard is written by an image pickup unit and play a piano sound in accordance with the key by pointing the key with an instruction mark as if playing. . Therefore, it is possible to easily perform and practice the piano without a real piano. Also, if the width of the key written on the sheet is made small so that even a child can easily finger, it is easy for a child to play, and in that case, a sound can be generated regardless of the key pressing force. Furthermore, if the sound to be output is stored as digital data, an electronic musical score can be easily created.
[0007]
The present invention includes a form in which an identification mark for identifying the type of output information is given to a sheet, and the control unit identifies the identification mark, extracts output information corresponding to the identification mark from the memory, and outputs the output information. Examples of the identification mark include an information form such as a barcode. As a specific example, when a sheet for a piano or a drum is prepared, the sound of the musical instrument must be output, so that the sheet is output with a barcode or the like so that the sound corresponding to the instrument, that is, the sheet is output. Is determined, and the output sound is made to correspond to the musical instrument.
[0008]
Further, according to the present invention, an optical filter capable of photographing only light of a specific wavelength is provided in the photographing optical path of the imaging unit so as to be detachable, and the indication mark is a light emitting unit that emits light including the specific wavelength. Including forms. In this embodiment, the optical filter is removed when the sheet holding the output information designation image is photographed, and the optical filter is inserted when the output information designation image is designated by the designation mark. Accordingly, when the output information designation image is designated by the designation mark, the imaging unit photographs only the light emitting unit, so that the designation mark can be reliably identified, and a stable output operation can be obtained.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows an input / output device 1 according to one embodiment, and FIG. 2 is a block diagram systematically showing the configuration and functions of the device 1. As shown in FIG. 1, the input / output device 1 has a sheet placing table 11 on which an input sheet 21 is placed on an upper surface, and is supported by the sheet placing table 11 via an arm 12 and placed on the sheet placing table 11. An imaging unit 13 for photographing the sheet 21; On the surface of the sheet 21, a predetermined image for outputting output information such as an image and a sound is held as an output information designation image by means of writing, printing, pasting, or the like. An input start switch 14 is provided at an appropriate position on the sheet mounting table 11.
[0010]
As shown in FIG. 2, the imaging unit 13 includes a camera 13a and an optical filter 13b. The camera 13a is supported by the arm 12 so that its optical axis is orthogonal to the sheet mounting table 11. The optical filter 13b is a filter capable of photographing only infrared light that is invisible light, and is set to be freely inserted into and removed from the photographing optical path of the camera 13a. In this case, insertion and removal of the optical filter 13b are manually performed. The image captured by the imaging unit 13 is captured by the control unit 15, processed, and output to the display device (output unit) 31. The display device 31 is a projector that projects an image on a screen, a display such as a CRT or a liquid crystal, or a sound generator such as a headphone or a speaker. The control unit 15 is built in, for example, the sheet mounting table 11.
[0011]
The control unit 15 generates the XY coordinates of the virtual screen S on the surface of the sheet mounting table 11 photographed by the imaging unit 13 by a general image processing method. As a method of generating the virtual screen S, infrared light is emitted at four corners of the sheet mounting table 11, and these four emission points are photographed by the imaging unit 13 in a state where the optical filter 13b is inserted into the photographing optical path of the camera 13a. Then, there is a method of recognizing the position of each light emitting point from the image data.
[0012]
The control unit 15 has a memory that stores a large number of output information corresponding to the output information designation image held on the sheet 21 (an output data storage unit in the control unit 15 in FIG. 2). When the sheet 21 is photographed by the imaging unit 13, the output information designation image is captured by the control unit 15, and the control unit 15 generates a region of the output information designation image on the virtual screen S as a coordinate region. Then, the control unit 15 stores the coordinate area of the output information designation image and the output information in association with each other. An instruction mark 41 that emits light including infrared light transmitted through the optical filter 13b is given to the output information designation image of the sheet 21 to give an instruction, and this state is inserted into the optical path of the camera 13a with the optical filter 13b inserted. When an image is captured by the imaging unit 13 in the state, the control unit 15 calculates the position of the instruction mark 41 on the virtual screen S to generate coordinates, and generates output information associated with the coordinate area where the coordinates exist. The output information is output to the display device 31 when the output information is an image, and is output from a speaker or the like when the output information is audio.
[0013]
As the instruction mark 41, a light emitting unit provided in the light pen 42 and the light emitting finger sack 43 shown in FIG. 1 is preferable. The light pen 42 has a light emitting unit 41 at the tip, and when pressed, the switch is turned on, and the light emitting unit 41 emits light. The light emitting finger sack 43 is used by fitting a finger. As shown in FIG. 3, an outer (claw side) light emitting section 41 and an inner (finger pad side) switch 44 are provided at a tip portion of the light emitting finger sack 43, respectively. It is provided in a state, and a battery 45 which is a power source for light emission is incorporated in a central portion.
[0014]
If the output information is the sound of a musical instrument, it is necessary to output a sound corresponding to the type of the musical instrument. Therefore, the control unit 15 also stores a tone color corresponding to the type of the musical instrument. On the other hand, a bar code 22 for discriminating the type of musical instrument is held at a predetermined position (upper left in FIG. 1) of the sheet 21 by means such as printing or sticking a sticker. The barcode 22 is captured by the control unit 15 as image data, and the control unit 15 recognizes the unique information held by the barcode 22 and outputs a sound in the tone. The barcode 22 is recognized by, for example, examining a minus direction (left direction) and a plus direction (right direction) in the X direction of the display area of the barcode 22, converting the barcode 22 into alphanumeric characters based on the contrast of the image, A method of recognizing data specific information based on the information can be adopted. Instead of automatically discriminating the type of sound using the barcode 22, the user may select the type of musical instrument by using a switch provided on the sheet mounting table 11 or the like.
[0015]
Next, a specific example of the operation of the input / output device 1 will be described by taking a case of playing a keyboard instrument as an example. As shown in FIG. 1, a sheet 21 on which a picture of a keyboard 51 is drawn as an output information designation image is placed on the sheet mounting table 11. On the upper left of the sheet 21, a bar code 22 for discriminating the tone color of the musical instrument as "piano" is printed. A reference position corresponding to the virtual screen S generated in advance as described above may be provided on the upper surface of the sheet mounting table 11, and the sheet 21 may be placed at this reference position.
[0016]
FIG. 4 shows a luminescent glove 46 which is particularly suitable for playing keyboard instruments. A light emitting unit 41 and a switch 44 are provided in a pair at the tip of each finger of the light emitting glove 46, similarly to the light emitting finger sack 43, and a battery 45 is incorporated near the opening. . The user puts on the light-emitting gloves 46 and fingers the keyboard 51 drawn on the sheet 21 so as to play the piano. When each key 51a of the keyboard 51 is touched, the switch is turned on, and the light emitting unit 41 emits light.
[0017]
In order to perform using the input / output device 1, the input start switch 14 of the sheet placing table 11 is turned on, and the input operation shown in FIG. 5 is started. As shown in the figure, first, the user removes the optical filter 13b of the imaging unit 13 from the photographing optical path of the camera 13a (step S101). The sheet 21 is photographed by the imaging unit 13, and the data of the photographed image is taken into the control unit 15 (step S102). The control unit 15 searches the display area of the determined bar code 22 from the image data, reads the bar code 22, recognizes the unique information, and determines the type of the instrument, that is, the generated sound to be a piano (step S103). ). Next, the control unit 15 generates the position (area) of each key 51a on the virtual screen S as a coordinate area, and associates the position of the key 51a with sound information, that is, sound frequencies such as do, re, mi,. Is stored (step S104).
[0018]
Next, the user inserts the optical filter 13b into the photographing optical path of the camera 13a (Step S105). The image captured by the imaging unit 13 is captured by the control unit 15 (step S106), and from this state, the user presses the key 51a drawn on the sheet 21 with the finger of the hand wearing the light-emitting glove 46 to play the piano. Touch to indicate. By touching the key 51a in this manner, the switch 44 is turned on, and the light emitting unit 41 emits light. The captured image captured by the control unit 15 is only the light emitting unit 41 due to the insertion of the optical filter 13b, and the control unit 15 determines whether or not the light emitting unit 41 has been captured (step S107). ). Then, when recognizing the light emitting unit 41, the control unit 15 calculates its position on the virtual screen S to generate coordinates (step S108), and then stores the coordinates in the coordinate area of the generated key 51a. It is determined whether or not the key is applicable (step S109). If the coordinate is applicable, a sound associated with the key is generated (step S110). If No is determined in steps S107 and S109, the process returns to step S107 and the above operation is repeated.
[0019]
As described above, according to the input / output device 1 of the present embodiment, when the key 51a of the sheet 21 is touched with the finger wearing the light-emitting glove 46 and the light-emitting portion 41 emits light, the sound of the piano is generated and the performance is performed. And can practice. Therefore, it is possible to easily perform and practice the piano without a real piano. Further, if the width of the key 51a drawn on the sheet 21 is reduced, the performance by the child becomes easy, and in this case, a sound can be generated regardless of the pressing force of the key 51a. Further, if the sounds to be output are stored as digital data, an electronic musical score can be easily created. Further, by instructing the key 51a of the sheet 21 according to the composed music and restoring the key to the electronic musical score, it is possible to confirm whether or not the musical score has been correctly described.
[0020]
In the above example, the present invention is applied to the performance of a keyboard instrument such as a piano. However, the present invention is not limited to keyboard instruments, but can be applied to musical instruments such as xylophones and drums. FIG. 6 shows an example of a drum performance sheet 23. In this sheet 23, elliptical regions 52a, 52b, and 52c indicating low, middle, and high sounds of the drum are respectively provided as output information designation images. , Are drawn with a predetermined size. At a predetermined position (upper right in this case) of the sheet 23, a bar code 24 for discriminating the timbre of the musical instrument from "taiko" is printed. The light pen 42 and the light-emitting finger sack 43 shown in FIG. 1 are suitable for instructing the areas 52a, 52b, and 52c of the sheet 23 to cause the control unit 15 to recognize the positions.
[0021]
When the drum sheet 23 is used, similarly to the keyboard instrument sheet 21, the drum sheet 23 is placed on the sheet mounting table 11 of the input / output device 1, and an image is taken by the imaging unit 13 with the optical filter 13 b removed from the imaging optical path. The control unit 15 generates the regions 52a, 52b, and 52c as coordinate regions on the virtual screen S, and stores the regions 52a, 52b, and 52c in association with the low, medium, and high sounds of the drum. . Then, in the imaging section 13 in which the optical filter 13b is inserted into the photographing optical path, light emission is instructed by the light pen 42 or the light-emitting finger sack 43 in each area 52a, 52b, 52c of the sheet 23, so that the instructed areas 52a, 52b, A drum sound corresponding to 52c is output.
[0022]
By the way, in the present invention, the sounds of various musical instruments can be output using the sheet as described above, but in the present invention, in addition to the sounds of the musical instruments, images and sounds corresponding to the output information designation image can also be output. Can be output. The following is an example of such a case.
[0023]
FIG. 7 shows a sheet used as a teaching material for infant education, for example, which outputs an image of an animal and a cry. Various animals are drawn on the sheet 25, rectangular areas 53a, 53b, 53c and 53d are defined for each animal, and a bar code 26 for identifying the animal sheet 25 is printed at a predetermined position. ing. On the other hand, the control unit 15 of the input / output device 1 stores an image (still image or moving image) of the animal drawn on the sheet 25 and a squeal, and stores the unique information held by the barcode 26 in the animal sheet 25. Is stored so as to be recognized.
[0024]
According to this sheet, when set on the input / output device 1 and instructing the animal areas 53a, 53b, 53c, 53d of the sheet 25 using the light pen 42 and the light-emitting finger sack 43 as described above, the designated animal Simultaneously with the output of the image, the animal's cry can be generated. The sheet 25 is useful as a teaching material for displaying the movement and ecology of an animal and generating a cry, thereby educating a user such as an infant or the like about the characteristics of the animal or raising interest.
[0025]
FIG. 8 shows a sheet 27 used as a Japanese language teaching material for infant education or for foreigners, for example, for uttering the Japanese syllabary. In this sheet 27, the characters of the Japanese syllabary are sequentially printed in areas 54 divided in a lattice shape, and a bar code 28 for discriminating as the Japanese syllabary sheet 27 is printed at a predetermined position. On the other hand, the control unit 15 of the input / output device 1 stores the utterance of each character of the Japanese syllabary so that the unique information held by the barcode 28 is recognized as the Japanese syllabary sheet 27. Have been.
[0026]
According to the sheet 27, when the user sets the sheet on the input / output device 1 and designates one area 54 using the light pen 42 or the light-emitting finger sack 43 as described above, characters in the designated area can be uttered. . The sheet 27 is useful as a teaching material for educating users such as infants and foreigners how to read Japanese syllabary characters. Here, Japanese characters are targeted, but the present invention can be applied to numbers, and further, it can be applied to foreign characters and numbers such as alphabets. In addition, for educational use, a form suitable for self-study is used, such as using a sheet on which answers to questions and alternatives are written, and indicating the position of the selected answer by notifying the correct answer or not by images and sounds. Can be developed.
[0027]
【The invention's effect】
As described above, according to the present invention, when an output information designation image held on a sheet is designated, output information corresponding to the output information designation image is output. This has the effect of being able to output.
[Brief description of the drawings]
FIG. 1 is a perspective view of an input / output device according to an embodiment of the present invention.
FIG. 2 is a block diagram systematically showing the configuration and functions of the input / output device according to the embodiment.
FIG. 3 is a perspective view of a light-emitting glove suitable for indicating a sheet for a keyboard instrument.
FIG. 4 is a perspective view showing a light-emitting finger cot;
FIG. 5 is a flowchart illustrating an operation example of the input / output device.
FIG. 6 is a diagram showing a drum playing sheet.
FIG. 7 is a diagram illustrating an example of an educational sheet.
FIG. 8 is a diagram showing another example of the educational sheet.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... I / O device 11 ... Sheet mounting table 13 ... Imaging part 13b ... Optical filter 15 ... Control parts 21, 23, 25, 27 ... Sheets 22, 24, 26, 28 ... Barcode 31 ... Display device (output part)
41 ... Light-emitting part (instruction mark)
51 ... keyboard (output information designation image)
52a-52c, 53a-53d, 54 ... area (output information designation image)
S ... virtual screen

Claims (5)

A sheet holding an output information specification image for specifying output information,
A sheet placing table on which the sheet is placed and a virtual screen is set on the surface thereof;
An imaging unit for photographing a sheet placed on the sheet placing table;
A control unit having a memory for storing the output information, and to which a captured image by the imaging unit is input;
An output unit that outputs the output information,
The control unit compares the image captured by the imaging unit with the coordinates of the virtual screen, calculates the coordinates of the instruction mark on the virtual screen, and stores the coordinates in a coordinate area corresponding to the output information designation image. An input / output device for extracting output information corresponding to the output information designation image from the memory and transmitting the output information to the output unit. The sheet is provided with an identification mark for identifying a type of the output information, the control unit identifies the identification mark, and extracts output information corresponding to the identification mark from the memory. Item 1. The input / output device according to Item 1. The input / output device according to claim 1, wherein the output information is an image or a sound. The input / output device according to claim 2, wherein the identification mark is in an information form such as a barcode. The imaging unit is provided with an optical filter that allows only a specific wavelength of light to be captured with respect to the imaging optical path, and is detachably provided. The indication mark is a light emitting unit that emits light including the specific wavelength. The input / output device according to claim 1, wherein:

Images