JP2003085571A - Coloring book toy - Google Patents

Abstract

(57) [Summary] [Problem] To provide a coloring toy that can enhance the fun by experiencing the real sense that a player paints a color in a virtual environment. SOLUTION: A coloring toy includes an image pickup means 3 for picking up an image of a player and outputting moving image data, and an action recognition means 19 for recognizing a player's movement as operation information based on the moving image data.
CG moving image creation means 32 for creating CG moving image data by computer graphics (CG) reflecting the player's movement based on the movement information, and symbol image output that outputs the symbol image data including the symbol that is the basis of the coloring book Means 13;
The image combining means 34 for combining the CG moving image data with the design image data and the display control means 35 for outputting the combined image data to the display device 5 and displaying the combined image are provided. That is, the coloring is performed by a CG moving image reflecting the player's action.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a coloring toy, and in particular, it is possible to apply a color to a design displayed on a display device by using a computer graphic (CG) moving image that reflects the motion of a player. It concerns coloring toys.

[0002]

2. Description of the Related Art Heretofore, as a coloring toy using a computer, there has been known a coloring toy in which the inside of a design displayed on a display device can be filled with an arbitrary color. Specifically, using a pointing device such as a mouse or touch pen, a desired color is selected from the color sample, and then, inside the pattern displayed on the screen, that is, a desired area surrounded by a contour line. When is selected, the area is filled with the selected color.

[0003] Further, for the design displayed on the screen,
There is also known a coloring toy in which linear coloring can be performed along the trajectory of the pointing device by operating the pointing device.

[0004]

However, in these coloring toys, the player holds the pointing device by hand and paints the color while selecting a desired color for each area, which is not interesting. Was there. In other words, since a simple operation of sequentially applying colors to the still images displayed on the display device is repeated, the display screen and the operation contents can be changed by the player even though the computer is used. Could not be satisfied.

Further, in the conventional coloring toys, since the applied color is not evaluated, the joy of applying the correct color cannot be given to the player, and even if the application is completed to the end. Since there is no change, it was not possible to improve the game feeling.

Therefore, in view of the above situation, the present invention makes it possible to enhance the fun by making the player feel the reality of painting a color in a virtual environment.
Moreover, the present invention provides a toy that can improve the game feeling by evaluating the selected color.

[0007]

According to another aspect of the present invention, there is provided a coloring toy which includes an image pickup means for picking up an image of a player and outputting moving image data, and the game based on the moving image data obtained by the image pickup means. The motion recognition means for recognizing the motion of the player as motion information, and the motion information obtained by the motion recognition means are input, and based on the motion information, computer graphics CG moving image data reflecting the motion of the player is input. CG moving image creating means to be created, pattern image output means for outputting the pattern image data including the design which is the basis of the coloring picture, and image composition for synthesizing the CG moving image data with the pattern image data given by the pattern image outputting means. And display control means for outputting the composite image data synthesized by the image synthesizing means to the display means and displaying the synthesized image. A. In particular, the pattern image output means is a color selection means for selecting a color to be colored from a plurality of color samples on the basis of the CG moving image data, and a color is selected from the symbols on the basis of the CG moving image data. It has area selection means for selecting an area to be formed, and coloring control means for filling the selected area with a selected color.

Therefore, according to the coloring toy of the first aspect of the invention, the image of the player is picked up by the image pickup means and the moving image data is output. The motion recognition means recognizes the motion of the player as motion information based on the output moving image data.
The CG moving image creating means inputs the motion information obtained by the motion recognizing means and creates a CG moving image reflecting the motion. The CG moving image data is created, for example, by performing deformation processing such as movement or rotation on component data such as an object registered in advance.

On the other hand, when the design image data is given from the design image output means, the CG moving image data is synthesized with the design image data by the image synthesis means, and the synthesized image is displayed on the display means. As a result, a moving image that reflects the player's own motion can be viewed on the display device, and a virtual environment is realized. In this environment, a desired color is selected from a plurality of colors and a region to be colored is set based on the CG moving image data. Then, when the color and the area are selected, the coloring control unit fills the selected area with the selected color. That is, the color of the pixel included in the area is changed to the selected color.

A coloring toy according to a second aspect of the present invention is the coloring toy according to the first aspect, further comprising a player image output means for outputting the moving image data obtained by the image pickup means as player image data. The image synthesizing means further synthesizes the player image data with the symbol image data.

Therefore, according to the coloring toy of the second aspect of the invention, in addition to the effect of the first aspect of the invention, when the player image data is output from the player image output means, the player image data is converted into a pattern image. The data is combined and the combined image is displayed on the display means. As a result, it becomes possible to see the image of the player himself / herself by the display means, and it is possible to give the player a feeling that he / she is active by entering the virtual environment.

The coloring toy according to the invention of claim 3 is the coloring toy according to claim 1 or 2, wherein the CG moving image data created by the CG moving image creating means comprises moving image data representing a brush. The movement of the player's hand is reflected.

Therefore, according to the coloring toy of the third aspect of the invention, in addition to the action of the first or second aspect of the invention, a feeling that the brush is moving according to the movement of the player's hand is given. It will be possible. In particular, by displaying the image of the player himself on the display means, it becomes possible to display the brush created by the CG moving image creating means while being held in the player's hand. That is, it is possible to experience a more realistic feeling.

[0014] The coloring toy according to the invention of claim 4 is the coloring toy according to any one of claims 1 to 3, wherein a color as an example corresponding to each area of the design is used as color information. Color information storage means stored in advance, color matching means for matching the filled color with the color information when the selected area is filled, and based on the matching result, the image combining means An image changing means for changing the pattern image data is further provided.

Therefore, according to the coloring toy of the invention of claim 4, in addition to the effect of any one of the inventions of claims 1 to 3, the color collating means, when the selected area is filled, The filled color (selected color) is collated with the color information (model color). Then, the image changing means changes the pattern image data based on the matching result. Note that it may be changed only when the two colors match (when the selected color is correct), and may be changed only when the two colors do not match (when the selected color is wrong). Alternatively, it may be changed in different modes depending on the matching result (correctness).

The coloring toy according to the invention of claim 5 is the coloring toy according to claim 4, further comprising voice output means for outputting a preset voice based on the collation result of the color collating means. It is a thing.

Therefore, according to the coloring toy of the fifth aspect of the invention, in addition to the effect of the fourth aspect of the invention, a preset voice is output according to the collation result. Also in this case, the sound may be output only in the case of one of right and wrong, or different sounds may be output.

A coloring toy according to a sixth aspect of the present invention is the coloring toy according to any one of the first to fifth aspects, in which the moving image data obtained by the imaging means is input and a predetermined time interval is entered. Frame separating means for separating the frame image, background image storing means for previously capturing only the background by the image capturing means and storing as background image data, area dividing means for dividing the frame image into a plurality of small areas, The background image data is read from the background image storage means, it is determined by using the background image data whether or not a characteristic region representing the player is included in each of the small regions, and based on the determination result. A conversion means for converting the frame image into numerical sequence data, and knowledge for applying the numerical sequence data to a learnable neural network to generate a pattern code Anda management unit, said operation recognizing means is for recognizing the contents of the operation based on the pattern codes obtained by the knowledge processing means.

Therefore, according to the coloring toy of the invention of claim 6, in addition to the effect of the invention of any one of claims 1 to 5, the moving image data imaged by the image pickup means is framed by the frame separation means. Separated into images. Then, the frame image is divided into a plurality of small areas by the area dividing means, and subsequent processing of the frame image is performed with the small areas as one unit.

The converting means determines whether or not each of the divided small areas includes a characteristic area, and gives a numerical value based on the result of the determination. For example, if it is determined that each small area includes a characteristic area, "1" is output, and if it is determined that the characteristic area is not included, "0" is output. It should be noted that the frame image and the background image data stored in the background image storage means may be compared to determine whether or not the characteristic region is included. In this way, when the frame image is converted into the numerical sequence data, the knowledge processing means applies the numerical sequence data to the neural network to generate the pattern code. This neural network is preliminarily learned and optimized so as to output a pattern code corresponding to a factor that is important for recognition processing, such as a player's posture and its movement. Therefore, by analyzing these pattern codes by the motion recognition means, it becomes possible to recognize the contents of the motion of the player.

The coloring toy according to the invention of claim 7 is the coloring toy according to claim 6, wherein the area dividing means reads the background image data from the background image storage means and uses the background image data. And detecting means for detecting the characteristic area in the frame image, and division setting means for dividing into the small areas based on the detection information of the detecting means.

Therefore, according to the coloring toy of the seventh aspect of the invention, in addition to the effect of the sixth aspect of the invention, first, the difference between the frame image and the background image data is obtained by the detecting means, and the frame image is obtained. Feature regions within are detected. Then, in the division setting means, based on the position, size, shape, etc. of the characteristic region in the frame image,
A division pattern of the frame image into small areas is set.
For example, in the characteristic region or in the vicinity thereof, in the region where it is determined that a large movement part such as the player's arm is located, the small size of each small region is reduced in consideration of recognition accuracy. Is set. On the contrary, in a region apart from the characteristic region where it is determined that no player exists, the size of the small regions is set to be larger, and the total number of the small regions in the entire frame image is suppressed.

The coloring toy according to the invention of claim 8 is the coloring toy according to claim 6 or 7, wherein the knowledge processing means inputs the numerical sequence data into the first neural network and outputs it. A first knowledge processing means for generating a first pattern code, a first code storage means for storing the generated first pattern code, a plurality of the first pattern codes read from the first code storage means, and a second Second knowledge processing means for time-series input to a neural network and generating from the output a second pattern code corresponding to one action of the player, and second code for storing the generated second pattern code Storage means, wherein the motion recognition means reads the second pattern code from the second code storage means, and Reading the first pattern code corresponding to the second pattern code from a code storage means, it is to recognize the contents of the operation using these pattern codes.

Therefore, according to the coloring toy of the eighth aspect of the invention, in addition to the operation of the sixth or seventh aspect of the invention, a plurality of knowledge processing means having different neural networks are multi-staged and each Neural networks have been learned and optimized for their respective purposes. Specifically, in the first knowledge processing means having the first neural network, the numerical sequence data obtained by the previous conversion step is input and the first pattern code corresponding to the numerical sequence data, that is, the content of the original frame image is generated. .
Then, in the second knowledge processing means having the second neural network, the plurality of first pattern codes are input in time series, and the second pattern code corresponding to the content of the operation performed between the series of frame images is generated. To be done. Therefore, in the subsequent motion recognition means, a multi-faceted recognition process in which pattern codes having different meanings are combined is performed.

A coloring toy according to a ninth aspect of the present invention is the coloring toy according to the first aspect, in which the frame separating means for inputting the moving image data obtained by the imaging means to separate frame images at predetermined time intervals. And an image storage unit for storing the frame image, a first frame image and a second frame image separated at a predetermined time interval from the image storage unit, and from the first frame image to the second frame image Change amount detecting means for detecting the change amount of each pixel for each pixel, and a moving pixel extracting means for comparing the change amount for each pixel with a threshold value and extracting a pixel with the change amount larger than the threshold value as a moving pixel. And an average coordinate calculating means for averaging the coordinates of the plurality of extracted moving pixels and calculating an average coordinate, and the average coordinate calculating means calculates the average coordinates at predetermined time intervals. A moving direction detecting means for detecting a moving direction of the average coordinate based on the one average coordinate; and the motion recognizing means recognizes the content of the motion based on the average coordinate and the moving direction. It is a thing.

Therefore, according to the coloring toy of the ninth aspect of the invention, in addition to the action of the first aspect of the invention, the moving image data picked up by the image pickup means is separated into frame images by the frame separation means, and the image storage means is provided. Are sequentially stored in. Then, the change amount detecting means stores the first frame image (for example, the previously separated frame image) separated at a predetermined time interval and the second frame image (for example, the currently separated frame image) from the image storage means. Reading is performed and the amount of change in these two images is detected for each pixel. Further, the moving pixel extraction means compares the variation amount for each pixel with a threshold value, a pixel larger than the threshold value is extracted as a moving pixel, and the average coordinate calculation means determines the average coordinates of a plurality of moving pixels. It is calculated. Further, the moving direction detecting means detects the moving direction of the average coordinates based on the two average coordinates calculated at a predetermined time interval. Then, the motion recognition means recognizes the content of the motion of the player based on the average coordinates and the moving direction. That is, by detecting how the calculated average coordinates (center of mass) change, the contents of the action of the player are recognized.

[0027]

BEST MODE FOR CARRYING OUT THE INVENTION A coloring toy according to a first embodiment of the present invention will be described below with reference to FIGS. FIG. 1 is an explanatory view showing a usage state of a coloring toy,
FIG. 2 is a block diagram showing the basic configuration of a coloring toy,
3 is an explanatory diagram showing the concept of region division of a frame image, FIG. 4 is an explanatory diagram showing an example of an input procedure to the first neural network, and FIG. 5 is an example of an input procedure to the second neural network. FIG. 6 is an explanatory diagram showing an example of a display screen of the coloring toy.

As shown in FIG. 1, the coloring toy 1 according to the first embodiment comprises an image pickup means 3, a control device 4, and a display device 5 such as a TV monitor. While looking at the screen 6, the patterns on the screen 6 are colored. Here, the display device 5 corresponds to the display means of the present invention.

The image pickup means 3 comprises a CCD camera or the like,
The player 2 and the background thereof are imaged and the moving image data is sent to the control device 4. The control device 4 is provided integrally with the image pickup means 3 and, as shown in FIG. 2, is composed of an image recognition section 10, a storage section 11, an image processing section 12, a pattern image output section 13, and the like. .

The image recognizing section 10 has, as a functional configuration, a frame separating means 15, an area dividing means 16, and a converting means 1.
7, knowledge processing means 18, and motion recognition means 19. The frame separation means 15 inputs the moving image data from the image pickup means 3 and generates frame images separated at predetermined time intervals. The area dividing means 16 has a detection function 21.
And a division setting function 22. The detection function 21 is for detecting the characteristic region representing the player 2 from the frame image, and the division setting function 22 is for dividing the frame image into a plurality of small regions based on the detection information of the characteristic region. Here, the detection function 21 corresponds to the detection means of the present invention, and the division setting function 22 corresponds to the division setting means of the present invention.

The converting means 17 determines whether or not each small region includes a characteristic region, and based on the determination result, converts the frame image data into numerical sequence data. The knowledge processing means 18 generates a pattern code from the numerical sequence data by a neural network, and the first neural network 24 and the second neural network 25.
It has a two-stage configuration. The motion recognition means 19 recognizes the content of the motion of the player 2 based on the pattern code.

Each means of the image recognition unit 10 is a C (not shown).
It can be realized by devices such as PU, RAM, and ROM. The CPU controls the processing of each of the above means, and the RAM
Is used as a work memory necessary for the operation of the CPU and a buffer memory for various data to be processed, and the ROM is the CP
Used to store the instruction program needed for U operation.

The storage means 11 includes a background image storage unit 27 and a code storage unit 28. Background image storage unit 27
Stores the background image data obtained by the image pickup means 3. The code storage unit 28 has a first code storage area 29 and a second code storage area 30, and stores the pattern code generated by the knowledge processing means 18.

The image processing unit 12 uses the CG moving image creating means 3
2. The player image output means 33, the image composition means 34, the display control means 35, and the image variation means 36 are provided.
The CG moving image creating means 32 creates CG moving image data that reflects the content of the action from the recognition data obtained by the action recognizing means 19. The player image output means 33 extracts only the image data of the player 2 excluding the background from the frame image in the frame separation means 15 and outputs it as the player image data. The image synthesizing means 34 is the symbol image output means 1
The CG moving image data by the CG moving image creating means 32 and the player image data by the player image outputting means 33 are combined with the symbol image data supplied by 3. The display control unit 35 causes the display device 5 to display the combined image obtained by the image combining unit 34. The image fluctuating means 36 fluctuates the symbol obtained by the symbol image outputting means 13 based on a predetermined condition. That is, the design is displayed as a moving image. Note that each unit of the image processing unit 12 can be realized by a device such as a CPU, a RAM, and a ROM (not shown), like the image recognition unit 10.

The design image output means 13 outputs design data relating to the design which is the basis of the coloring picture. The design data includes a contour line forming the design, a division of a colored area surrounded by the contour line, and the like. Further, in the pattern image output means 13, the CG moving image data created by the CG moving image creating means 32, that is, the motion recognition means 19
Color selecting means 38 for selecting one color from a plurality of colors on the basis of the motion information recognized by, and area selecting means 39 for selecting an area to be colored from the design based on the CG moving image data. Is equipped with. Further, the coloring control means 40 is provided to set the color of the pixels included in the selected area to the selected color so that the selected area is filled with the selected color.

Further, in the control device 4, the color selected by the color selecting means 38, that is, the color applied to a predetermined area of the design, and the color information stored in the color information storing means 43 (the model in each area) Color matching means 4 for matching
2 is provided. That is, the color matching unit 42 determines whether or not the selected color matches the model color. Then, as a result of collation, if they match (in the case of correct answer), a signal is transmitted to the image changing means 36,
The pattern is changed by the image changing means 36. If they do not match (incorrect answer), the voice output means 44
A signal is sent to the user to inform him / her of the error by voice.

Next, detailed contents of the processing in the main part of the control device 4 will be described. In the coloring toy 1, first, before the recognition process of the player 2, only the background excluding the player 2 is imaged by the imaging unit 3, and the image obtained by frame-separating the image is used as background image data, and the background image is stored in the storage unit 11. It is stored in the unit 27. This background image data serves as a monochrome background in the conventional chroma key method. In other words, by performing this procedure, the recognition process can be performed even under special conditions in which a monochrome background is installed.

The frame separation means 15 separates the captured moving image data into frames and outputs frame images at predetermined time intervals. The frame image is input to the area dividing unit 16 and divided into a plurality of small areas. For example, as shown in FIG. 3 (a), a player 2 standing upright facing the front.
The frame image 46 is divided into a total of 11 small areas so as to surround the area. When such a division pattern is applied, when the player 2 performs an arm raising motion, a part of the arm appears in the small areas S1 to S4. Therefore, by paying attention to the presence or absence of the characteristic regions in these small regions S1 to S4, it becomes possible to know to what extent the player 2 is raising his or her arms.

However, since the physical characteristics such as the height of the player 2 are different from each other and the distance between the image pickup means 3 and the player 2 is different, for example, as shown in FIG. If the same division pattern is applied to different players 2b,
The head overlaps the small areas S2 and S3, which may hinder the detection of the target arm position. Therefore, as shown in FIG. 3C, it is desirable to change the division pattern of the small area according to the player 2b.

Therefore, in the area dividing means 16, first, the detecting function 21 detects the characteristic area representing the player 2 in the frame image 46. Specifically, the characteristic region can be detected by reading the background image data from the background image storage unit 27 and obtaining the difference from the frame image 46 in pixel units or block units. Then, based on the information such as the shape and size of the detected characteristic region, the division setting function 22 dynamically sets the division pattern of the frame image 46 and divides it into a plurality of small regions. For example, the height of the detected characteristic region is regarded as the height of the player 2, and a division pattern that matches the height is applied. By doing so, it is possible to deal with different physical characteristics and distances to the image pickup means 3.

Note that FIG. 3 merely shows an example of the division pattern, and the number, shape, size, etc. thereof can be set arbitrarily. For example, in order to detect the movement of the whole body of the player 2 in more detail, it goes without saying that finer area division is necessary. In addition, when starting the recognition process, it is preferable to perform initial setting by, for example, standing upright in an upright posture and capturing an image of the player 2, and to accurately detect accurate data such as height.
Thereby, the frame image 4 in the area dividing unit 16
The division pattern of No. 6 can be set more accurately.

The conversion means 17 determines whether or not the frame-divided frame image 46 includes a characteristic region for each of the divided small regions, and gives a numerical value based on the result of the determination. For example, as shown in FIG. 4, "1" is output when it is determined that each of the small regions S1 to S6 includes a characteristic region, and "0" is output when it is determined that the characteristic region is not included. . Specifically, each small area S1
S6 is compared with the corresponding area of the background image data stored in the background image storage unit 27 on a pixel-by-pixel or block-by-pixel basis, and the characteristic area, that is, the number of pixels or blocks that are not the background image is set in advance. If the threshold value is exceeded, “1” is output, and in other cases, “0” is output. These numerical values are arranged in a predetermined order, for example, in the order of S1, S2, ..., As numerical value sequence data (000011).

Then, the numerical sequence data is applied to the neural network of the knowledge processing means 18 to generate a pattern code. The neural network is composed of two stages of a first neural network 24 and a second neural network 25. As shown in FIG. 4, when the numerical sequence data (000011) is input to the input layer 24a of the first neural network 24, the first pattern code (d1) is output from the output layer 24c. The first pattern code is stored in the first code storage area 29, read out as needed, and applied to the subsequent processing.

As shown in FIG. 5, in the input layer 25a of the second neural network 25, the first pattern code (d) generated by the first neural network 24 is generated.
1, d2, ..., Di) are the base frame images (f
, F2, ..., Fi) are input in time series. Then, the second pattern code (ri) is output from the output layer 25c. This second pattern code (ri) is f1
The data is stored in the second code storage area 30 as a content corresponding to the content of the operation performed between the series of frames from to fi.

The above neural networks 2
4, 25 are preliminarily learned and optimized so as to output pattern codes corresponding to factors that are important for the subsequent recognition processing, such as the posture and movement of the player 2. Learning of a neural network is generally performed by a backpropagation method. This is to update the weighting coefficient inside the neural network based on the input / output data and the teacher signal. The recognition processing by the learned neural network can be performed at high speed, and the recognition result can be output instantaneously even when handling a huge amount of data. Further, even when an unlearned pattern is input, it is possible to obtain an almost accurate recognition result by the generalization action of the neural network.

In the motion recognition means 19, the code storage unit 28
By reading and analyzing the pattern code stored in, the content of the action performed by the player 2 is recognized.
Specifically, the second code storage area 30 of the code storage unit 28
The second pattern code is read out, the first pattern code which is the basis of the second pattern code is read out from the first code storage area 29, and both pattern codes are used. In this way, by combining two types of pattern codes having different meanings, it becomes possible to perform multifaceted recognition processing, and it is possible to cope with more complicated operations. Further, the recognition accuracy of the motion is further improved by performing the analysis process repeatedly while gradually changing the conditions such as the type and combination of the pattern code used for the analysis and selecting the optimum result from the information obtained sequentially. be able to.

In the image processing unit 12, the image recognition unit 1 described above is used.
By applying the recognition information based on 0, an image having a novel effect can be obtained. That is, the CG moving image creating means 32 and the player image output means 33 provide the image of the player 2 itself and the image data reflecting the recognition result of the motion of the player 2 by the symbol image output means 13. By synthesizing with the pattern image data and displaying it on the display device 5, it is possible to create a virtual environment in which a sense of reality can be experienced.

Specifically, as shown in FIG. 6 (a), on the screen 6, a plurality of symbols 48 showing a dinosaur, a background and the like are displayed at the end of the composite screen, and the player image is output in front of it. The three images are combined so that the image of the player 2 output by the means 33 is displayed and the brush 50 created by the CG moving image creating means 32 is displayed in front of the image, and the pseudo three-dimensional image is displayed. Has been realized. In particular, the brush 50 is set so that the pattern portions thereof are displayed so as to overlap each other at the position of the player 2's right hand, and the movement of the brush 50 reflects the action of the right hand. That is, when the player 2 moves the right hand, the position of the brush 50 and the direction of the brush 50 change corresponding to the movement.

In the lower part of the screen 6, a color sample area 4 in which a plurality of color samples reminiscent of a palette of paints are displayed.
9 is provided. Then, as shown in FIG. 6B, when the hair tip of the brush 50 matches any one of the color samples in the color sample region 49 by the operation of the player 2, the color selection of the pattern image output means 13 is performed. The color is selected by means 38 as the color to be colored. At this time, the tip of the brush 50 is displayed in the selected color. After that, as shown in FIG. 6C, the tip of the brush 50 is moved by the movement of the player 2.
When the left hand of the player 2 operates so as to match the desired area 48a of the design 48 and to press the decision button 50a provided on the head of the brush 50 in that state (not shown in FIG. 6C). ), Area selection means 39 of the pattern image output means 13
The area 48a is selected as an area to be colored by. Then, the coloring control means 40 applies the selected color to the selected area 48a.

At this time, the color matching means 42 determines whether or not the applied color is correct, and if the answer is correct, the image varying means 36 causes the area 48a (dinosaur) to start moving. On the other hand, in the case of an incorrect answer, the voice output means 44 outputs a voice indicating that it is wrong.

After that, similarly for other areas,
By using the brush 50, the selection of the color and the selection of the region are repeated in order, so that all the patterns 48 displayed on the screen 6 are displayed.
Is filled. Then, when all the correct colors are applied to the plurality of areas, all the symbols 48 start moving by the image changing means 36, and the first screen is cleared by the voice output means 44 (to show completion). Is output. After that, another screen is displayed, and it becomes possible to color another pattern.

As described above, in the coloring toy 1 described above, the moving image data of the brush 50 reflecting the imaged motion of the player 2 is combined with the pattern image data and displayed, thereby displaying the CG moving image of the brush 50. Color and area can be selected. That is, the coloring can be performed by the movement of the player 2's own body in the virtual environment without using a pointing device or the like. In particular, since the image of the player 2 itself can be viewed on the display device 5, the player 2 can experience the sensation of being active by entering the virtual environment. . Therefore, it is possible to provide the player 2 with fun and to enhance the satisfaction of the player 2.

Further, in the coloring toy 1 described above, the pattern 48 changes or a sound is output depending on whether the colored color is correct or incorrect, which makes the selection of the color more interesting and enhances the game feeling. it can. Therefore, it can be enjoyed not only by infants but also by a wide range of age groups.

Further, in the coloring toy 1 described above, the amount of data to be recognized can be reduced by efficiently processing the moving image data having a large amount of ambiguous information. In addition, by using a learnable neural network, the recognition effect of the motion can be improved by the learning effect. In particular, depending on the physical characteristics of the player 2 and the imaging position,
Even if different image information is supplied, the efficiency of the recognition process can be improved by performing the area division suitable for each image information.

Next, a coloring toy according to a second embodiment of the present invention will be described with reference to FIG. FIG. 7 is a block diagram showing the basic configuration of a coloring toy. The coloring toy 55 of this embodiment synthesizes only the CG moving image (for example, the moving image of the brush 50) reflecting the recognized motion of the player 2 without synthesizing the image of the player 2. Compared with the coloring toy 1 of the first embodiment, only the configuration of the control device 56, particularly the configurations of the image recognition unit 57, the image storage unit 58, and the image processing unit 59, are different, and the other configurations are the same. . Therefore, in the present embodiment, only different points will be described, the same configurations will be denoted by the same reference numerals, and detailed description thereof will be omitted.

The image recognizing unit 57 has a functional structure, such as a frame separating unit 15, a change amount detecting unit 60, a moving pixel extracting unit 61, an average coordinate calculating unit 62, and a moving direction detecting unit 6.
3, and a motion recognition means 64. The frame separation means 15 inputs the moving image data from the image pickup means 3,
A frame image separated at a predetermined time interval is generated.
Then, the sequentially generated frame images are stored in the image storage unit 58. The change amount detecting means 60 detects the frame images generated by the frame separating means 15, particularly the latest frame image (referred to as a second frame image) separated at predetermined time intervals, and the previous frame image (first frame image). Is called from the image storage means 58, and the amount of change from the first frame image to the second frame image is detected for each pixel.

The moving pixel extraction means 61 compares the amount of change for each pixel with a predetermined threshold value 65, and extracts the pixels whose amount of change is greater than the threshold value 65 as moving pixels (moving pixels). To do. The average coordinate calculation means 62 averages the coordinates (XY coordinates) of the extracted moving pixels to calculate an average coordinate. The moving direction detecting means 63 determines the moving direction (XY) of the average coordinates based on the two average coordinates calculated at a predetermined time interval.
Vector) on the coordinates. Motion recognition means 64
Recognizes the content of the action of the player 2 based on the average coordinates and the moving direction.

The image processing unit 59 is different from the image processing unit 12 in the first embodiment in that the player image output means 3 is used.
3 is not provided. That is, since the image of the player 2 is not synthesized, the player image output means 33 is omitted.

Next, detailed contents of the processing in the main part of the control device 56 will be described. The coloring toy 55 of the second embodiment does not perform the process of capturing the background in advance or obtaining the difference between the frame image and the background image data as in the first embodiment, but separates at a predetermined interval. By comparing the frame images with each other, the operated portion is recognized. Specifically, the change amount detecting means 60 detects the change amount (difference) by comparing the bitmap data of the first frame image and the second frame image in pixel units every time the latest frame image is generated. To do. That is,
If the brightness and color components change due to the movement of the player 2, the bitmap data of the corresponding pixels change, and therefore the amount of change is detected, so that the player 2
The presence or absence of movement in each part of is recognized.

By the way, in the method in which the data regarding the variation amount is held for each pixel and the data is arithmetically processed to recognize the movement, the data amount becomes enormous and the processing may take a relatively long time. In particular, in a toy, it is difficult to mount a high-performance CPU such as a computer, and thus complicated arithmetic processing cannot be instantaneously performed. On the other hand, even if the head and feet slightly move when the hand is swung down from above, the minute movements of the head and feet are meaningless information for recognizing the movement of the player 2. Moreover, the information is noise.

Therefore, the moving pixel extracting means 61 of the present embodiment.
Then, a pixel whose change amount is larger than the threshold value 65 is a moving pixel (a pixel that has moved), and a pixel whose change amount is smaller than the threshold value 65 is a static pixel (a pixel that does not move). As a result, the amount of data is extremely small and noise can be removed.

In order to obtain the center of mass of the plurality of moving pixels extracted by the moving pixel extracting means 61, the average coordinate calculating means 62 averages the plurality of moving coordinates to calculate the average coordinates. That is, one average coordinate is determined for a plurality of moving coordinates. Further, the moving direction detecting means 63 detects the moving direction of the average coordinates in a predetermined time, that is, the vector in the XY coordinates.

In the motion recognizing means 64, based on the average coordinates calculated by the average coordinate calculating means 62 and the moving direction of the average coordinates detected by the moving direction detecting means 63,
Recognize the movement of the player 2. That is, the CG moving image is created so that the tip of the brush 50 matches the vector and the position of the brush 50 matches the average coordinates. As a result, similarly to the first embodiment, it becomes possible to select a color and a region.

In this way, the coloring toy 5 according to the second embodiment.
In 5, in addition to the effect of the coloring toy 1 of the first embodiment, by comparing the amount of change for each pixel with a threshold value and extracting moving pixels, moving image data having enormous and ambiguous information can be efficiently provided. Can be processed. Further, the efficiency of the recognition process can be improved by calculating the average coordinates of the moving pixels and extracting the motion information corresponding to the average coordinates.

Although the present invention has been described above with reference to two preferred embodiments, the present invention is not limited to these embodiments, and the scope of the present invention is not deviated as shown below. Therefore, various improvements and design changes are possible.

For example, in the first and second embodiments, C
Although the brush 50 is shown as a G moving image, its shape is not particularly limited. However, by displaying the brush 50, it is possible for the player 2 to experience the actual sensation of painting a color using the brush 50.

In the first and second embodiments, the TV monitor is used as the display device 5, but a dedicated display may be used. In addition, the display device 5
May be formed integrally with the control device 4.

[0068]

According to the first aspect of the present invention, the CG moving image data reflecting the player's motion is combined with the pattern image data and displayed, thereby selecting the color and the area via the CG moving image. Is possible. Therefore, the coloring can be performed by the movement of the player's own body in the virtual environment without using a pointing device or the like. In particular, since the CG image reflecting the player's action is displayed, it is possible to provide the player with interest and enhance the player's satisfaction.

According to the invention of claim 2, in addition to the effect of the invention of claim 1, since it becomes possible to see the image of the player himself / herself by the display means, the player can be provided with a virtual environment. You can experience the feeling that you are inside and are active. Therefore, the interest can be further enhanced.

According to the invention of claim 3, in addition to the effect of the invention of claim 1 or claim 2, it is possible to experience the actual feeling that the player himself paints with a brush.

According to the invention of claim 4, in addition to the effect of any one of the inventions of claims 1 to 3, the design changes depending on whether the colored color is correct or incorrect. It is possible to increase the fun of the game and enhance the game feeling. Therefore, it can be enjoyed not only by infants but also by a wide range of age groups.

According to the invention of claim 5, in addition to the effect of the invention of claim 4, it is possible to entertain with both visual and auditory sensations, and the player's satisfaction can be further enhanced.

According to the invention of claim 6, in addition to the effect of any one of the inventions of claims 1 to 5, by efficiently processing moving image data having enormous and ambiguous information, the recognition processing is performed. It is possible to reduce the amount of target data of. In addition, by using a learnable neural network, the recognition effect of the motion can be improved by the learning effect.

According to the invention of claim 7, in addition to the effect of the invention of claim 6, even when different image information is supplied depending on the height difference of the player and the imaging condition, the area division suitable for each image information is performed. By doing so, the efficiency of the recognition processing can be improved.

According to the invention of claim 8, in addition to the effect of the invention of claim 6 or claim 7, a neural network configured in multiple stages is used, and learning is performed in each neural network according to its purpose. As a result, each neural network can be optimized according to its purpose. Further, by combining a plurality of types of pattern codes, multifaceted recognition processing can be performed, and the recognition accuracy can be improved.

According to the ninth aspect of the present invention, in addition to the effect of the first aspect of the invention, the amount of change for each pixel is compared with a threshold value to extract a moving pixel, thereby having a huge amount of ambiguous information. Video data can be processed efficiently. Further, by calculating the average coordinates of the moving pixels and recognizing the motion based on the average coordinates, the efficiency of the recognition process can be improved.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a usage state of a coloring toy that is a first embodiment of the present invention.

FIG. 2 is a block diagram showing the basic configuration of a coloring toy that is the first embodiment of the present invention.

FIG. 3 is an explanatory diagram showing a concept of area division of a frame image in the coloring toy according to the first embodiment of the present invention.

FIG. 4 is an explanatory diagram showing an example of an input procedure to the first neural network in the coloring toy according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of an input procedure to the second neural network in the coloring toy according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram showing an example of a display screen of the coloring toy according to the first embodiment of the present invention.

FIG. 7 is a block diagram showing a basic configuration of a coloring toy according to a second embodiment of the present invention.

[Explanation of symbols]

1,55 coloring toys 2 players 3 Imaging means 5 Display device (display means) 13 Design image output means 15 frame separation means 16 area dividing means 17 Conversion means 18 Knowledge processing means 19 Motion recognition means 21 Detection function (detection means) 22 Split setting function (split setting means) 24 First Neural Network (Knowledge Processing Means) 25 Second Neural Network (Knowledge Processing Means) 27 Background Image Storage Unit (Background Image Storage Means) 29 first code storage area (first code storage means) 30 second code storage area (second code storage means) 32 CG movie creation means 33 image composition means 34 Display control means 35 Player image output means 36 Image variation means 38 color selection means 39 area selection means 40 Coloring control means 42 color matching means 43 Color information storage means 44 audio output means 46 frame images 48 designs 50 brushes 58 image storage means 60 Change amount detecting means 61 Video Element Extraction Means 62 means for calculating average coordinates 63 moving direction detecting means 64 motion recognition means 65 threshold

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 3/00 300 G06T 3/00 300 7/00 350 7/00 350C 7/20 300 7/20 300A 11 / 80 11/80 A (72) Inventor Norihiro Kasano 4-179-1, Suue-cho, Kakamigahara-shi, Gifu 1 Ocean Ocean Co., Ltd. (72) Inventor Saori Makino 4-179, Sue-cho, Kakamigahara-shi 1 F-term in Ocean Network Co., Ltd. (reference) 2C001 AA14 BA01 BA03 BA06 BB03 BC05 BC06 CA08 CB01 CB04 CB06 CC02 5B050 BA06 BA07 BA08 BA12 BA18 CA07 DA04 EA03 EA04 EA06 EA10 EA19 EA24 EA30 FA02 FA05 5B057 CA08 CA12 CA16 CA16 CA11 CA01 CA16 CA11 CA01 CA16 CB12 CB16 CE08 CE09 CE16 CG07 DA08 DA17 DA20 DB02 DB06 DB09 DC08 DC30 5B080 FA02 GA25 5L096 AA02 AA06 CA04 EA21 EA35 FA06 FA32 FA67 GA08 HA03 HA04 HA11

Claims (9)

[Claims] 1. An image pickup means for picking up an image of a player and outputting moving picture data, and a motion recognition means for recognizing a movement of the player as movement information based on the moving picture data obtained by the image pickup means. Input the motion information obtained by the motion recognition means,
CG for creating CG moving image data by computer graphic reflecting the action of the player based on the action information
A moving image creating means, a symbol image output means for outputting symbol image data including a pattern which is a basis of a coloring picture, an image synthesizing means for synthesizing the CG moving image data with the symbol image data given by the symbol image output means, And a display control unit for displaying the combined image by outputting the combined image data combined by the image combining unit to the display unit, wherein the symbol image output unit includes a plurality of color samples based on the CG moving image data. A color selecting means for selecting a color to be colored from among the above, an area selecting means for selecting an area to be colored from the pattern based on the CG moving image data, and a selected area with a selected color. A coloring toy having a coloring control means for filling. 2. The player further comprises a player image output means for outputting the moving image data obtained by the image pickup means as player image data, wherein the image synthesizing means further includes the player image data in the symbol image data. The coloring toy according to claim 1, wherein the coloring toy is synthesized. 3. The CG moving image data created by the CG moving image creating means comprises moving image data representing a brush,
The coloring toy according to claim 1 or 2, wherein the motion of the player's hand is reflected. 4. A color information storage unit in which model colors are stored in advance as color information corresponding to respective areas of the design, and the filled color and the color when the selected area is filled. 4. A color collating means for collating information, and an image varying means for varying the pattern image data in the image synthesizing means on the basis of the collating result, further comprising: an image varying means. The coloring toy described in one. 5. Based on the matching result of the color matching means,
The coloring toy according to claim 4, further comprising voice output means for outputting a preset voice. 6. A frame separating unit that inputs the moving image data obtained by the image capturing unit and separates the moving image data into frame images at predetermined time intervals, and the image capturing unit previously captures only a background and stores it as background image data. Background image storage means, area dividing means for dividing the frame image into a plurality of small areas, and the background image data is read from the background image storage means, and using the background image data, each of the small areas is stored. It is determined whether or not a characteristic region representing the player is included, and based on the result of the determination, a conversion unit that converts the frame image into numerical sequence data, and the numerical sequence data is applied to a learnable neural network. Further comprising knowledge processing means for generating a pattern code by means of the motion recognition means, wherein the motion recognition means uses the pattern code obtained by the knowledge processing means. Coloring toy according to any one of claims 1 to 5, characterized in that to recognize the contents of the operation. 7. The area dividing means reads the background image data from the background image storage means, detects the characteristic area in the frame image using the background image data, and the detecting means. 7. The coloring toy according to claim 6, further comprising division setting means for performing division into the small areas based on the detection information of. 8. The knowledge processing means inputs the numerical sequence data into a first neural network and generates a first pattern code from the output thereof, and stores the generated first pattern code. A first code storing means for reading a plurality of the first pattern codes from the first code storing means and inputting the same in time series to a second neural network, and outputting the output corresponding to one motion of the player. A second knowledge processing means for generating a two-pattern code, and a second code storage means for storing the generated second pattern code, wherein the motion recognition means is configured to operate from the second code storage means. The second pattern code is read, and the first pattern code corresponding to the second pattern code is read from the first code storage means. Coloring toy of claim 6 or claim 7, characterized in that to recognize the contents of the operation using these pattern codes. 9. A frame separation means for inputting the moving image data obtained by the image pickup means and separating the frame images into frame images at predetermined time intervals, an image storage means for storing the frame images, and separation at predetermined time intervals. Change amount detecting means for detecting the change amount from the first frame image to the second frame image for each pixel by reading the first frame image and the second frame image from the image storage means; Moving pixel extraction means for comparing the amount of change with a threshold value and extracting pixels with the amount of change larger than the threshold value as moving pixels; and averaging coordinates of the plurality of extracted moving pixels to obtain an average coordinate. An average coordinate calculating means for calculating, and a moving direction detecting means for detecting a moving direction of the average coordinate based on the two average coordinates calculated at a predetermined time interval by the average coordinate calculating means. And means, said operation recognizing means, Coloring toy of claim 1, based on the average coordinate and the moving direction, and recognizes the contents of the operation.