JP5936670B2 - Experience-type video game device - Google Patents

Description

  The present invention relates to a bodily sensation video game apparatus which is an interactive video game apparatus that receives a body action recognized through a camera.

  2. Description of the Related Art A sensory video game apparatus that is an interactive video game apparatus that receives a body action recognized through a camera has been known (see, for example, Patent Documents 1 and 2 and Non-Patent Document 1).

  In general, this type of sensation-type video game apparatus includes an image display having a display screen (usually a substitute for a stationary television) and a camera for photographing a player's body action (integrated with or separate from the game main unit). And a game computer (game device main body) to which an application program for a sensation-type video game is to be installed, and the player recognized through the camera by executing the application program on the game computer An interactive game image with the body action as an input is displayed on the screen of the image display.

  According to such a bodily sensation-type video game device that recognizes body action using a camera, unlike the bodily sensation-type video game device that recognizes body action using a sensor such as an acceleration sensor, body action recognition Therefore, there is no need for the player to hold the operation tool with a built-in sensor, the cost of the entire apparatus can be reduced, and the player also has the advantage that the degree of freedom of body action selection is increased.

JP 2002-306846 WO2007 / 069618

http://en.wikipedia.org/wiki/Kinect

  Of course, even in the above-mentioned experience-type video game device, the camera and the game device main body can be freely carried to some extent, but the presence of a stationary television is indispensable for the carrying destination. The degree of freedom was still limited and lacked the convenience of being able to carry and play anywhere and anytime.

  In this case, the present inventors are the tablet devices such as smart phones, tablet PCs, and tablet notebook PCs that are becoming widespread in recent years. , An image display having a display screen, a camera (referred to as “front camera”, “in-camera”, or “sub-camera”) for shooting the body action of the player; Since the game computer to which the application program should be installed (CPU built in the terminal) is already provided and has sufficient portability, the application program for the sensation-type video game (hereinafter abbreviated as “game application”). If you have a proper design, you can easily carry and play anywhere and anytime Bets can be, moreover inspired the operable sensory video game device via the body action can be realized.

  However, the present inventors can easily enjoy a game with a simple operation by any user of the tablet-type terminal even in such a sensation-type video game apparatus using the tablet-type terminal and the game application. In order to do so, we have found that there are still problems to be solved.

  That is, for a game application applied to a sensation-type video game device using such a tablet-type terminal and a game application, for example, image development that expresses the spread in the depth direction using perspective is used. Because it adopts, it demands a screen rotation posture that becomes a vertically long screen, or it demands a screen rotation posture that becomes a horizontally long screen because it adopts image development that expresses the spread in the horizontal direction. In addition, there is a request regarding the screen rotation posture due to the development direction of the game image.

  In addition, for this type of game application, for example, in order to improve recognition accuracy, a shooting posture that looks down on the body action to be recognized is adopted, so the camera's shooting window is located at the top of the screen. Because it requires a rotation posture or, conversely, adopts a shooting posture that looks up, the camera's shooting window requires a screen rotation posture that is located at the bottom of the screen. There is a demand for a screen rotation posture due to the body action shooting posture.

  On the other hand, there are various types of tablet-type terminals, such as smart phones, tablet PCs, and notebook PCs for tablet use. As long as there is a simple support tool, a vertically long screen rotation posture and a horizontally long screen rotation posture can be freely selected.

  For this reason, a video game device is realized by installing and executing an arbitrary game application on a tablet-type terminal, and also has a sufficient display field of view and good visibility and understanding of the game image. In order to achieve this, the player must be encouraged to take the necessary screen rotation posture so that the requirements regarding the screen rotation posture due to the development direction of the game image are sufficiently satisfied.

  In addition, depending on the type of video game, in order to ensure sufficient body action recognition accuracy and good input responsiveness, there is a sufficient demand for screen rotation posture due to the body action shooting posture. To be satisfied, the player must be encouraged to take the necessary screen rotation posture.

  However, although there are many models of the sub-camera shooting window in this type of tablet-type terminal that are arranged at the upper part of the screen in the standard rotation posture, there are also models that are arranged at the lower part of the screen. In addition, in the first place, there are many models that have a vertically long screen, but there are some models that have a horizontally long screen.

  In addition, an operating system (hereinafter referred to as “OS”) of this type of tablet terminal has an image corresponding to the standard rotation posture according to the setting content of “image display orientation setting” that can be set by the operation on the terminal side. There is an image orientation determination function that can set the rotation angle arbitrarily (0 degrees, 90 degrees, 180 degrees, 270 degrees, sensor follow-up). However, in order to perform setting operations related to “image display orientation setting”, it is necessary to have a certain level of knowledge about terminal operation, so it is expected that all users of this type of tablet terminal will be able to use this function. I don't get it.

  Therefore, if instruction is given to the terminal user via attachment of an appropriate manual or guidance display of an explanatory text in order to take a necessary screen rotation posture, the terminal user can appropriately It is very doubtful whether the screen rotation posture can be realized immediately. This is because the standard rotation posture and camera position are not the same on all terminals, and even if the same game application is used, the image rotation angle with respect to the standard rotation posture differs for each terminal model. It is.

  In addition, since the setting content of “image display orientation setting” in this type of tablet-type terminal is usually “sensor tracking” by default, it is displayed on the screen in such a state. The game image is always faced upward regardless of the screen rotation posture, so that the proper screen rotation posture for the game application becomes more difficult to understand and confuses the terminal user.

  On the contrary, once the setting content of “image display orientation setting” is set to any rotation angle, in such a state, the game image displayed on the screen is always on the screen. Since the rotation angle is constant, the terminal user easily misunderstands that the screen rotation posture in which the image is upward is correct, and this also confuses the terminal user.

  As a result, in order to provide the necessary screen rotation posture, when instructing the terminal user through attachment of an appropriate manual or guidance display of an explanatory text, an inappropriate screen rotation posture is often used. As such, there is a problem that the sensation type video game is played, and the game image cannot be sufficiently developed on the screen, thereby impairing the visibility, and the input responsiveness is impaired due to a decrease in recognition accuracy of the body action.

  The present invention has been made paying attention to the above-mentioned problems, and the object of the present invention is to determine which of the users of the tablet terminal in the sensation-type video game apparatus using the tablet terminal and the game application. The purpose is to make it easy to enjoy the game with simple operations.

  Other objects and operational effects of the present invention will be easily understood by those skilled in the art by referring to the following description of the specification.

  The above-mentioned object is solved by a sensation type video game apparatus, an application program, a recording medium, a client and server system, and a program providing server according to the present invention having the following configurations.

<Experienced video game device>
That is, the sensation type video game device according to the present invention includes an image display having a display screen, a camera for photographing a body action of a player, and a game computer in which an application program for the sensation type video game is to be installed. And the game computer executes the application program, so that an interactive game image with the player's body action recognized through the camera as an input is displayed on the display screen of the image display. An experienced video game apparatus that is displayed, wherein the image display device, the camera, and the game computer have a display screen integrally, such as a smartphone, a tablet PC, and the like. Window for the front side of the display screen The application program includes the tablet type terminal regardless of the contents of the image display orientation setting set on the tablet type terminal side. A program portion is included for causing a game image related to the program to function as a means for directing the game image to be displayed on the display screen in a state where the rotation angle of the game image with respect to the display screen is fixed to a specific rotation angle. Further, when the rotation orientation of the display screen is determined so that the orientation on the image of the game image and the orientation on the real space coincide with the specific rotation angle, the game image is displayed in a portrait screen or landscape screen. The proper screen rotation posture that satisfies the requirements for image development such as which should be displayed automatically Those defined so as to obtain.

  According to such a configuration, the rotation orientation of the display screen is simply determined so that the orientation on the image of the game image (for example, upward) matches the orientation in the real space (for example, upward). Since an appropriate screen rotation posture that satisfies the requirements for image development such as whether to display the game image on a portrait screen or landscape screen is automatically obtained, for example, a game image for a portrait screen is displayed. It is possible to avoid image visibility degradation due to an error in the screen rotation posture, such as displaying on a landscape screen or conversely displaying a game image for a landscape screen on a portrait screen, and the body action to be recognized is particularly strict Unless the user demands a proper shooting posture, any user of the tablet-type terminal can enjoy the experience-based video game easily.

  In a preferred embodiment (1) of the present invention, the rotation angle of the display screen is determined so that the specific rotation angle matches the orientation of the game image on the image and the orientation on the real space. Then, an appropriate screen rotation posture that automatically satisfies the body action shooting posture, such as from which angle the camera should take the body action of the player through the shooting window, is automatically obtained. It may be determined to be.

  According to such a configuration, when the rotation orientation of the display screen is determined so that the orientation on the image of the game image matches the orientation on the real space, the body action of the player is changed to the shooting window. The camera can automatically obtain a proper screen rotation posture that satisfies the shooting posture requirement of the body action, such as from which angle the camera should shoot. The body action that should be taken as if looking down from the window is taken from the shooting window located at the bottom of the screen, or conversely, the body action that should be taken as seen from the shooting window located at the bottom of the screen is displayed at the top of the screen It is possible to avoid a decrease in body action recognition accuracy due to an error in the screen rotation posture, such as shooting from the shooting window located at As well as a body action of interest were those that require particularly rigorous shooting attitude, it is possible to anyone of the user of the tablet-type terminal to enjoy easy access to experience video game.

  In a preferred embodiment (2) of the present invention, the specific rotation angle is based on a screen height / width request on the application program side and a screen height / width specification of the terminal derived from the model information of the tablet terminal. It may be determined.

  In any tablet terminal, whether the screen is vertically long or horizontally long in the standard rotation posture is uniquely determined by the model information of the tablet terminal. Therefore, for example, if the game application side holds information indicating whether the portrait screen / landscape screen in the standard rotation posture for each assumed terminal model, etc., it corresponds to the model of the tablet type terminal. By only providing information, the rotation angle of the screen can be easily obtained so as to obtain an appropriate vertical and horizontal posture for developing the game image on the screen.

  In a preferred embodiment (3) of the present invention, the specific rotation angle is determined based on the screen length request and the screen request on the application program side and the shooting window position request and the model information of the tablet terminal. It may be determined based on the screen vertical / horizontal specification and the shooting window position specification.

In an arbitrary tablet terminal, whether the screen is vertically long / horizontal in a standard rotation posture or whether the camera shooting window is superscript / subscript is uniquely determined by the model information of the tablet terminal. Therefore, for example, on the game application side, information indicating whether a vertical screen / horizontal screen in a standard rotation posture for each assumed terminal model, and whether the camera shooting window is superscript / subscript If information etc. are kept, simply giving the information corresponding to the model of the tablet type terminal, it is possible to easily determine the rotation angle of the screen so as to obtain an appropriate shooting posture for shooting the body action. Can do.
By the way, as a method of giving information corresponding to the model of the tablet-type terminal, for example, when downloading an application from the application providing server, a method of reporting the model information of the tablet-type terminal to the user, and providing a game application There are various methods such as a method of allowing the user to input model information of the tablet type terminal after downloading from the server. However, in such a method, since the user of the tablet terminal is requested to input accurate model information of the own tablet terminal, the burden on the user side is large.

  On the other hand, as the model information, if the model information read from the tablet type terminal itself in which the application program is installed is used, the user of the tablet type terminal is notified of his / her own tablet type terminal. It is no longer necessary to request input of accurate model information, and the user is freed from the burden of input operation.

<Application program>
Viewed from another aspect, the present invention is a commercially available camera having a display screen integrated therein, such as a smartphone, a tablet PC, etc., and a camera with a shooting window facing the front side of the display screen. It can also be understood as an application program to be installed on the tablet terminal.

  That is, the application program includes a means for displaying an interactive game image on the display screen, with the tablet terminal as an input of a body action of a player recognized via the camera, and the tablet Regardless of the content of the image display orientation setting set on the terminal side, the game image related to the application program is displayed on the display screen in a state where the rotation angle of the game image with respect to the display screen is fixed to a specific rotation angle. The display screen is configured to function as a means for directing display, and the specific rotation angle is a rotation orientation of the display screen so that an orientation on the image of the game image coincides with an orientation on a real space. The game image should be displayed on the portrait screen or landscape screen. Proper screen rotation posture that satisfies the requirements of the image development has is determined so as to automatically obtain.

  In a preferred embodiment (1) of the present invention, the rotation angle of the display screen is determined so that the specific rotation angle matches an orientation on the image of the game image with an orientation on the real space. Then, an appropriate screen rotation posture that automatically satisfies the body action shooting posture, such as from which angle the camera should take the body action of the player through the shooting window, is automatically obtained. It may be determined to be.

  In a preferred embodiment (2) of the present invention, the specific rotation angle is the terminal derived from a screen vertical / horizontal request on the application program side and model information of the tablet terminal. It may be determined based on the screen vertical and horizontal specifications.

  In a preferred embodiment (3) of the present invention, the specific rotation angle is determined based on the screen length request and the screen request on the application program side and the shooting window position request and the model information of the tablet terminal. It may be determined based on the screen vertical / horizontal specification and the shooting window position specification.

  According to the above-described application program according to the present invention, the same operational effects as the above-described bodily sensation type video game apparatus according to the present invention are exhibited.

<Recording medium>
Viewed from another aspect, the present invention is a commercially available camera having a display screen integrated therein, such as a smartphone, a tablet PC, etc., and a camera with a shooting window facing the front side of the display screen. It can also be understood as a recording medium storing an application program to be executed by the tablet terminal.

  In other words, the application program stored in the recording medium displays an interactive game image on the display screen using the tablet terminal as an input of the body action of the player recognized through the camera. Regardless of the content of the image display orientation setting set on the tablet terminal and the game image according to the application program, the rotation angle of the game image with respect to the display screen is fixed to a specific rotation angle. In the state, it is made to function as a means for directing display on the display screen, and the specific rotation angle is set so that the orientation on the image of the game image matches the orientation on the real space. When the rotation orientation of the display screen is determined, the game image is displayed on either the portrait screen or landscape screen. Proper screen rotation posture that satisfies the requirements on the image development that say should be displayed on is determined so as to automatically obtain.

  In a preferred embodiment (1) of the present invention, the rotation angle of the display screen is determined so that the specific rotation angle matches an orientation on the image of the game image with an orientation on the real space. Then, an appropriate screen rotation posture that automatically satisfies the body action shooting posture, such as from which angle the camera should take the body action of the player through the shooting window, is automatically obtained. It may be determined to be.

  In a preferred embodiment (2) of the present invention, the specific rotation angle is based on a screen height / width request on the application program side and a screen height / width specification of the terminal derived from the model information of the tablet terminal. It may be determined.

  In a preferred embodiment (3) of the present invention, the specific rotation angle is determined based on the screen length request and the screen request on the application program side and the shooting window position request and the model information of the tablet terminal. It may be determined based on the screen vertical / horizontal specification and the shooting window position specification.

  According to the recording medium of the present invention described above, the same operational effects as those of the above-described sensation type video game apparatus according to the present invention are exhibited.

<Client and server system>
Another aspect of the present invention incorporates a program providing server and a camera having a display screen as an integral unit, such as a smartphone and a tablet PC, and a shooting window facing the front side of the display screen. It is also possible to grasp a commercially available tablet terminal as a client and server system connected by a network.

  In this client and server system, the program providing server displays, on the display screen, an interactive game image using the tablet terminal as an input of the player's body action recognized through the camera. The rotation angle of the game image relative to the display screen is fixed at a specific rotation angle for the game image according to the application program, regardless of the content of the image display orientation set on the tablet terminal In such a state, it is made to function as a means for directing to display on the display screen, and the specific rotation angle is such that the orientation on the image of the game image coincides with the orientation on the real space. When the rotation posture of the display screen is determined, the game image is displayed in a portrait screen or landscape orientation. One or more application programs are stored so that an appropriate screen rotation posture that satisfies the image development requirement such as which of the screens should be displayed is automatically obtained. The program providing server further includes the tablet terminal in the one or more types of application programs in response to a download request from the tablet terminal via the network. The application program specified from the side is configured to be transmitted to the tablet terminal on the condition of predetermined authentication, and the tablet terminal responds to the download request from the program providing server. Receive and store the application program that is sent, and By executing the application program on the condition of starting operation of the player, an interactive game image is input on the display screen with the player's body action recognized through the camera as an input. It is a thing.

  In a preferred embodiment (1) of the present invention, the rotation angle of the display screen is determined so that the specific rotation angle matches an orientation on the image of the game image with an orientation on the real space. Then, an appropriate screen rotation posture that automatically satisfies the body action shooting posture, such as from which angle the camera should take the body action of the player through the shooting window, is automatically obtained. It may be determined to be.

  In a preferred embodiment (2) of the present invention, the specific rotation angle is based on a screen height / width request on the application program side and a screen height / width specification of the terminal derived from the model information of the tablet terminal. It may be determined.

  In a preferred embodiment (3) of the present invention, the specific rotation angle is determined based on the screen length request and the screen request on the application program side and the shooting window position request and the model information of the tablet terminal. It may be determined based on the screen vertical / horizontal specification and the shooting window position specification.

  According to the client and server system according to the present invention described above, the same operational effects as those of the above-described sensation type video game apparatus according to the present invention are exhibited.

<Program provider server>
According to another aspect of the present invention, there is provided a camera having a display screen integrated with a program providing server, a smartphone, a tablet PC, etc., and a shooting window directed to the front side of the display screen. It is also possible to grasp the built-in commercially available tablet terminal as the program providing server constituting the client and server system connected by the network.

  The program providing server includes means for displaying an interactive game image on the display screen using the tablet terminal as an input of a body action of a player recognized through the camera, and the tablet type Regardless of the content of the image display orientation setting set on the terminal side, the game image related to the application program is displayed on the display screen in a state where the rotation angle of the game image with respect to the display screen is fixed to a specific rotation angle. The display screen is configured to function as a means for directing display, and the specific rotation angle is a rotation orientation of the display screen so that an orientation on the image of the game image coincides with an orientation on a real space. On the image development that says whether the game image should be displayed on the portrait screen or landscape screen Proper screen rotation posture that satisfies the request is defined as automatically obtained.

  In a preferred embodiment (1) of the present invention, the rotation angle of the display screen is determined so that the specific rotation angle matches an orientation on the image of the game image with an orientation on the real space. Then, an appropriate screen rotation posture that automatically satisfies the body action shooting posture, such as from which angle the camera should take the body action of the player through the shooting window, is automatically obtained. It may be determined to be.

  In a preferred embodiment (2) of the present invention, the specific rotation angle is based on a screen height / width request on the application program side and a screen height / width specification of the terminal derived from the model information of the tablet terminal. It may be determined.

  In a preferred embodiment (3) of the present invention, the specific rotation angle is determined based on the screen length request and the screen request on the application program side and the shooting window position request and the model information of the tablet terminal. It may be determined based on the screen vertical / horizontal specification and the shooting window position specification.

  According to the above-described program providing server of the present invention, the same operational effects as those of the above-described sensation type video game apparatus according to the present invention are exhibited.

  According to the present invention, in a bodily sensation type video game apparatus using a tablet terminal and a game application, any user of the tablet terminal can easily enjoy a game with a simple operation.

FIG. 1 is an explanatory diagram showing the configuration of the entire application providing system. FIG. 2 is a block diagram showing a general hardware configuration of the tablet terminal. FIG. 3 is a block diagram illustrating a hardware configuration of the application providing server. FIG. 4 is a flowchart for explaining the application download operation. FIG. 5 is a flowchart for explaining processing on the application side. FIG. 6 is a flowchart for explaining display processing on the OS side. FIG. 7 is a flowchart showing the configuration of the image rotation program part. FIG. 8 is a detailed flowchart of the image rotation angle determination process (according to a request from the application side). FIG. 9 is a detailed flowchart of image rotation angle determination processing (depending on terminal specifications and application-side requests) FIG. 10 is an explanatory diagram showing the positional relationship between the screen and the sub camera for each model of smart phone. FIG. 11 is an explanatory diagram showing the positional relationship between the screen of each model of tablet PC and the sub camera. FIG. 12 is an explanatory diagram showing the positional relationship between the screen of the tablet PC and the sub camera. FIG. 13 is an explanatory diagram (camera is superscript) when the image is fixed at a rotation angle of 0 degree with respect to the screen. FIG. 14 is an explanatory diagram (camera subscript) when the image is fixed at a rotation angle of 180 degrees with respect to the screen. FIG. 15 is an explanatory diagram (with a camera attached) when the image is fixed at a rotation angle of 90 degrees with respect to the screen. FIG. 16 is an explanatory diagram (with a camera attached) when the image is fixed at a rotation angle of 90 degrees with respect to the screen. FIG. 17 is an explanatory diagram of a table tennis game during play. FIG. 18 is an explanatory diagram of an image displayed on the screen during play. FIG. 19 is an explanatory diagram showing a product sales form. FIG. 20 is an explanatory diagram of a table tennis game playground equipment.

  Hereinafter, a preferred embodiment of the sensation type video game apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

  As described above, the conventional sensation-type video game apparatus includes an image display having a display screen (usually a substitute for a stationary television) and a camera for photographing a player's body action (integrated with the game main unit). Or a game computer (game device main body) to be installed with an application program for a sensation-type video game (ie, “game app”), and the game app is a game computer. By executing the above, an interactive game image using the body action of the player recognized through the camera as an input is displayed on the screen of the image display.

  On the other hand, the sensation type video game apparatus according to the present invention integrates a display screen with functions realized by the above-described image display, camera, and game computer, such as a smartphone and a tablet PC. And a commercially available tablet terminal in which a camera with a shooting window facing the front side of the display screen is incorporated.

  In addition, in the game application, the game device according to the game application is displayed on the display screen regardless of the image display orientation setting content set on the tablet terminal. A program portion for causing the game image to function as a means for directing the game image to be displayed on the display screen in a state where the rotation angle of the game image is fixed to a specific rotation angle is included.

  Here, when the rotation orientation of the display screen is determined such that the orientation on the image of the game image and the orientation on the real space coincide with the specific rotation angle, the game image is displayed in a portrait screen or landscape orientation. It is determined so that an appropriate screen rotation posture that satisfies the requirement on image development such as which screen should be displayed is automatically obtained.

<Provision of game application>
As for how to install the game application on the tablet-type terminal side by using a providing method, a providing method via a portable recording medium such as an SD memory or a USB memory, a provision provided via a short-range radio such as an infrared ray, etc. Various methods are conceivable. In this embodiment, an application providing system that is a client-and-server system is constructed, and a providing method of downloading from an application providing server on the Internet to a tablet terminal is provided. It has been adopted.

-Configuration of application provision system-
An explanatory diagram showing the configuration of the entire application providing system is shown in FIG. As shown in the figure, various types of smartphones 1, 1, 1... And various types of tablet PCs 2, 2, 2,. The internet 5 can be wirelessly connected, while the application providing server 4 is connected to the internet 5 by a known method.

-Configuration of tablet terminal-
A block diagram showing a general hardware configuration of the tablet terminal 6 is shown in FIG. Since this type of hardware configuration is known from various documents, the description thereof will be briefly made except for the main part of the present invention.

  In the figure, the microcomputer 6a performs overall control of the entire terminal, and is usually composed of a known microprocessor, ASIC, FPGA or the like that realizes a specific circuit function.

  The microphone 6b and the speaker 6c are used, for example, for voice calls. The touch panel control unit 6d and the touch panel 6c are for realizing a touch panel function on the display screen. The vibrator 6f is for notifying the user of an incoming call notification or any other state. The lamp driving unit 6g and the LED lamp 6h are for operating various LED lamps for status display.

  The main camera module 6i is a functional module related to the main camera whose shooting window is directed to the rear of the display screen, and specifically includes a lens, an image sensor, and the like necessary for shooting. . The image data acquired by the main camera module 6i is edited via a known image processing engine, stored in a buffer image memory built in the microcomputer 6a, and then displayed on a display screen as necessary. Or stored in a flash memory 6o described later.

  The sub camera module 6j is a functional module related to a sub camera (also referred to as an in camera, a front camera, etc.) whose shooting window is directed to the front of the display screen, and constitutes a main part of the present invention. It is. This sub camera module is also configured to include a lens, an image sensor, and the like necessary for photographing. The image data acquired by the sub camera module 6j is edited via a known image processing engine, and then stored in a buffer image memory built in the microcomputer 6a. If necessary (for example, self-photographing) Displayed on the display screen or stored in a flash memory 6o described later. In the case of the present invention, the edited image data stored in the buffer memory is provided for body action recognition performed by the player.

  The wireless communication unit 6k is used when the tablet terminal is connected to the Internet through a wireless LAN or the like by a known method. The key input device 61 is composed of a power-on button and various push button operation keys, and is used to give necessary instructions to the tablet terminal.

  The image display unit 6n and the display controller 6m constitute an image display (liquid crystal image display, organic EL image display, etc.), and the display controller 6m includes an image display unit 6m that is a display device. It controls image display control between a video RAM (not shown) built in the microcomputer (memory for storing image data to be displayed). There are various known methods for rotating an image to be displayed on the image display unit 6n. For example, when transferring image data from the video RAM to the image display unit 6a, a first method for rotating the image data. When writing the image data on the buffer memory to the video RAM, the image data on the OS side. There are a second method for performing the rotation operation, a third method for rotating the image data on the buffer memory from the generation stage, and the like. In the embodiment described below, the second method is adopted.

  The flash memory 6o is used to store various setting data in addition to storing various application programs to be downloaded. The RAM 6p is used as a working area for various arithmetic processes.

-Configuration of application providing server-
A block diagram showing the hardware configuration of the application providing server is shown in FIG. Since this type of hardware configuration is known from various documents, the description thereof will be briefly made except for the main part of the present invention.

  As shown in the figure, the application providing server 4 includes a control unit 4a including a CPU that controls the entire server device, a ROM, a RAM, a hard disk, and the like, and includes the various types of bodily sensation game applications described above. A storage unit 4b for storing various programs and various data, a communication unit 4c for communicating with an external device via a LAN or the Internet, a display unit 4d for performing display processing, and an external device An I / F unit 4e that performs input and output, an input operation unit 4f that performs input operations, and the like are included. The storage unit 4b may be provided as a storage or the like separately from the server device.

-Collecting money for game apps-
Table tennis games, baseball games, tennis games, boxing games, gun fight games, ice hockey games, American football games, basketball games, drum playing games, transformer games, fishing games, racing games, weapons There are various game applications, such as a battle game that has been played.

  The game application itself is provided from the application providing server 4 through the above-described application providing system. However, in this embodiment, the price collection is not a virtual store on the Internet but a supermarket. This is done through product sales at actual stores such as convenience stores and city toy stores.

  For example, as shown in FIG. 19, a product package 7 is prepared for each type of game application. Note that the product package 7 in FIG. 6A corresponds to a sensation-type table tennis game, and includes a playground equipment (in this example, a finger insertion holding type small-sized device that is indispensable for playing a table tennis game). In addition to 8 rackets, there is a card that displays a product package code (for example, a 16-digit character string code consisting of a combination of numerical values and alphabets) unique to each product package that can identify each product package for each game type It is. The product package 7 in FIG. 6B corresponds to a bodily sensation-type baseball game, and includes a play equipment that is indispensable for playing a baseball game (in this example, a small bat with a finger insertion holding type). In addition to 8, a card displaying a product package code (for example, a 16-digit character string code in combination of a numerical value and an alphabet) unique to each product package that can identify each product package for each game type is included. Yes. The product package 7 may further include a simple support 12 (see, for example, FIG. 17) for leaning and supporting a general smart phone or tablet PC at an appropriate angle.

  Therefore, when a user purchases a product package 7 corresponding to a favorite game type at a real store such as a supermarket, a convenience store, a city toy store, etc., the play equipment 8 and the product package code necessary for the game are displayed. The card (not shown) and, if necessary, the simple support 12 can be obtained, while the sales side can collect the price.

-Application download operation-
An explanatory diagram of the application download operation is shown in FIG. The user purchased the product package 7 corresponding to the favorite game type at a real store such as a supermarket, a convenience store, a toy store in the city, etc. Relying on the URL acquired through the search site, connection is made from the tablet terminal 6 to the application providing server 4.

  In this state, when a predetermined user operation is performed on the tablet type terminal side, the tablet type terminal 6 transmits a download request to the application providing server 4, and then an authentication code request is returned from the application providing server 4. Waiting to come (step 101).

  On the side of the application providing server 4 that has received the download request (step 111), the authentication code request is sent to the tablet terminal 6 (step 112), and then the authentication code is returned from the tablet terminal 6. stand by.

  The tablet terminal 6 waits for an authentication code request to be received (step 102), transmits the authentication code to the application providing server 4 (step 103), and then returns an authentication error from the application providing server 4. (Step 104), or waits for the intended bodily sensation game application to be returned (Step 105). Here, as the authentication code, the product package code displayed on the card previously acquired by purchasing the product package 7 is used.

  The application providing server 4 waits for the authentication code to be returned from the tablet-type terminal 6, and the predetermined authentication process (for example, the product package code sent as the authentication code is already sold, After determining whether or not the download is incomplete or meets the predetermined download permission condition (step 113), if the authentication is successful (step 114 YES), the designated sensation game is given to the tablet terminal 6 When the authentication application fails (step 114), the authentication error code is transmitted to the tablet terminal 6.

  On the tablet-type terminal 6 side, when an application for experiencing games is received from the application providing server 4, after receiving it (step 105), it is stored in a predetermined storage area (for example, the flash memory 6o). While waiting for a predetermined application activation operation and executing this, when an authentication error code is sent, the processing is immediately terminated.

<Model and form of tablet terminal and OS mechanism for image rotation>
One of the features of the present invention is that “the rotation angle of the game image is fixed at a specific rotation angle” means that the tablet terminal to which the present invention is applicable has various screen shapes and camera positions different from each other. In addition to the existence of models, it is considered that the general configuration of the OS related to image rotation control can be understood clearly only after knowing the general configuration of the OS. Therefore, an outline of the model and form of this type of tablet terminal and the mechanism of image rotation in the OS incorporated in those terminals will be described below.

-Positional relationship between the screen and sub camera for each smartphone model-
An explanatory diagram showing the positional relationship between the screen and the sub camera for each model of the smart phone is shown in FIG. As shown in FIGS. 4A to 4F, various types of smart phones 1 exist. However, regarding the shape of the screen 1a in the standard posture, the size and aspect ratio are somewhat different. Although a difference is recognized, as long as it is within the scope of the inventors' investigation, it was recognized that all models were vertically long. On the other hand, regarding the position of the shooting window 1b of the sub-camera, although it is superscript (arranged at the top of the screen 1a) in most models, as shown in FIG. It is recognized that there are also those with a mark (located at the bottom of the screen 1a).

-Positional relationship between screen and sub camera for each model of tablet PC-
An explanatory diagram showing the positional relationship between the screen and the sub camera for each model of tablet PC is shown in FIG. As shown in (a) to (c) of the figure, there are various types of tablet PCs 2, but when looking at the shape of the screen 2a in the standard posture, many models are vertically long. As shown in FIG. 5C, it was recognized that there was a horizontally long one. On the other hand, regarding the position of the sub-camera shooting window 2b, it was recognized that all models were superscript (arranged at the upper part of the screen 2a) as far as the present inventors' investigation scope was concerned.

-Positional relationship between screen and sub-camera for each model of tablet PC
FIG. 12 shows an explanatory diagram showing the positional relationship between the screen of the tablet PC and the sub camera. As shown in the figure, as far as the tablet PC 3 is concerned, it was recognized that the shape of the screen 3a in the standard posture is horizontally long in all models. On the other hand, as for the position of the shooting window 3b of the sub-camera, it was recognized that it was superscript (arranged in the upper part of the screen 3a) in all models as far as the present inventors' investigation range.

-OS configuration for image rotation control-
Various OSs such as Apple's iOS, Google's Android, and Microsoft's Windows (both are registered trademarks) are known as the OS in this type of tablet-type terminal 6. However, the screen rotation angle with respect to the standard orientation involves two pieces of setting information consisting of “app-side orientation setting” that can be set via an application and “terminal-side orientation setting” that can be set by operation on the terminal side. It is recognized that

As the contents of “application side orientation setting”, for example, the following six types of setting values can be selectively taken. In the old model, the setting value 3 and the setting value 4 may not exist.
・ Set value 1: Vertical (rotation angle 0 degrees with respect to standard rotation posture)
Setting value 2: Right side (rotation angle 90 degrees with respect to the standard rotation posture)
・ Set value 3: Vertical down (rotation angle 180 degrees with respect to the standard rotation posture)
Setting value 4: Left side (rotation angle 270 degrees with respect to the standard rotation posture)
・ Setting value 5: Sensor (according to the detected posture of the posture sensor such as an acceleration sensor)
・ Setting value 6: Terminal (according to the terminal orientation setting)

As the contents of “terminal side orientation setting”, for example, the following five types of setting values can be selectively taken. In the old model, the setting value 3 and the setting value 4 may not exist.
・ Set value 1: Vertical (rotation angle 0 degrees with respect to standard rotation posture)
Setting value 2: Right side (rotation angle 90 degrees with respect to the standard rotation posture)
・ Set value 3: Vertical down (rotation angle 180 degrees with respect to the standard rotation posture)
Setting value 4: Left side (rotation angle 270 degrees with respect to the standard rotation posture)
・ Setting value 5: Sensor (according to the detected posture of the posture sensor such as an acceleration sensor)

  In determining the final image display orientation, the content of the application side orientation setting has priority over the content of the terminal side orientation setting. That is, when the setting content of the application side azimuth setting is one of the setting values 1 to 5, the final image display azimuth is the setting content of the application side azimuth setting regardless of the setting content of the terminal side azimuth setting. It is determined according to the set values 1 to 5, which are The setting contents of the terminal side azimuth setting are valid only when the setting contents of the application side azimuth setting are the setting value 6 (“terminal”). That is, when the content of the application side azimuth setting is the setting value 6 (“terminal”), the final image display azimuth follows the setting content of the terminal side azimuth setting value (setting values 1 to 5).

<Operation to fix the rotation angle of the game image to a specific rotation angle>
Next, the configuration of the main part of the present invention will be described in detail on the premise of the model and form of the tablet-type terminal described above and the OS mechanism relating to image rotation.

  As described above, in the game application according to the present invention, an interactive game image using a tablet terminal as an input of the player's body action recognized via the sub camera is displayed on the display screen. Regardless of the content of the image display orientation set on the tablet-type terminal side and the means for displaying the tablet-type terminal 6, the game image related to the game application is displayed at a rotation angle of the game image with respect to the display screen. A “program portion” is included for functioning as a means for directing to be displayed on the display screen in a state in which is fixed at a specific rotation angle.

  Here, the “specific rotation angle” means that if the orientation of the display screen is determined so that the orientation on the image of the game image matches the orientation on the real space, the game image is displayed on the portrait screen or landscape screen. It is determined so that an appropriate screen rotation posture that satisfies the requirement on image development such as which one should be displayed is automatically obtained.

  Further, the “specific rotation angle” means that when the rotation orientation of the display screen is determined so that the orientation on the image of the game image matches the orientation on the real space, the body action of the player is changed to the shooting window. An appropriate screen rotation posture that also satisfies the requirements on the shooting posture of the body action such as from which angle the camera should be shot may be obtained automatically.

  The reason why the above-mentioned “program portion” can cause the tablet-type terminal 6 to function as a desired means is because the processing on the application side and the display processing on the OS side cooperate with each other.

-Description of application processing-
A flowchart for explaining the processing on the application side is shown in FIG. In the figure, when execution of an application is started by a predetermined application activation operation, first, initial processing (step 201) is executed.

  In this initial process (step 201), various processes for customizing (localizing) the game application downloaded from the application providing server 4 for the tablet-type terminal are performed. Rotation angle fixing processing relating to an image that is a main part of the invention, that is, the tablet type terminal 6 displays a game image related to the game application regardless of the content of the image display orientation setting set on the tablet type terminal side. A process for functioning as a means for directing the game image to be displayed on the display screen in a state where the rotation angle of the game image with respect to the screen is fixed to a specific rotation angle is executed (see the flowchart in FIG. 7). Here, the “specific rotation angle” means that if the orientation of the display screen is determined so that the orientation on the image of the game image matches the orientation on the real space, the game image is displayed on the portrait screen or landscape screen. It is determined so that an appropriate screen rotation posture that satisfies the requirement on image development such as which one should be displayed is automatically obtained.

The content of the rotation angle fixing process is slightly different depending on how far the model range of the tablet terminal in which the game application can operate is assumed. For example, if the assumed model range is narrowed down to a model in which the screen is vertically long and the shooting window is attached to the top of the screen in a standard posture (it includes a considerable number of models) (FIG. 7). Step 402 (vertically long and superscript)), the screen rotation angle to be fixed can be determined only by a request on the application side as described below (see Steps 403 and 404 and the flowchart of FIG. 8). In the flowchart of FIG. 7, step 401 is a process of reading from the application information that is pre-installed in the application and that specifies the target terminal specification.

Application request Image rotation angle

Long screen and camera top Rotation angle 0 degree (vertical)
Vertical screen with camera bottom Rotation angle 180 degrees (bottom)
Long screen and no camera specified Rotation angle 0 degree (upper) or 180 degree (lower)
Horizontal screen and camera top Rotation angle 90 degrees (right side) or 270 degrees (left side)
Horizontal screen and under camera rotation angle 90 degrees (right side) or 270 degrees (left side)
Horizontal screen and no camera specified Rotation angle 90 degrees (right side) or 270 degrees (left side)

  On the other hand, when the model range assumed is expanded to a model in which the screen is horizontally long in the standard posture, or is expanded to a model in which the shooting window is a subscript of the screen (Step 402 in FIG. 7). According to device usage ”), the screen rotation angle to be fixed cannot be determined uniquely only by the request on the app side, and it cannot be determined without checking the specifications on the terminal side each time ( (See steps 405, 406, 407, and 408 in FIG. 7).

  Here, the specifications of the terminal (vertical / horizontal in the standard orientation, superscript / subscript in the standard orientation) and the model information of the terminal (model number, code, etc., directly specifying the model) (Including any specific information that can indirectly identify the model, etc.) is uniquely determined, so that the relationship is obtained in advance and incorporated into each game application as a table. Then, based on the model information read from the installed terminal (step 405), by drawing the above table, the specifications of the terminal (vertical / horizontal in standard posture, superscript / in standard posture) Recognize the subscript (step 406), and then request the application (screen portrait and camera superscript, screen portrait and camera subscript, screen portrait and no camera designation) Rotation angle (vertical on the image satisfying no screen horizontally and camera specification), the vertical under the right side can be determined accurately left side) (step 407, 408).

That is, when it is determined that the screen is vertically long and the shooting window is a subscript of the screen based on the model information (YES in step 603 in FIG. 9), the rotation angle of the image is as follows. (Step 604).

Application request Image rotation angle

Long screen and camera top Rotation angle 180 degrees (bottom)
Vertical screen and with a camera rotation angle 0 degree (vertical)
Long screen and no camera specified Rotation angle 0 degree (upper) or 180 degree (lower)
Horizontal screen and camera top Rotation angle 90 degrees (right side) or 270 degrees (left side)
Horizontal screen and under camera rotation angle 90 degrees (right side) or 270 degrees (left side)
Horizontal screen and no camera specified Rotation angle 90 degrees (right side) or 270 degrees (left side)

Further, when it is determined that the screen is horizontally long in the standard posture and the shooting window is the top of the screen based on the model information (YES in step 605 in FIG. 9), the rotation angle of the image is as follows. (Step 606).

Application request Image rotation angle

Vertical screen and camera top Rotation angle 90 degrees (right side) or 270 degrees (left side)
Screen portrait and camera subordinate Rotation angle 90 degrees (right side) or 270 degrees (left side)
Long screen and no camera specified Rotation angle 90 degrees (right side) or 270 degrees (left side)
Horizontal screen and camera top Rotation angle 0 degree (vertical top)
Horizontal screen with camera under rotation angle 180 degrees (bottom down)
Horizontal screen and no camera specified Rotation angle 0 degree (upper) or 180 degree (lower)

When it is determined that the screen is horizontally long and the shooting window is a subscript of the screen based on the model information (see step 607 in FIG. 9), the image rotation angle is (Step 608).

Application request Image rotation angle

Vertical screen and camera top Rotation angle 90 degrees (right side) or 270 degrees (left side)
Screen portrait and camera subordinate Rotation angle 90 degrees (right side) or 270 degrees (left side)
Long screen and no camera specified Rotation angle 90 degrees (right side) or 270 degrees (left side)
Horizontal screen and camera top Rotation angle 180 degrees (bottom)
Horizontal screen and under camera rotation angle 0 degree (vertical)
Horizontal screen and no camera specified Rotation angle 0 degree (upper) or 180 degree (lower)

  Thereafter, the content of the above-mentioned “application side orientation setting” is updated by the content of the rotation angle of the image thus determined. As a result, as described later, when the display processing on the OS side is executed, the content is updated in this way. By referring to the content of “application side orientation setting” later (see step 301 in FIG. 6), the display orientation of the game image displayed on the display screen is the orientation on the image of the game image and the real space. When the rotation orientation of the display screen is determined so that the orientations coincide with each other, a request for image development such as whether the game image should be displayed on a vertically long screen or a horizontally long screen and / or on the image of the game image If the rotation orientation of the display screen is determined so that the orientation of the player and the orientation in the real space coincide with each other, from which angle the player's body action should be taken by the camera through the shooting window Proper screen rotation attitude that also meet the requirements of on the photographing posture of the body actions saying will be automatically obtained. As is clear from the above example, regarding a model in which the screen is horizontally long in the standard posture and the shooting window is superscript or subscript (often in a tablet PC), if this is a vertically long posture, The shooting window is located on the right or left side, and the application's request cannot be fully satisfied. However, this can be solved to some extent by a separate camera side view calibration process. So it does n’t matter so much.

  In this way, when the initial process is completed (step 201), the original process (steps 202 to 208) of the sensory video game apparatus is executed, and the player's body action recognized through the camera is input. An interactive game image is displayed on the display screen of the image display.

  That is, in FIG. 5, when the game image generation and display instruction process (step 202) is executed, for example, as shown in FIG. 18, an interactive game image G (this example) is displayed on the screen 1a of the smartphone 1. Then, a moving image corresponding to a ball driving action by a table tennis opponent is displayed, and then a recognition process of a scheduled body action (in this example, a body action corresponding to an action of hitting a ball with a racket) ( Step 203) is executed.

  In this body action recognition process (step 203), for example, an actual body action (by a player) (based on a moving image shot by the sub camera via the shooting window 1b and appropriately edited by the image processing engine) For example, the movement of a bright spot (LED marker) corresponding to a racket swing performed for hitting a ball) and a bright spot corresponding to a presumed body action (for example, a racket swing when the ball is successfully hit) The collation with the movement of the (LED marker) is executed with a predetermined allowable width and a predetermined allowable time (step 205).

  Here, if the matching match of both body actions is confirmed within a predetermined time (NO in step 205) and with a predetermined allowable range, it is determined that the assumed body action has been detected (YES in step 204), and An image generation and display instruction corresponding to the detection action (step 206) is executed. In this process (step 206), for example, after the ball bounced back by the player flies toward the opponent, the moving image corresponding to the movement of the ball bounced back from the opponent toward the player An image generation and display instruction is performed.

  By repeating the above processing (steps 203, 204, 206) on the condition that the game is continuing (NO in step 208), a virtual table tennis rally between the player and the opponent can be enjoyed.

  On the other hand, because the body action of the player is unsuccessful, the assumed body action is not detected within a predetermined time (step 204 NO, step 205 YES), and then the assumed action is not detected. A corresponding image generation and display instruction (step 207) is executed. In this process (step 207), for example, after the player misses the ball, the opponent performs a serve, and an instruction to generate and display a moving image corresponding to the movement in which the ball is shot is performed.

  In FIG. 18, reference numeral 9 denotes a playground equipment cursor indicating the position of a bright spot (LED marker) in a moving image acquired by an image sensor of a sub camera. Based on the position of the playground equipment cursor 9 on the screen, the player can learn an appropriate swing while confirming what kind of swing the player's racket swing in the real space corresponds to in the game image. it can.

  FIG. 17 shows an explanatory diagram of the table tennis game during play. Since the tablet-type terminal 6 can be easily carried anytime and anywhere, as shown in the figure, the tablet-type terminal 6 is leaned on an appropriate desk 10 with an appropriate support 12 at an appropriate angle, and then the chair 11 It is possible to easily enjoy this kind of sensation-type video game anytime and anywhere, as the player 13 operates the play equipment 8 toward the screen while sitting down.

-Explanation of display processing on the OS side-
A flowchart for explaining the display processing on the OS side is shown in FIG. This process is executed each time a display instruction is given in the flowchart shown in FIG. 5 (for example, steps 202, 206, 207, etc.).

  In the figure, when processing is started, the contents of “application side orientation setting” are read (step 301). Here, as described above, the setting contents of the “app-side orientation setting” are as follows. In the initial process (step 201) in the flowchart of FIG. The rotation angle is set to any one of a rotation angle of 90 degrees with respect to the standard rotation attitude, a vertical direction (rotation angle of 180 degrees with respect to the standard rotation attitude), and a left side (rotation angle with respect to the standard rotation attitude of 270 degrees).

  Therefore, the subsequent determination process of “according to terminal setting” is denied (NO in step 302), and the image orientation determination process (steps 306 to 313) according to the setting content of “application side orientation setting” is immediately executed. It becomes.

  That is, if the setting content of “application side orientation setting” is “right side” (step 306 YES), right direction display processing (step 310) is executed, and if the setting content is “left side” (step 307 YES). If the left direction display process (step 311) is executed and the setting content is “vertically up” (step 308 YES), the vertically upward direction display process (step 312) is executed, and the setting content is “vertically down”. If there is (YES in step 309), the vertically downward display process (step 313) is executed.

  Here, in the display processing in each direction (steps 310 to 313), the image data in the buffer memory that stores the image data corresponding to the generated image in the direction corresponding to the reference orientation of the screen is transferred to the video RAM. The image data orientation is rotated so that the image orientation is changed in the specified direction ("right side", "left side", "upside", "downside"). The Rukoto.

  Although it is not directly related to the present invention, since the setting content of “application side orientation setting” is normally set to “according to terminal setting” in the default, in that case The determination process of “according to the terminal side setting” is affirmed (YES in step 302), and thereafter, the image orientation determination process according to the setting content of the “terminal side orientation setting” is executed.

  That is, after reading the setting content of “terminal side orientation setting” (step 303), if it is not “according to sensor value” (step 304NO), as described above, “terminal side orientation setting” The image orientation determination process (steps 306 to 313) according to the setting contents is executed. Therefore, if the setting content of “terminal side orientation setting” is set in any direction by performing some manual operation on the terminal side, image orientation determination processing in that direction (steps 306 to 306). 313) is executed.

  In addition, since the setting content of “terminal side orientation setting” is normally set to “follow the detection value of a sensor (for example, an attitude sensor such as acceleration)”, in that case, The determination process asking whether to “follow the sensor value” is affirmed (YES in step 304), and thereafter, the sensor value reading process (step 305) is performed, so that the image always faces upward according to the terminal posture at that time. An image orientation determination process (steps 306 to 313) is executed. Therefore, for users who are unfamiliar with the terminal operation, the setting content of “terminal orientation setting” is usually often “following the detection value of the sensor”. According to the terminal orientation, the image displayed on the screen rotates rapidly, and even if the application side has any intention to specify the direction, it is difficult for the user to know this.

<Specific Example of Image Rotation Processing According to the Present Invention>
In the following, according to the present invention, what kind of image rotation processing occurs according to the request on the application side and the specification on the terminal side will be described with some specific examples.

-Specific Example 1-
FIG. 13 shows an example of screen rotation when the request on the application side is a vertically long screen and attached to the camera, and the specification on the terminal side is the vertically long screen and attached to the camera in the standard posture.

  As shown in the figure, in this case, the rotation angle of the screen is fixed at 0 degree (vertically upward). Then, as shown in FIG. 5A, the orientation of the image G matches the orientation of the real space only when the orientation of the terminal 1 that is a smartphone (the orientation of the screen 1a) is the standard orientation. Since the image G is displayed in an appropriate state, the orientation of the image G does not match the orientation of the real space in the other screen orientations shown in FIGS. The image G is not displayed in an appropriate state. Therefore, the user simply selects the rotation orientation of the screen 1a so that the orientation of the image G matches the orientation of the real space, and the orientation of the screen 1a requested by the application (vertically long) and the camera 1b alone. A position (superscript) can be acquired.

-Specific Example 2-
FIG. 14 shows an example of screen rotation when the application side request is a portrait screen and camera superscript, and the terminal side specification is a portrait screen and camera subscript in the standard posture.

  As shown in the figure, in this case, the rotation angle of the screen is fixed to 180 degrees (vertically below). Then, as shown in FIG. 5C, the orientation of the image G and the orientation of the real space are the same only when the orientation of the terminal 1 that is a smartphone (the orientation of the screen 1a) is a 180-degree rotational orientation. Then, while the image G is displayed in an appropriate state, the orientation of the image G and the real space of the other screen postures shown in FIGS. (A), (b), and (d) are the same. Since it does not coincide with the azimuth, the image G is not displayed in an appropriate state. Therefore, the user simply selects the rotation orientation of the screen 1a so that the orientation of the image G matches the orientation of the real space, and the orientation of the screen 1a requested by the application (vertically long) and the camera 1b alone. A position (superscript) can be acquired.

-Specific Example 3-
FIG. 15 shows an example of screen rotation when a request on the application side is a vertically long screen and attached to the camera, and a specification on the terminal side is a horizontally long screen and attached to the camera in the standard posture.

  As shown in the figure, in this case, the rotation angle of the screen is fixed at 270 degrees (left side). Then, as shown in FIG. 5B, the orientation of the image G and the orientation of the real space are identical only when the orientation of the terminal 2 that is the tablet PC (the orientation of the screen 2a) is a 90-degree rotational orientation. Then, while the image G is displayed in an appropriate state, the orientation of the image G and the real space of the other screen postures shown in FIGS. (A), (c), and (d) are the same. Since it does not coincide with the azimuth, the image G is not displayed in an appropriate state. Therefore, the user simply selects the rotation orientation of the screen 1a so that the orientation of the image G matches the orientation of the real space, and the orientation of the screen 2a requested by the application side (vertically long) and the camera 2b alone. A position (superscript) can be acquired.

  At this time, the camera position does not exactly satisfy the request on the application side, but this can be supplemented by an automatic calibration process on the camera side.

-Specific Example 4-
FIG. 16 shows an example of screen rotation when the request on the application side is a vertically long screen and attached to the camera, and the specification on the terminal side is the vertically long screen and attached to the camera in the standard posture.

  As shown in the figure, in this case, the rotation angle of the screen is fixed at 270 degrees (left side). Then, as shown in FIG. 5B, the orientation of the image G and the orientation of the real space are the same only when the orientation of the terminal 1 that is a smartphone (the orientation of the screen 1a) is a 90-degree rotational orientation. Then, while the image G is displayed in an appropriate state, the orientation of the image G and the real space of the other screen postures shown in FIGS. (A), (c), and (d) are the same. Since it does not coincide with the azimuth, the image G is not displayed in an appropriate state. Therefore, the user simply selects the rotation orientation of the screen 1a so that the orientation of the image G matches the orientation of the real space, and the orientation of the screen 2a requested by the application side (vertically long) and the camera 2b alone. A position (superscript) can be acquired.

  At this time, the camera position does not exactly satisfy the request on the application side, but this can be supplemented by an automatic calibration process on the camera side.

<Others>
-Contents of body action-
The content of the body action is different for each game. For example, in the case of a body action corresponding to the action of hitting a ball such as tennis, table tennis, or baseball, a predetermined target portion in a specific area on the camera field of view ( For example, if it is a body action corresponding to an aiming action in a shooting game (such as a racket, a bat, etc.) Aiming (position and orientation).

-Recognizing method of a predetermined target part-
There are also various methods for recognizing a predetermined target portion, and some of them can be recognized directly from an image using shape recognition technology, while others of point markers (for example, bright spots of light emitting diodes) held by the player are used. Some rely on images and recognize them indirectly. In the example of FIG. 20, the locus of the racket tip can be easily recognized by incorporating the high-intensity LED 8a so that it can be seen from the front and back and the tip of the racket as a play equipment. FIG. 5A shows a hand-held table tennis play equipment constructed in a normal racket size, and FIG. 5B shows a finger-push type play table tennis play equipment constructed in a small size. Also, the bat which is the baseball play equipment 8 shown in FIG. 19B has a color (for example, one of the primary colors red, green, or blue) that can be distinguished from the surrounding colors.

-Image rotation processing-
In the previous embodiment (steps 310, 311, 312 and 313 in FIG. 6), the image orientation at the time of game image generation is assumed to correspond to the standard orientation of the screen, and from the image buffer memory to the video RAM. When transferring to, the OS side is configured to perform screen rotation processing according to the screen height and width designation requested by the application side. Conversely, when the game image is generated, the application side requests it. The same image rotation is fixed even if the screen rotation processing is executed in advance according to the vertical and horizontal designation of the screen, and the image rotation processing is not performed when transferring from the image buffer memory to the video RAM. Processing can be easily realized. Furthermore, when the game image is generated, a plurality of types of images having different directions are generated in advance, and when transferring from the image buffer memory to the video RAM, one of the images is selected. In addition, the image rotation processing in the designated direction can be realized. In the case of generating and holding a plurality of types of images as described above, when the present invention is applied to an old tablet type terminal in which the screen rotation function is limited to two types of directions (vertical and vertical) , Especially effective.

-Other customization processes-
In the present invention, by preparing a common game application for as many terminal models as possible, it is intended to reduce the cost on the application producer side and avoid the complexity of the model declaration on the user side, Of course, in order to apply it to terminals with different sizes and specifications in a common game application, other necessary customization processing (screen size and aspect ratio adjustment, camera field of view calibration, etc.) is required for each terminal. Of course, since some known methods have already been established for the customization process, it is practically problematic to appropriately combine these established known methods. There is no.

  According to the present invention, in a bodily sensation type video game apparatus using a tablet terminal and a game application, any user of the tablet terminal can easily enjoy a game with a simple operation.

1 Smartphone 1a Display screen 1b Sub camera shooting window 2 Tablet PC
2a Display screen 2b Sub-camera shooting window 3 Tablet / notebook PC
3a Display screen 3b Sub camera shooting window 3A Tablet part 3B Keyboard part 4 Application providing server 4a Control unit 4b Storage unit 4c Communication unit 4d Display unit 4e I / F unit 4f Input operation unit 5 Internet 6 Tablet type terminal 6a Microcomputer 6b Microphone 6c Speaker 6d Touch panel control unit 6e Touch panel 6f Vibrator 6g Lamp drive unit 6h LED lamp 6i Main camera module 6j Sub camera module 6k Wireless communication unit 6l Key input device 6m Display controller 6n Image display unit 6o Flash memory 6p RAM
7 Package 8 Playground equipment 8a High-intensity LED constituting a point marker
9 Playground equipment marker 10 Desk 11 Chair 12 Support tool 13 Player G Image

Claims (20)

An image display having a display screen; a camera for photographing a body action of a player; and a game computer on which an application program for a sensation-type video game is to be installed. An interactive video game apparatus in which an interactive game image is displayed on the display screen of the image display device by executing a program and inputting a player's body action recognized through a camera. And
The image display device, the camera, and the game computer each have a display screen, such as a smartphone, a tablet PC, and the like, and a camera with a shooting window facing the front side of the display screen. A game according to the application program, which is composed of a commercially available tablet terminal incorporated therein, and the application program includes the tablet terminal regardless of the content of the image display orientation setting set on the tablet terminal side. A program portion for causing the image to function as a means for displaying the image on the display screen in a state where the rotation angle of the game image with respect to the display screen is fixed to a specific rotation angle; The rotation angle of the game image coincides with the orientation on the image of the game image and the orientation on the real space. When the rotation orientation of the display screen is determined in this way, an appropriate screen rotation orientation that satisfies the requirements for image development such as whether the game image should be displayed on a portrait screen or landscape screen is automatically obtained. An experience-type video game apparatus that is defined as described above.   When the rotation orientation of the display screen is determined such that the orientation on the image of the game image and the orientation on the real space coincide with the specific rotation angle, the body action of the player is changed to the shooting window. 2. It is determined so that an appropriate screen rotation posture can be automatically obtained that satisfies the photographing posture requirement of the body action such as from which angle the camera should be photographed via The experience-type video game device described in 1.   The bodily sensation type according to claim 1, wherein the specific rotation angle is determined based on a screen vertical / horizontal request on the application program side and a screen vertical / horizontal specification of the terminal derived from model information of the tablet terminal. Video game device.   The specific rotation angle is based on the screen height request and shooting window position request on the application program side, and the screen height and width specifications and shooting window position specifications of the terminal derived from the model information of the tablet terminal. The bodily sensation type video game device according to claim 2, which is determined. An application program to be installed on a commercially available tablet terminal having a display screen integrated therein, such as a smart phone, a tablet PC, etc., and a camera with a shooting window facing the front side of the display screen. Because
The tablet terminal
Means for displaying an interactive game image on the display screen, with the player's body action recognized via the camera as an input;
Regardless of the content of the image display orientation setting set on the tablet-type terminal side, the game image according to the application program, with the rotation angle of the game image relative to the display screen fixed to a specific rotation angle, The display screen is configured to function as a means for directing display on a display screen, and the specific rotation angle is such that an orientation on an image of the game image and an orientation on a real space coincide with each other. When the rotation orientation of the game is determined, an appropriate screen rotation orientation that satisfies the requirements for image development such as whether the game image should be displayed on a portrait screen or landscape screen is automatically obtained. Is an application program.   When the rotation orientation of the display screen is determined such that the orientation on the game image matches the orientation on the real space with the specific rotation angle, the body action of the player is changed to the shooting window. 6. An appropriate screen rotation posture that automatically satisfies the requirements on the shooting posture of the body action, such as from which angle the camera should be shot, is determined automatically. Application program described in. The specific angle of rotation, and the screen aspect of the application program side required is determined based on the screen aspect specifications of the terminal derived from the model information of the tablet terminal, an application program according to claim 5 . The specific rotation angle is based on the screen height request and shooting window position request on the application program side, and the screen height and width specifications and shooting window position specifications of the terminal derived from the model information of the tablet terminal. The application program according to claim 6, which is determined. An application program to be executed by a commercially available tablet terminal having a display screen integrated therein, such as a smart phone, a tablet PC, etc., and incorporating a camera whose shooting window faces the front side of the display screen A storage medium storing
The application program includes
The tablet terminal
Means for displaying an interactive game image on the display screen, with the player's body action recognized via the camera as an input;
Regardless of the content of the image display orientation setting set on the tablet-type terminal side, the game image according to the application program, with the rotation angle of the game image relative to the display screen fixed to a specific rotation angle, The display screen is configured to function as a means for directing display on a display screen, and the specific rotation angle is such that an orientation on an image of the game image and an orientation on a real space coincide with each other. When the rotation orientation of the game is determined, an appropriate screen rotation orientation that satisfies the requirements for image development such as whether the game image should be displayed on a portrait screen or landscape screen is automatically obtained. A recording medium.   When the rotation orientation of the display screen is determined such that the orientation on the game image matches the orientation on the real space with the specific rotation angle, the body action of the player is changed to the shooting window. 10. An appropriate screen rotation posture satisfying the requirement on the shooting posture of the body action such as from which angle the camera should be shot through is determined automatically. The recording medium described in 1. The recording medium according to claim 9, wherein the specific rotation angle is determined based on a screen vertical / horizontal request on the application program side and a screen vertical / horizontal specification of the terminal derived from model information of the tablet terminal. . The specific rotation angle is based on the screen height request and shooting window position request on the application program side, and the screen height and width specifications and shooting window position specifications of the terminal derived from the model information of the tablet terminal. The recording medium according to claim 10, wherein the recording medium is determined . A program providing server;
Smart phone, as such a tablet PC, which has a display screen integrally, photographing window and commercial tablet terminal camera directed to the front side is incorporated in the display screen, connected by a network A client and server system,
In the program providing server,
The tablet terminal
Means for displaying an interactive game image on the display screen, with the player's body action recognized via the camera as an input;
Regardless of the content of the image display orientation setting set on the tablet-type terminal side, the game image according to the application program, with the rotation angle of the game image relative to the display screen fixed to a specific rotation angle, The display screen is configured to function as a means for directing display on a display screen, and the specific rotation angle is such that an orientation on an image of the game image and an orientation on a real space coincide with each other. When the rotation orientation of the game is determined, an appropriate screen rotation orientation that satisfies the requirements for image development such as whether the game image should be displayed on a portrait screen or landscape screen is automatically obtained. 1 or 2 or more types of application programs are stored, and the program providing server includes:
In response to a download request coming from the tablet terminal via the network, an application program designated by the tablet terminal is selected from the one or more application programs. It is configured to send to the tablet type terminal on the condition of authentication,
The tablet type terminal
Receiving and storing the application program transmitted from the program providing server in response to the download request and executing the application program on the condition of a predetermined activation operation, the application program is recognized through the camera. A client and server system configured to display an interactive game image on the display screen using a player's body action as an input.   When the rotation orientation of the display screen is determined such that the orientation on the game image matches the orientation on the real space with the specific rotation angle, the body action of the player is changed to the shooting window. 14. An appropriate screen rotation posture that satisfies the requirement on the photographing posture of the body action, such as from which angle the camera should be photographed via, is automatically determined. Client and server system as described in. The client-and-client according to claim 13, wherein the specific rotation angle is determined based on a screen vertical / horizontal request on the application program side and a screen vertical / horizontal specification of the terminal derived from model information of the tablet terminal. Server system . The specific rotation angle is based on the screen height request and shooting window position request on the application program side, and the screen height and width specifications and shooting window position specifications of the terminal derived from the model information of the tablet terminal. The client and server system of claim 14, wherein the client and server system is determined . A program providing server, scan Matohon, as such a tablet PC, which has integrally a display screen, photographing window and commercial tablet terminal camera directed to the front side is incorporated in the display screen The program providing server constituting a client and server system connected via a network,
In the program providing server,
The tablet terminal
Means for displaying an interactive game image on the display screen, with the player's body action recognized via the camera as an input;
Regardless of the content of the image display orientation setting set on the tablet-type terminal side, the game image according to the application program, with the rotation angle of the game image relative to the display screen fixed to a specific rotation angle, The display screen is configured to function as a means for directing display on a display screen, and the specific rotation angle is such that an orientation on an image of the game image and an orientation on a real space coincide with each other. When the rotation orientation of the game is determined, an appropriate screen rotation orientation that satisfies the requirements for image development such as whether the game image should be displayed on a portrait screen or landscape screen is automatically obtained. 1 or 2 or more types of application programs are stored, and
The program providing server includes:
In response to a download request coming from the tablet terminal via the network, an application program designated by the tablet terminal is selected from the one or more application programs. It is configured to send to the tablet type terminal on the condition of authentication,
The tablet type terminal
Receiving and storing the application program transmitted from the program providing server in response to the download request and executing the application program on the condition of a predetermined activation operation, the application program is recognized through the camera. A program providing server configured to display an interactive game image on the display screen using a player's body action as an input. When the rotation orientation of the display screen is determined such that the orientation on the game image matches the orientation on the real space with the specific rotation angle, the body action of the player is changed to the shooting window. 18. An appropriate screen rotation posture that satisfies the requirement on the photographing posture of the body action such as from which angle the camera should be photographed via is determined so as to be automatically obtained. The program providing server described in 1. The program provision according to claim 17, wherein the specific rotation angle is determined based on a screen height / width request on the application program side and a screen height / width specification of the terminal derived from model information of the tablet terminal. Server . An application from a program providing server to a commercially available tablet terminal having a display screen integrated therein and having a camera with a shooting window facing the front side of the display screen, such as a smart phone or a tablet PC. A toy set for a bodily sensation video game that is performed by downloading a program and executing the application program on the tablet-type terminal,
The program providing server and the tablet terminal are connected via a network,
In the program providing server,
The tablet terminal
Means for displaying an interactive game image on the display screen, with the player's body action recognized via the camera as an input;
Regardless of the content of the image display orientation setting set on the tablet-type terminal side, the game image according to the application program, with the rotation angle of the game image relative to the display screen fixed to a specific rotation angle, Function as a means to direct display on the display screen, and
When the rotation orientation of the display screen is determined such that the orientation on the image of the game image and the orientation on the real space coincide with the specific rotation angle, the game image is displayed as either a portrait screen or a landscape screen. One or more types of application programs are stored, which are determined so as to automatically obtain a proper screen rotation posture that satisfies the image development requirement that should be displayed on the screen. ,further
The program providing server includes:
In response to a download request coming from the tablet terminal via the network, an application program designated by the tablet terminal is selected from the one or more application programs. It is configured to send to the tablet type terminal on the condition of authentication,
The tablet type terminal
Receiving and storing the application program transmitted from the program providing server in response to the download request and executing the application program on the condition of a predetermined activation operation, the application program is recognized through the camera. It is configured to display an interactive game image on the display screen using the player's body action as an input,
The toy set is
A play equipment to be gripped by the player or fixed to a part of the player's body and serving as a detection criterion for body action;
An authentication code display card for displaying an authentication code used for the predetermined authentication for transmitting the application program designated from the tablet terminal to the tablet terminal;
A toy set for experiencing video games.

Images