JP6454041B1 - GAME DEVICE AND GAME SYSTEM - Google Patents

Abstract

An operation of a player in a virtual space is supported. A seat for a player to sit, a head mount device that can be mounted on the head of the player, a controller that generates a game input according to the movement of the player's legs or other parts, A game device is provided that includes an audio output unit that outputs audio information when audio information addressed to a player is received. [Selection] Figure 11

Description

  The present disclosure relates to a game device and a game system for providing a virtual experience.

  In recent years, various arcade games that allow a user to experience virtual reality using a head-mounted device (HMD) are provided at amusement facilities and the like.

JP 2017-188827 A

  In such facilities, before the start of game play, instructors often explain how to play the game and how to operate the device to the user. However, even if an explanation is given in advance from an instructor, it is not always possible for the user to perform an appropriate operation during game play. In particular, since the user is deprived of the field of view in the real space by wearing the head mounted device, it tends to be difficult to perform an appropriate operation as compared with a normal game.

  The present invention has been made in view of the above points, and one of its purposes is to assist a user who plays a game in a virtual space with an operation during game play.

  In order to solve the above-described problem, one aspect of the present disclosure provides a riding unit for a player to ride, a head-mounted device that can be attached to the player's head, and body movements excluding the player's head. A controller that generates a game input according to the game, a processor that displays a posture image of the head mounted device and a view field image corresponding to the game input from the controller on the head mounted device, and audio information for attending the player. And a sound output unit that outputs the sound information when received.

  ADVANTAGE OF THE INVENTION According to this invention, operation in game play can be assisted with respect to the user who plays the game on virtual space.

1 schematically shows the configuration of an HMD system 100. FIG. 2 is a block diagram illustrating an example of a basic hardware configuration of a computer according to an embodiment. It is a figure which represents notionally the uvw visual field coordinate system set to HMD by one Embodiment. FIG. 3 is a diagram conceptually illustrating an aspect of expressing a virtual space according to an embodiment. It is a figure showing the head of the user who wears HMD110 from the top by one embodiment. It is a figure showing the yz cross section which looked at the visual recognition area from the x direction in virtual space. It is a figure showing the xz cross section which looked at the visual recognition area from the y direction in virtual space. It is a figure showing the schematic structure of the controller by one Embodiment. It is a block diagram which shows the function of the computer for implement | achieving the display process of the virtual space, etc. in the HMD system by one Embodiment. It is a flowchart of the general process for displaying the image of the virtual space where a user immerses on a display part. 1 is a diagram schematically illustrating a configuration of a system according to an embodiment of the present disclosure. FIG. 2 is a flowchart of a method 1200 according to an embodiment of the present disclosure. 4 is a flowchart of a method 1300 according to another embodiment of the present disclosure.

[Description of Embodiment of Present Disclosure]
First, the contents of the embodiment of the present disclosure will be listed and described. One embodiment of the present disclosure includes the following configuration.

  (Item 1) A riding part for a player to ride, a head mount device that can be mounted on the player's head, a controller that generates game input in accordance with the movement of the body excluding the player's head, A processor that displays on the head mounted device a view image corresponding to the posture of the head mounted device and a game input from the controller, and a sound that outputs the sound information when the sound information for attending the player is received. An output unit.

(Item 2)
The riding part is a seating part for the player to sit, and is arranged in the vicinity of the seating part, and is configured to physically contact the player's body when the audio information is received. Item 4. The game device according to Item 1, further comprising a movable doll.

(Item 3)
Item 3. The game device according to Item 2, wherein the movable doll is configured to change a position in contact with the player according to the content of the audio information.

(Item 4)
4. The game apparatus according to any one of items 1 to 3, wherein the voice information is a voice instruction issued by an attendant to guide a game operation to the player.

(Item 5)
The audio information includes first audio information and second audio information, and the first audio information is generated or reproduced by a computer according to the game play of the player, and guides a game operation to the player. The game apparatus according to any one of items 1 to 3, wherein the second voice information is a voice comment for a game play performed by the player, which is issued by an attendant.

(Item 6)
The audio information includes first audio information and second audio information, and the first audio information is an audio instruction issued by a secondary attendant to guide the game operation to the player. 4. The game apparatus according to any one of items 1 to 3, wherein the voice information is a voice comment issued by an attendant to a game play performed by the player.

(Item 7)
Item 7. The game device according to Item 5 or 6, wherein the movable doll is configured to physically contact the player's body when the second audio information addressed to the player is received.

(Item 8)
A voice recognition unit that recognizes the voice instruction or the voice comment issued by the attendant; and a control unit that controls the movement of the movable doll based on a result of voice recognition by the voice recognition unit. 8. The game device according to any one of items 4 to 7.

(Item 9)
8. The game device according to any one of items 4 to 7, wherein the movable doll is configured to move according to a manual operation by the attendant.

(Item 10)
Each of the plurality of game devices described in any one of Items 4 to 9 is connected to each of the plurality of game devices in a communicable manner, and is generated for each player generated according to the game play of the player in each game device. A display device that presents a visual field image to the attendant, a voice input device for inputting a voice instruction or voice comment from the attendant, and an output destination of the voice instruction or voice comment by the operation of the attendant A game system comprising: a switching device capable of selecting at least one of a plurality of game devices.

(Item 11)
Each of the plurality of game devices includes an audio input unit that receives an audio instruction or an audio comment input from a player of the game device to a player of another game device after the game play on the game device ends. The described game system.

(Item 12)
Item 12. The game system according to Item 11, wherein the head mounted device in each of the plurality of game devices is configured to display a view field image in another game device after the game play in the game device ends.

[Details of Embodiment of the Present Disclosure]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  A configuration of a head-mounted device (HMD) system 100 will be described with reference to FIG. FIG. 1 schematically shows the configuration of the HMD system 100. In one example, the HMD system 100 is provided as a home system or a business system. The HMD may be a so-called head mounted display including a display unit, or may be a head mounted device on which a terminal such as a smartphone having the display unit can be attached.

  The HMD system 100 includes an HMD 110 (image display device), a tracking sensor 120, a controller 160, and a computer 200. The HMD 110 includes a display unit 112 and a gaze sensor 140. The controller 160 may include a motion sensor 130.

  In one example, the computer 200 may be connectable to a network 192 such as the Internet, and may be communicable with a computer such as a server 150 connected to the network 192. In another aspect, the HMD 110 may include a sensor 114 instead of the tracking sensor 120.

  The HMD 110 may be worn on the head of the user 190 and provide a virtual space to the user during operation. More specifically, the HMD 110 displays an image for the right eye and an image for the left eye on the display unit 112, respectively. When each eye of the user visually recognizes each image, the user can recognize the image as a three-dimensional image based on the parallax of both eyes.

  The display unit 112 is realized as, for example, a non-transmissive display device. In one example, the display unit 112 is disposed on the main body of the HMD 110 so as to be positioned in front of both eyes of the user. Therefore, when the user visually recognizes the three-dimensional image displayed on the display unit 112, the user can be immersed in the virtual space. In an embodiment, the virtual space includes, for example, a background, an object that can be operated by the user, an image of a menu that can be selected by the user, and the like. In an embodiment, the display unit 112 may be realized as a liquid crystal display unit or an organic EL (Electro Luminescence) display unit included in an information display terminal such as a smartphone.

  In one example, the display unit 112 may include a sub display unit for displaying an image for the right eye and a sub display unit for displaying an image for the left eye. In another aspect, the display unit 112 may be configured to display an image for the right eye and an image for the left eye as a single unit. In this case, the display unit 112 includes a high-speed shutter. The high-speed shutter operates so that an image for the right eye and an image for the left eye can be displayed alternately so that the image is recognized only by one of the eyes.

  In one example, the HMD 110 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. The tracking sensor 120 has a position tracking function for detecting the movement of the HMD 110. More specifically, the tracking sensor 120 may read a plurality of infrared rays emitted from the HMD 110 and detect the position and inclination of the HMD 110 in the real space.

  In an aspect, the tracking sensor 120 may be realized by a camera. In this case, the tracking sensor 120 can detect the position and inclination of the HMD 110 by executing image analysis processing using image information of the HMD 110 output from the camera. The tracking sensor 120 may be configured to detect the position of a part of the user's body (for example, the head or hand) by analyzing the user's image captured by the camera.

  In another aspect, the HMD 110 may include a sensor 114 instead of the tracking sensor 120 as a position detector. The HMD 110 can detect the position and inclination of the HMD 110 itself using the sensor 114. For example, when the sensor 114 is an angular velocity sensor, a geomagnetic sensor, an acceleration sensor, a gyro sensor, or the like, the HMD 110 detects its own position and inclination using any one of these sensors instead of the tracking sensor 120. obtain. As an example, when the sensor 114 is an angular velocity sensor, the angular velocity sensor detects angular velocities around the three axes of the HMD 110 in real space over time. The HMD 110 calculates a temporal change in the angle around the three axes of the HMD 110 based on each angular velocity, and further calculates an inclination of the HMD 110 based on the temporal change in the angle. The HMD 110 may include a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transmissive display device by adjusting the transmittance. Moreover, the view field image may include a configuration for presenting the real space in a part of the image configuring the virtual space. For example, an image captured by a camera mounted on the HMD 110 may be displayed so as to be superimposed on a part of the field-of-view image, or a part of the field-of-view image is set by setting a high transmittance of a part of the transmissive display device. Real space may be visible from a part.

  The gaze sensor 140 detects a direction (line of sight) in which the lines of sight of the right eye and the left eye of the user 190 are directed. The detection of the direction is realized by, for example, a known eye tracking function. The gaze sensor 140 is realized by a sensor having the eye tracking function. In one aspect, the gaze sensor 140 preferably includes a right eye sensor and a left eye sensor. The gaze sensor 140 may be, for example, a sensor that irradiates the right eye and the left eye of the user 190 with infrared light and detects the rotation angle of each eyeball by receiving reflected light from the cornea and iris with respect to the irradiated light. . The gaze sensor 140 can detect the line of sight of the user 190 based on each detected rotation angle.

  Server 150 may send a program to computer 200. In another aspect, the server 150 may communicate with other computers 200 for providing virtual reality to HMDs used by other users. For example, when a plurality of users play a participatory game in an amusement facility, each computer 200 communicates a signal based on each user's operation with another computer 200, and a plurality of users are common in the same virtual space. Allows you to enjoy the game.

  The controller 160 is connected to the computer 200 by wire or wireless. The controller 160 receives input of commands from the user 190 to the computer 200. In an aspect, the controller 160 is configured to be gripped by the user 190. In another aspect, the controller 160 is configured to be wearable on the body of the user 190 or a portion of clothing. In another aspect, the controller 160 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the computer 200. In another aspect, the controller 160 receives an operation from the user 190 for controlling the position and movement of an object arranged in the virtual space.

  In an aspect, the motion sensor 130 is attached to the user's hand to detect movement of the user's hand. For example, the motion sensor 130 detects the rotation speed, rotation speed, etc. of the hand. The detected signal is sent to the computer 200. The motion sensor 130 is provided in the glove-type controller 160, for example. In some embodiments, for safety in real space, it is desirable that the controller 160 be worn on something that does not fly easily by being worn on the hand of the user 190, such as a glove shape. In another aspect, a sensor that is not worn by the user 190 may detect the movement of the user's 190 hand. Optical sensors that detect the position, orientation, direction of movement, distance of movement, etc. of various parts of the body of the user 190 may be used. For example, a signal from a camera that captures the user 190 may be input to the computer 200 as a signal representing the operation of the user 190. For example, the motion sensor 130 and the computer 200 are connected to each other wirelessly. In the case of wireless communication, the communication form is not particularly limited, and for example, Bluetooth (registered trademark) or other known communication methods are used.

  A computer 200 according to an embodiment of the present disclosure will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of a basic hardware configuration of the computer 200 according to an embodiment of the present disclosure. The computer 200 includes a processor 202, a memory 204, a storage 206, an input / output interface 208, and a communication interface 210 as main components. Each component is connected to the bus 212.

  The processor 202 executes a series of instructions included in a program stored in the memory 204 or the storage 206 based on a signal given to the computer 200 or when a predetermined condition is satisfied. In one aspect, the processor 202 is realized as a device such as a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or an FPGA (Field-Programmable Gate Array).

  The memory 204 temporarily stores programs and data. The program is loaded from the storage 206, for example. The data includes data input to the computer 200 and data generated by the processor 202. In one aspect, the memory 204 is implemented as a volatile memory such as a RAM (Random Access Memory).

  The storage 206 holds programs and data permanently. The storage 206 is realized, for example, as a nonvolatile storage device such as a ROM (Read-Only Memory), a hard disk device, or a flash memory. Programs stored in the storage 206 include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, a program for realizing communication with another computer 200, and the like. Data stored in the storage 206 includes data and objects for defining the virtual space.

  In another aspect, the storage 206 may be realized as a removable storage device such as a memory card. In still another aspect, a configuration using a program and data stored in an external storage device may be used instead of the storage 206 built in the computer 200. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used as in an amusement facility, it is possible to update programs and data collectively.

  In some embodiments, the input / output interface 208 communicates signals between the HMD 110, the tracking sensor 120, and the motion sensor 130. In one aspect, the input / output interface 208 is realized using a terminal such as USB (Universal Serial Bus, USB), DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), or the like. The input / output interface 208 is not limited to the above.

  In certain embodiments, the input / output interface 208 may further communicate with the controller 160. For example, the input / output interface 208 receives signals output from the controller 160 and the motion sensor 130. In another aspect, the input / output interface 208 sends instructions output from the processor 202 to the controller 160. The command instructs the controller 160 to vibrate, output sound, emit light, and the like. When the controller 160 receives the command, the controller 160 executes vibration, sound output, light emission, and the like according to the command.

  The communication interface 210 is connected to the network 192 and communicates with other computers (for example, the server 150) connected to the network 192. In one embodiment, the communication interface 210 is realized as a wired communication interface such as a LAN (Local Area Network) or a wireless communication interface such as WiFi (Wireless Fidelity), Bluetooth (registered trademark), or NFC (Near Field Communication). Is done. The communication interface 210 is not limited to the above.

  In an aspect, the processor 202 accesses the storage 206, loads one or more programs stored in the storage 206 into the memory 204, and executes a series of instructions included in the program. The one or more programs may include an operating system of the computer 200, an application program for providing a virtual space, game software that can be executed in the virtual space, and the like. The processor 202 sends a signal for providing a virtual space to the HMD 110 via the input / output interface 208. The HMD 110 displays an image on the display unit 112 based on the signal.

  In the example shown in FIG. 2, the computer 200 is provided outside the HMD 110. However, in another aspect, the computer 200 may be embedded in the HMD 110. As an example, a portable information communication terminal (for example, a smartphone) including the display unit 112 may function as the computer 200.

  Further, the computer 200 may be configured to be used in common for a plurality of HMDs 110. According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so that each user can enjoy the same application as other users in the same virtual space.

  In an embodiment, in the HMD system 100, a global coordinate system is preset. The global coordinate system has three reference directions (axes) parallel to the vertical direction in the real space, the horizontal direction orthogonal to the vertical direction, and the front-rear direction orthogonal to both the vertical direction and the horizontal direction. In the present embodiment, the global coordinate system is one of the viewpoint coordinate systems. Therefore, the horizontal direction, the vertical direction (vertical direction), and the front-rear direction in the global coordinate system are defined as an x axis, a y axis, and a z axis, respectively. More specifically, in the global coordinate system, the x axis is parallel to the horizontal direction of the real space. The y axis is parallel to the vertical direction of the real space. The z axis is parallel to the front-rear direction of the real space.

  In some embodiments, the tracking sensor 120 includes an infrared sensor. When the infrared sensor detects the infrared rays emitted from each light source of the HMD 110, the presence of the HMD 110 is detected. The tracking sensor 120 further detects the position and inclination of the HMD 110 in the real space according to the movement of the user 190 wearing the HMD 110 based on the value of each point (each coordinate value in the global coordinate system). More specifically, the tracking sensor 120 can detect temporal changes in the position and inclination of the HMD 110 using each value detected over time. As will be described later, the tracking sensor 120 may be configured to detect the position and inclination of the controller 160 in the real space by detecting infrared rays emitted from the controller 160.

  The global coordinate system is parallel to the real space coordinate system. Therefore, each inclination of the HMD 110 detected by the tracking sensor 120 corresponds to each inclination around the three axes of the HMD 110 in the global coordinate system. The tracking sensor 120 sets the uvw visual field coordinate system to the HMD 110 based on the inclination of the HMD 110 in the global coordinate system. The uvw visual field coordinate system set in the HMD 110 corresponds to a viewpoint coordinate system when the user 190 wearing the HMD 110 views an object in the virtual space.

  The uvw visual field coordinate system will be described with reference to FIG. FIG. 3 is a diagram conceptually showing the uvw visual field coordinate system set in the HMD 110 according to an embodiment. The tracking sensor 120 detects the position and inclination of the HMD 110 in the global coordinate system when the HMD 110 is activated. The processor 202 sets the uvw visual field coordinate system to the HMD 110 based on the detected value.

  As shown in FIG. 3, the HMD 110 sets a three-dimensional uvw visual field coordinate system with the head (origin) of the user wearing the HMD 110 as the center (origin). More specifically, the HMD 110 includes a horizontal direction, a vertical direction, and a front-rear direction (x-axis, y-axis, and z-axis) that define the global coordinate system by an inclination around each axis of the HMD 110 in the global coordinate system. Three directions newly obtained by tilting around the axis are set as the pitch direction (u-axis), yaw direction (v-axis), and roll direction (w-axis) of the uvw visual field coordinate system in the HMD 110.

  In an aspect, when the user 190 wearing the HMD 110 stands upright and is viewing the front, the processor 202 sets the uvw visual field coordinate system parallel to the global coordinate system to the HMD 110. In this case, the horizontal direction (x-axis), vertical direction (y-axis), and front-back direction (z-axis) in the global coordinate system are the pitch direction (u-axis) and yaw direction (v-axis) of the uvw visual field coordinate system in the HMD 110. And the roll direction (w-axis).

  After the uvw visual field coordinate system is set to the HMD 110, the tracking sensor 120 can detect the inclination (the amount of change in inclination) of the HMD 110 in the set uvw visual field coordinate system based on the movement of the HMD 110. In this case, the tracking sensor 120 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD 110 in the uvw visual field coordinate system as the inclination of the HMD 110. The pitch angle (θu) represents the inclination angle of the HMD 110 around the pitch direction in the uvw visual field coordinate system. The yaw angle (θv) represents the inclination angle of the HMD 110 around the yaw direction in the uvw visual field coordinate system. The roll angle (θw) represents the inclination angle of the HMD 110 around the roll direction in the uvw visual field coordinate system.

  Based on the detected tilt angle of the HMD 110, the tracking sensor 120 sets the uvw visual field coordinate system in the HMD 110 after the HMD 110 has moved to the HMD 110. The relationship between the HMD 110 and the uvw visual field coordinate system of the HMD 110 is always constant regardless of the position and inclination of the HMD 110. When the position and inclination of the HMD 110 change, the position and inclination of the uvw visual field coordinate system of the HMD 110 in the global coordinate system change in conjunction with the change of the position and inclination.

  In one aspect, the tracking sensor 120 is based on the intensity of infrared light acquired based on the output from the infrared sensor and the relative positional relationship between a plurality of points (for example, the distance between the points). The position in the real space may be specified as a relative position to the tracking sensor 120. Further, the processor 202 may determine the origin of the uvw visual field coordinate system of the HMD 110 in the real space (global coordinate system) based on the specified relative position.

  The virtual space will be further described with reference to FIG. FIG. 4 is a diagram conceptually illustrating one aspect of expressing a virtual space 400 according to an embodiment. The virtual space 400 has a spherical shape that covers the entire 360 ° direction of the center 406. In FIG. 4, the upper half of the celestial sphere in the virtual space 400 is illustrated in order not to complicate the description. In the virtual space 400, each mesh is defined. The position of each mesh is defined in advance as coordinate values in the XYZ coordinate system defined in the virtual space 400. The computer 200 associates each partial image constituting content (still image, moving image, etc.) that can be developed in the virtual space 400 with each corresponding mesh in the virtual space 400 so that a virtual space image that can be visually recognized by the user is obtained. A virtual space 400 to be developed is provided to the user.

  In one aspect, the virtual space 400 defines an xyz coordinate system with the center 406 as the origin. The xyz coordinate system is, for example, parallel to the global coordinate system. Since the xyz coordinate system is a kind of viewpoint coordinate system, the horizontal direction, vertical direction (vertical direction), and front-rear direction in the xyz coordinate system are defined as the x-axis, y-axis, and z-axis, respectively. Therefore, the x axis (horizontal direction) of the xyz coordinate system is parallel to the x axis of the global coordinate system, the y axis (vertical direction) of the xyz coordinate system is parallel to the y axis of the global coordinate system, and the xyz coordinate system The z-axis (front-rear direction) is parallel to the global coordinate system z-axis.

  When the HMD 110 is activated, that is, in the initial state of the HMD 110, the virtual camera 404 is disposed at the center 406 of the virtual space 400. In an aspect, the processor 202 displays an image captured by the virtual camera 404 on the display unit 112 of the HMD 110. The virtual camera 404 similarly moves in the virtual space 400 in conjunction with the movement of the HMD 110 in the real space. Thereby, changes in the position and orientation of the HMD 110 in the real space can be similarly reproduced in the virtual space 400.

  As in the case of the HMD 110, the virtual camera 404 has a uvw visual field coordinate system. The uvw visual field coordinate system of the virtual camera 404 in the virtual space 400 is defined so as to be linked to the uvw visual field coordinate system of the HMD 110 in the real space (global coordinate system). Therefore, when the inclination of the HMD 110 changes, the inclination of the virtual camera 404 changes accordingly. The virtual camera 404 can also move in the virtual space 400 in conjunction with the movement of the user wearing the HMD 110 in the real space.

  The processor 202 of the computer 200 defines the viewing area 410 in the virtual space 400 based on the arrangement position of the virtual camera 404 and the reference line of sight 408. The visual recognition area 410 corresponds to an area of the virtual space 400 that is visually recognized by the user wearing the HMD 110.

  The line of sight of the user 190 detected by the gaze sensor 140 is the direction in the viewpoint coordinate system when the user 190 visually recognizes the object. The uvw visual field coordinate system of the HMD 110 is equal to the viewpoint coordinate system when the user 190 visually recognizes the display unit 112. Further, the uvw visual field coordinate system of the virtual camera 404 is linked to the uvw visual field coordinate system of the HMD 110. Therefore, the HMD system 100 according to an aspect can regard the line of sight of the user 190 detected by the gaze sensor 140 as the line of sight of the user in the uvw visual field coordinate system of the virtual camera 404.

  The determination of the user's line of sight will be described with reference to FIG. FIG. 5 is a top view of the head of a user 190 wearing the HMD 110 according to an embodiment.

  In one aspect, the gaze sensor 140 detects each line of sight of the user's 190 right eye and left eye. In an aspect, when the user 190 is looking nearby, the gaze sensor 140 detects the lines of sight R1 and L1. In another aspect, when the user 190 is looking far away, the gaze sensor 140 detects the lines of sight R2 and L2. In this case, the angle formed by the lines of sight R2 and L2 with respect to the roll direction w is smaller than the angle formed by the lines of sight R1 and L1 with respect to the roll direction w. The gaze sensor 140 transmits the detection result to the computer 200.

  When the computer 200 receives the detection values of the lines of sight R1 and L1 from the gaze sensor 140 as the detection result of the line of sight, the computer 200 specifies the gazing point N1 that is the intersection of the lines of sight R1 and L1. On the other hand, when the detected values of the lines of sight R2 and L2 are received from the gaze sensor 140, the computer 200 specifies the intersection of the lines of sight R2 and L2 as the point of sight. The computer 200 specifies the line of sight N0 of the user 190 based on the specified position of the gazing point N1. For example, the computer 200 detects, as the line of sight N0, the extending direction of the straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user 190 and the gazing point N1. The line of sight N0 is a direction in which the user 190 is actually pointing the line of sight with both eyes. The line of sight N0 corresponds to a direction in which the user 190 actually directs the line of sight with respect to the visual recognition area 410.

  In another aspect, the HMD system 100 may include a microphone and a speaker in any part of the HMD system 100. The user can give a voice instruction to the virtual space 400 by speaking to the microphone.

  In another aspect, the HMD system 100 may include a television broadcast receiving tuner. According to such a configuration, the HMD system 100 can display a television program in the virtual space 400.

  In still another aspect, the HMD system 100 may include a communication circuit for connecting to the Internet or a call function for connecting to a telephone line.

  The visual recognition area 410 will be described with reference to FIGS. FIG. 6 is a diagram illustrating a yz section when the visual recognition area 410 is viewed from the x direction in the virtual space 400. FIG. 7 is a diagram illustrating an xz cross section when the visual recognition area 410 is viewed from the y direction in the virtual space 400.

  As shown in FIG. 6, the visual recognition area 410 in the yz section includes an area 602. The area 602 is defined by the arrangement position of the virtual camera 404, the reference line of sight 408, and the yz section of the virtual space 400. The processor 202 defines a range including the polar angle α around the reference line of sight 408 in the virtual space as a region 602.

  As shown in FIG. 7, the visible region 410 in the xz cross section includes a region 702. The area 702 is defined by the arrangement position of the virtual camera 404, the reference line of sight 408, and the xz cross section of the virtual space 400. The processor 202 defines a range including the azimuth angle β around the reference line of sight 408 in the virtual space 400 as a region 702. The polar angles α and β are determined according to the arrangement position of the virtual camera 404 and the orientation of the virtual camera 404.

  In one aspect, the HMD system 100 provides the user 190 with a field of view in the virtual space by causing the display unit 112 to display a field of view image based on a signal from the computer 200. The view field image corresponds to a portion of the virtual space image 402 that is superimposed on the viewing area 410. When the user 190 moves the HMD 110 worn on the head, the virtual camera 404 moves in conjunction with the movement. As a result, the position of the visual recognition area 410 in the virtual space 400 changes. Thereby, the visual field image displayed on the display unit 112 is updated to an image that is superimposed on the visual recognition area 410 in the virtual space 400 in the direction in which the user faces in the virtual space image 402. The user can visually recognize a desired direction in the virtual space 400.

  Thus, the orientation (tilt) of the virtual camera 404 corresponds to the user's line of sight (reference line of sight 408) in the virtual space 400, and the position where the virtual camera 404 is arranged corresponds to the user's viewpoint in the virtual space 400. Therefore, by moving the virtual camera 404 (including the operation of changing the arrangement position and the operation of changing the orientation), the image displayed on the display unit 112 is updated, and the field of view (including the viewpoint and the line of sight) of the user 190 is moved. Is done.

  While wearing the HMD 110, the user 190 can visually recognize only the virtual space image 402 developed in the virtual space 400 without visually recognizing the real world. Therefore, the HMD system 100 can give the user a high sense of immersion in the virtual space 400.

  In an aspect, the processor 202 may move the virtual camera 404 in the virtual space 400 in conjunction with movement of the user 190 wearing the HMD 110 in real space. In this case, the processor 202 specifies an image region (that is, the visual recognition region 410 in the virtual space 400) projected on the display unit 112 of the HMD 110 based on the position and orientation of the virtual camera 404 in the virtual space 400.

  According to an embodiment, the virtual camera 404 may include two virtual cameras: a virtual camera for providing an image for the right eye and a virtual camera for providing an image for the left eye. Also, appropriate parallax may be set in the two virtual cameras so that the user 190 can recognize the three-dimensional virtual space 400.

  An example of the controller 160 will be described with reference to FIG. FIG. 8 is a diagram illustrating a schematic configuration of the controller 160 according to an embodiment.

  In certain aspects, the controller 160 may include a right controller and a left controller. For simplicity of explanation, only the right controller 800 is shown in FIG. The right controller 800 is operated with the right hand of the user 190. The left controller is operated with the left hand of the user 190. In an aspect, the right controller 800 and the left controller are configured symmetrically as separate devices. Therefore, the user 190 can freely move the right hand holding the right controller 800 and the left hand holding the left controller, respectively. In another aspect, the controller 160 may be an integrated controller that accepts operations of both hands. Hereinafter, the right controller 800 will be described.

  The right controller 800 includes a grip 802, a frame 804, and a top surface 806. The grip 802 is configured to be gripped by the right hand of the user 190. For example, the grip 802 can be held by the palm of the right hand of the user 190 and three fingers (middle finger, ring finger, little finger).

  The grip 802 includes buttons 808 and 810 and a motion sensor 130. The button 808 is disposed on the side surface of the grip 802 and receives an operation with the middle finger of the right hand. The button 810 is disposed on the front surface of the grip 802 and receives an operation with the index finger of the right hand. In some embodiments, the buttons 808, 810 are configured as trigger buttons. The motion sensor 130 is built in the housing of the grip 802. Note that the grip 802 may not include the motion sensor 130 when the operation of the user 190 can be detected from around the user 190 by a camera or other device.

  The frame 804 includes a plurality of infrared LEDs 812 arranged along the circumferential direction thereof. The infrared LED 812 emits infrared light in accordance with the progress of the program while the program using the controller 160 is being executed. Infrared rays emitted from the infrared LED 812 can be used by the tracking sensor 120 to detect the positions and postures (tilt and orientation) of the right controller 800 and the left controller (not shown). In the example shown in FIG. 8, infrared LEDs 812 arranged in two rows are shown, but the number of arrays is not limited to that shown in FIG. 8. An array of one column or three or more columns may be used.

  The top surface 806 includes buttons 814 and 816 and an analog stick 818. Buttons 814 and 816 are configured as push buttons. Buttons 814 and 816 receive an operation with the thumb of the right hand of user 190. In a certain aspect, the analog stick 818 accepts an operation in an arbitrary direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object arranged in the virtual space 400.

  In an aspect, the right controller 800 and the left controller include a battery for driving a member such as the infrared LED 812. The battery may be either a primary battery or a secondary battery, and the shape thereof may be arbitrary, such as a button type or a dry battery type. In another aspect, the right controller 800 and the left controller may be connected to a USB interface of the computer 200, for example. In this case, the right controller 800 and the left controller may be supplied with power via the USB interface.

  FIG. 9 is a block diagram illustrating functions of the computer 200 for realizing display processing or the like of the virtual space 400 in the HMD system 100 according to an embodiment of the present disclosure. The computer 200 controls image output to the display unit 112 mainly based on inputs from the tracking sensor 120, the motion sensor 130, the gaze sensor 140, and the controller 160.

  The computer 200 includes a processor 202, a memory 204, and a communication control unit 205. The processor 202 includes a virtual space specifying unit 902, an HMD motion detection unit 903, a visual line detection unit 904, a reference visual line determination unit 906, a visual field region determination unit 908, a visual field image generation unit 912, and a visual field image output unit 926. Can be included. Memory 204 may be configured to store various information. In one example, the memory 204 may include virtual space data 928, object data 930, application data 932, and other data 934. The memory 204 also includes various data necessary for calculation to provide output information corresponding to inputs from the tracking sensor 120, the motion sensor 130, the gaze sensor 140, the controller 160, etc., to the display unit 112 associated with the HMD 110. But you can. The object data 930 may include data related to operation objects, virtual objects, and the like arranged in the virtual space. The display unit 112 may be built in the HMD 110 or may be a display of another device (for example, a smartphone) that can be attached to the HMD 110.

  In FIG. 9, the components included in the processor 202 are just one example of expressing the function executed by the processor 202 as a specific module. The functions of multiple components may be realized by a single component. The processor 202 may be configured to perform the functions of all components.

  FIG. 10 is a flowchart of a general process for displaying an image of the virtual space in which the user is immersed on the display unit 112.

  A general process of the HMD system 100 for providing a virtual space image will be described with reference to FIGS. 9 and 10. The virtual space 400 may be provided by the interaction of the tracking sensor 120, the gaze sensor 140, the computer 200, and the like.

  Processing begins at step 1002. As an example, a game application included in the application data may be executed by the computer 200. In step 1004, the processor 202 (virtual space specifying unit 902) refers to the virtual space data 928 and generates a celestial spherical virtual space image 402 constituting the virtual space 400 in which the user is immersed. The tracking sensor 120 detects the position and inclination of the HMD 110. Information detected by the tracking sensor 120 is transmitted to the computer 200. In step 1006, the HMD motion detection unit 903 acquires position information and tilt information of the HMD 110. In step 1008, the viewing direction is determined based on the acquired position information and tilt information.

  When the gaze sensor 140 detects the movement of the left and right eyeballs of the user, the information is transmitted to the computer 200. In step 1010, the line-of-sight detection unit 904 identifies the direction in which the right eye and the left eye are directed, and determines the line-of-sight direction N0. In step 1012, the reference visual line determination unit 906 determines the visual field direction determined by the inclination of the HMD 110 or the user's visual line direction N0 as the reference visual line 408. The reference line of sight 408 may also be determined based on the position and tilt of the virtual camera 404 that follows the position and tilt of the HMD 110.

  In step 1014, the view area determination unit 908 determines the view area 410 of the virtual camera 404 in the virtual space 400. As shown in FIG. 4, the visual field area 410 is a part of the virtual space image 402 that constitutes the visual field of the user. The field of view area 410 is determined based on the reference line of sight 408. A yz sectional view of the visual field region 410 viewed from the x direction and an xz sectional view of the visual field region 410 viewed from the y direction are shown in FIGS. 6 and 7 described above, respectively.

  In step 1016, the visual field image generation unit 912 generates a visual field image based on the visual field region 410. The field-of-view image includes two two-dimensional images for the right eye and the left eye. These two-dimensional images are superimposed on the display unit 112 (more specifically, the right-eye image is output to the right-eye display unit, and the left-eye image is output to the left-eye display unit). A virtual space 400 as an image is provided to the user. In step 1018, the view image output unit 926 outputs information related to the view image to the display unit 112. The display unit 112 displays the view image based on the received view image information. The process ends at step 1020.

  Hereinafter, embodiments of the present disclosure will be specifically described. Here, as a specific example to which the embodiment of the present disclosure can be applied, an arcade game provided in an amusement facility or the like, where a player (user) becomes an avatar and immerses in the virtual space, Assume a game of maneuvering a vehicle. However, embodiments of the present disclosure are not necessarily limited to such an aspect. It will be apparent to those skilled in the art that the embodiments of the present disclosure can take various forms that fall within the scope of the claims.

  FIG. 11 is a diagram schematically illustrating a configuration of a game system 1100 according to an embodiment of the present disclosure. The game system 1100 includes two player devices 1120A and 1120B (hereinafter collectively referred to as “player device 1120”), an attendant device 1140, and a switching device 1146. The player device 1120, the attendant device 1140, and the switching device 1146 are installed in the same facility and are connected to each other wirelessly or by wire. Alternatively, the player device 1120, the attendant device 1140, and the switching device 1146 are installed at remote locations, and are mutually connected via a network (not shown) such as a LAN (local area network), a WAN (wide area network), or the Internet. May be communicably connected. The configuration in FIG. 11 is an exemplification, and the configuration is not limited thereto. For example, the number of player devices 1120 may be one or three or more. One player device 1120 may be configured to be operable by a plurality of players 1132. Note that since the configuration of the player device 1120B is the same as the configuration of the player device 1120A, details of the configuration of the player device 1120B are not shown in FIG.

  The player device 1120 is a device operated by the player 1132. The player 1132 can enjoy an arcade game by operating the player device 1120. As shown in FIG. 11, the player device 1120A includes an HMD 110, an HMD sensor (tracking sensor) 120, a computer 200, a seating portion 1122, and controllers 1124A and 1124B (hereinafter collectively referred to as “controller 1124”). And an audio output unit 1126 and a movable doll 1128. The HMD 110, the HMD sensor 120, and the computer 200 have the same configuration as that already described with reference to FIG. The HMD 110, the HMD sensor 120, the controller 1124, the audio output unit 1126, and the movable doll 1128 are connected to the computer 200 (input / output interface 208 shown in FIG. 2). Note that the configuration in FIG. 11 is an example, and the present invention is not limited to this. For example, one computer 200 may be connected to a plurality of player devices 1120.

  The player 1132 sits on the seating section 1122 with the HMD 110 mounted, and operates the controller 1124 while viewing the game image representing the field of view in the virtual space displayed on the display section 112 of the HMD 110. Play games. During game play, the player 1132 can control the movement of a vehicle (or other object) that is an object in the virtual space by operating the controller 1124 in various ways.

  The controller 1124 includes a handle 1124A that the player 1132 holds and operates with a hand, and a pedal 1124B that the player 1132 operates with a foot. The pedal 1124B may be configured to rotate by a rowing operation like a pedal of a bicycle, or may be configured to be operated by a depressing operation like an accelerator pedal of an automobile. The controller 1124 receives an operation of the player 1132 (for example, an operation in which the player 1132 moves the handle 1124A or the pedal 1124B), generates a signal corresponding to the operation, and outputs the signal to the computer 200. This signal represents a game input from the player 1132 to the game system 1100. Note that the handle 1124 </ b> A and the pedal 1124 </ b> B are merely examples of the controller 1124. The controller 1124 may have the configuration described above with respect to FIG. Various modes of controllers that can be worn on other parts of the user's body or operable on other parts of the user's body may alternatively be applied. The seating section 1122 may be included in the controller 1124. For example, the seating section 1122 is configured to be tilted by the weight shift of the player 1132, and a signal corresponding to the tilt of the seating section 1122 is generated and output to the computer 200 to control the movement of the vehicle in the virtual space. You may make it do. In this case, the seating section 1122 may be anything that the player 1132 can ride (boarding section) such as a board, and does not have to be seated. When the seating portion 1122 is included in the controller 1124, the handle 1124 </ b> A and the pedal 1124 </ b> B may not be included in the controller 1124.

  The audio output unit 1126 is disposed in the vicinity of the seating unit 1122, for example, at a position close to the head of the player 1132 when the player 1132 sits on the seating unit 1122. The audio output unit 1126 can output the audio information from the attendant 1134 and the audio information generated or reproduced by the computer 200. The audio output unit 1126 may be configured integrally with the HMD 110 without being provided in the vicinity of the seating unit 1122, or may be configured as a headphone worn together with the HMD 110.

  The movable doll 1128 is arranged in the vicinity of the seating section 1122, for example, at a position behind the player 1132 when the player 1132 sits on the seating section 1122. The movable doll 1128 is configured to be able to physically contact the body of the player 1132 seated on the seating portion 1122. For example, the movable doll 1128 includes two movable arms that can physically contact the body of the player 1132 seated on the seating section 1122. The arm of the movable doll 1128 is driven so as to make physical contact with the body of the player 1132 when audio information is received from the attendant 1134 or the computer 200.

  The attendant device 1140 is a device operated by an attendant 1134 which is an assistant who assists the player 1132 during the game play. As shown in FIG. 11, the attendant device 1140 includes a display unit 1142 and a voice input unit 1144. The display unit 1142 is configured to display the same game image that is displayed on the HMD 110 of the player device 1120. As the display unit 1142, for example, a liquid crystal display, an organic EL display, a plasma display, or a projector is applicable. The voice input unit 1144 is a microphone for converting the voice of the attendant 1134 into an electrical signal.

  The attendant 1134 inputs voice guidance or the like for the player 1132 using the voice input unit 1144 while viewing the game image of the player 1132 displayed on the display unit 1142. Audio information from the attendant 1134 is transmitted to the player device 1120 via the switching device 1146 and output from the audio output unit 1126 of the player device 1120. Thereby, the attendant 1134 can support the game operation of the player 1132.

  FIG. 12 is a flowchart of a method 1200 according to the first embodiment. In the first embodiment, when the attendant 1134 emits sound during the game play of the player 1132, the sound of the attendant 1134 is output from the sound output unit 1126 of the player device 1120, and the movable doll 1128 becomes the sound of the attendant 1134. In response, the player 1132 is driven to come into contact with the body. In an embodiment of the present disclosure, a computer program may cause the processor 202 (or the computer 200) to execute the steps illustrated in FIG. Also, another embodiment of the present disclosure may be implemented as the processor 202 (or computer 200) that performs the method 1200.

  Processing begins at step 1202. The processor 202 reads and executes the game program included in the application data 932 stored in the memory 204.

  The process proceeds to step 1204, and the controller 1124 generates a game input corresponding to the body movement of the player 1132 and outputs it to the computer 200. For example, when the player 1132 turns or tilts the handle 1124A by hand, the rotation amount or the tilt amount of the handle 1124A is continuously detected at a predetermined time interval, and a signal indicating the detection result indicates the vehicle in the virtual space. It is output from the handle 1124A to the computer 200 as a game input for maneuvering. Further, for example, when the player 1132 steps on or turns (rotates) the pedal 1124B with his / her leg, the depression amount or the rotation amount of the pedal 1124B is continuously detected at predetermined time intervals, and a signal indicating the detection result is obtained. The game 1 is output from the pedal 1124B to the computer 200 as a game input for manipulating the vehicle in the virtual space.

  The process proceeds to step 1206, and the processor 202 determines the position and inclination of the vehicle (eg, bicycle, car, motorcycle, etc.) on which the avatar of the player 1132 is riding in the virtual space based on the game input output from the controller 1124. To control. For example, when the player 1132 turns the handle 1124A to the right or left, the vehicle in the virtual space changes the traveling direction or the inclination of the aircraft to the right or left according to the rotation angle of the handle 1124A. Also, for example, when the player 1132 pedals the pedal 1124B, the vehicle in the virtual space moves up or down in the virtual space or changes its moving speed according to the number of rotations of the pedal 1124B. As described above, the player 1132 can operate the controller 1124 to control the movement of the vehicle in the virtual space.

  The process proceeds to step 1208, where the HMD sensor 120 continuously detects the position and inclination of the HMD 110 at predetermined time intervals and outputs the detection result to the computer 200.

  The process proceeds to step 1210, and the processor 202 determines the position and inclination of the vehicle in the virtual space reflecting the control in step 1206, and the position and inclination of the HMD 110 detected in step 1208 (that is, the position and inclination of the head of the player 1132). And a game image in which the virtual space is viewed from the position of the avatar of the player 1132 riding on the vehicle in the virtual space. Specifically, the processor 202 determines the relative position in the virtual space corresponding to the detection position of the HMD 110 by the HMD sensor 120 based on the position and inclination of the vehicle in the virtual space controlled according to the processing in step 1206. The position of the avatar's head, that is, the virtual viewpoint is determined, and a virtual camera is arranged at the virtual viewpoint. The processor 202 determines a direction in the virtual space corresponding to the inclination of the HMD 110 detected by the HMD sensor 120 as a reference line of sight (for example, the reference line of sight 408 shown in FIG. 4), and orients the virtual camera in the direction of the reference line of sight. Let As described above with reference to FIG. 4, a field of view (for example, a viewing area 410 as shown in FIG. 4) that the player 1132 can visually recognize in the virtual space based on the position of the virtual camera and the reference line of sight (for example, the reference line of sight 408). Is determined. The image within the field of view is the game image generated in step 1210. The processor 202 outputs the game image generated in this way to the HMD 110 and the attendant device 1140 for display. By viewing the game image displayed on the HMD 110, the player 1132 can visually recognize the virtual space from the viewpoint of the avatar riding on the vehicle in the virtual space.

  The process proceeds to step 1212, and the attendant device 1140 acquires the game image output from the player device 1120 and displays it on the display unit 1142. Thereby, the attendant 1134 can see the same game image as the game image displayed on the HMD 110 of the player 1132. The display unit 1142 may display a plurality of different game images acquired from the plurality of player devices 1120 at the same time by dividing the screen, or sequentially display each game image by switching them over time. Also good. The attendant 1134 can grasp the play situation of each player 1132 by looking at the game image displayed on the display unit 1142, and based on that, the audio input unit 1144 is sent to each player 1132 or one specific player. Can be used to give instructions for playing the game properly, and to convey comments (impressions, etc.) regarding the play situation.

  The process proceeds to step 1214, and the voice input unit 1144 of the attendant apparatus 1140 acquires voice information from the attendant 1134. This audio information includes an audio instruction issued by the attendant 1134 to guide the game operation to the player 1132 and an audio comment on the game play performed by the player 1132. For example, if the attendant 1134 determines that the operation of the player 1132 is not appropriate, a voice instruction for guiding an appropriate game operation (for example, “turn right” or “swivel the pedal fast”) Utterance). Alternatively, the attendant 1134 can utter a voice comment (for example, an utterance such as “Do your best” or “Wow”) that supports or compliments the play of the player 1132. In addition to producing a sound, the attendant 1134 can operate the switching device 1146 to select at least one player device 1120 among the plurality of player devices 1120 as a destination of the sound information. The attendant device 1140 transmits audio information to the destination player device 1120 via the switching device 1146. Instead of the attendant 1134 manually operating the switching device 1146, the switching device 1146 recognizes the voice information transmitted from the attendant device 1140 (for example, from the attendant 1134 included in the voice information to a specific player 1132). The destination player device 1120 may be determined according to the recognition result.

  The process proceeds to step 1216, and the player device 1120 selected by the attendant 1134 acquires the audio information transmitted from the attendant device 1140, and outputs the audio information from the audio output unit 1126. Since the sound is emitted from the voice output unit 1126 disposed near the player 1132, the player 1132 can surely recognize that the voice instruction or voice comment is addressed to him. In this way, the attendant 1134 selects the operation support target player device 1120 as the destination of the audio information, and outputs this audio information from the audio output unit 1126 of the operation support target player device 1120. For this reason, a player who plays on the operation support target player device 1120 can surely recognize that it is a voice instruction or a voice comment addressed to himself, not to other players, Therefore, it is possible to accurately perform operation support for a player who has not performed the above operation.

  The process proceeds to step 1218, and the processor 202 of the player device 1120 that has acquired the voice information from the attendant device 1140 recognizes the acquired voice information, and the predetermined part of the movable doll 1128 is the body of the player 1132 according to the recognition result. The movable doll 1128 is driven so as to make physical contact with a part of the head (eg, head, shoulder, arm, hand, waist, leg, etc.). The correspondence relationship between the recognition result of the voice information and the operation content of the movable doll 1128 corresponding to the recognition result (for example, the part that moves the movable doll 1128, the amount that the part is moved, the speed, etc.) It is stored in the memory 204 of the computer 200. For example, when the voice information is a voice instruction “turn left”, the processor 202 recognizes the voice information so that the left arm of the movable doll 1128 contacts the left arm or the left hand of the player 1132. The movable doll 1128 is driven. Alternatively, if the voice information is a voice instruction “please rise”, the processor 202 recognizes the voice information and recognizes the voice information so that both arms of the movable doll 1128 are in contact with both legs of the player 1132. The doll 1128 is driven. As described above, the processor 202 changes the position where the movable doll 1128 contacts the body of the player 1132 in accordance with the contents of the audio information.

  Since the player 1132 wears the non-transparent HMD 110 and is immersed in the virtual space, whether the physical contact with the player 1132 during the game play is due to the movable doll 1128 or the attendant 1134. Cannot be determined. On the other hand, the sound produced by the attendant 1134 is heard from the sound output unit 1126 at the ear of the player 1132. Therefore, when the player 1132 touches the body with the movable doll 1128 while immersing in the virtual space and playing the game, the attendant 1134, who is a living person, is immediately next to him, You can feel as if you are guiding or cheering on the operation.

  Note that the movable doll 1128 may be configured to move according to a manual operation by the attendant 1134 instead of voice recognition by the processor 202. For example, the attendant 1134 looks at the game image displayed on the display unit 1142 to grasp the play status of the player 1132, issues voice guidance or voice comments to the voice input unit 1144, and operates a remote controller (not shown). Then, the movable doll 1128 may be driven manually.

  The process proceeds to step 1220, and the processor 202 determines whether or not the game is over. This determination can be made based on, for example, whether the vehicle in the virtual space that the player 1132 controls has reached the goal, whether the game end command is input from the player 1132, or the like. If it is determined in step 1220 that the game has not yet ended, the process returns to step 1206. On the other hand, if it is determined in step 1220 that the game has ended, the process ends in step 1222.

  In the method 1200 described above, the audio information from the attendant 1134 is output from the audio output unit 1126 in step 1216. However, as another aspect, the audio information automatically generated by the computer 200 may be output from the audio output unit 1126 together with the audio information from the attendant 1134 or instead of the audio information from the attendant 1134. Good. For example, the attendant 1134 does not utter the voice instruction for guiding the game operation to the player 1132, but the computer 200 automatically generates the voice instruction for cheering the player 1132 or the like. May be issued.

  Specifically, the processor 202 determines whether or not the movement of the vehicle in the virtual space controlled by the processing of step 1206 (ie, by the operation of the player 1132) is appropriate (for example, the vehicle passes). Judgment based on the comparison with the recommended course). If it is determined that the movement of the vehicle is not appropriate, the processor 202 guides the player 1132 of an operation necessary to cause the vehicle in the virtual space to take an appropriate movement (for example, to bring the position of the vehicle closer to the recommended course). The voice instruction to be generated is automatically generated and output from the voice output unit 1126. The voice output unit 1126 also outputs a voice comment issued by the attendant 1134. The processor 202 may drive the movable doll 1128 in accordance with the automatically generated voice instruction, or may drive the movable doll 1128 in accordance with a voice comment such as cheering from the attendant 1134. According to this aspect, since the attendant 1134 does not give a voice instruction and only issues a voice comment, the work load on the attendant 1134 can be reduced. In addition, when the movable doll 1128 is driven in accordance with the voice comment from the attendant 1134 and comes into contact with the body of the player 1132, the player 1132 will cheer during the game as if the attendant 1134 was right next to him. You can feel as if you are.

  Further, in the method 1200, in step 1216, audio information emitted by one attendant 1134 is output from the audio output unit 1126. However, as another aspect, audio information emitted by a plurality of attendants 1134 may be output from the audio output unit 1126. The plurality of attendants 1134 include a main attendant (not shown) in which the player 1132 is interested, and a secondary attendant (not shown) in which the player 1132 is not as interested as the main attendant. As an example, when the player 1132 is a male, the main attendant may be a female and the secondary attendant may be a male. For example, the main attendant issues a voice comment on the game play performed by the player 1132, and the sub attendant issues a voice instruction for guiding the game operation to the player 1132. The movable doll 1128 may be configured to be driven according to the content of the audio information from the main attendant and not to be driven according to the content of the audio information from the sub attendant.

  Specifically, the computer 200 outputs a voice comment from the main attendant from the voice output unit 1126, recognizes the voice comment, and drives the movable doll 1128 according to the recognition result. On the other hand, the computer 200 simply outputs the voice instruction from the voice output unit 1126 and does not drive the movable doll 1128 according to the voice instruction. Since the main attendant only needs to give a voice comment without giving a voice instruction, the workload of the main attendant can be reduced. As another example, the main attendant may issue a voice instruction, the secondary attendant may give a voice comment, and the movable doll 1128 may be configured to be driven according to the voice instruction from the main attendant. As described above, in this aspect, the voice of the main attendant and the secondary attendant is output from the voice output unit 1126, and the movable doll 1128 is driven according to the voice of the main attendant (voice instruction or voice comment). Therefore, the player 1132 can obtain a feeling as if the main attendant is right next to him and guides or supports the operation of the game.

  Further, in the method 1200, only the specific player device 1120 selected by the attendant 1134 among the plurality of player devices 1120 outputs the sound information from the sound output unit 1126, and drives the movable doll 1128. However, as another aspect, audio information may be output from the audio output unit 1126 of all player apparatuses 1120, and the movable doll 1128 may be driven only by the selected specific player apparatus 1120. Good. In this case, all the players 1132 listen to the audio information, but only the player 1132 touched by the movable doll 1128 can grasp that the audio information is addressed to himself / herself. Even if the player 1132 not touched by the movable doll 1128 listens to the voice information, the player 1132 can determine that the voice information is not addressed to himself / herself.

  FIG. 13 is a flowchart of a method 1300 according to the second embodiment. In the second embodiment, among a plurality of players who are playing the same game in the virtual space, a player who has finished the game play first (for example, the player 1132 of the player device 1120A, hereinafter referred to as the first player 1132A). It is possible to send a voice instruction or a voice comment to another player who continues the game (for example, the player 1132 of the player device 1120B, hereinafter referred to as the second player 1132B). The first player 1132A and the second player 1132B may each be a plurality of people. The player devices 1120A and 1120B perform processing according to the method 1200 according to the first embodiment described above until the first player 1132A ends the game play in the player device 1120A. The flowchart of the method 1300 of FIG. 13 shows the process performed in each device after the first player 1132A ends the game play.

  The process of the method 1300 starts in step 1302 when the game play ends in the player device 1120A of the first player 1132A (step 1222 in the method 1200).

  Processing proceeds sequentially to step 1304, step 1306, and step 1308. The processes in these steps are the same as the processes in steps 1204, 1206, and 1208, respectively. Therefore, even after the first player 1132A finishes the game play, the player device 1120B of the second player 1132B continues to process the game, and the second player 1132B continues to control the vehicle in the virtual space.

  Processing proceeds to step 1310 and processor 202 of player device 1120B generates a game image in the same manner as step 1210 of method 1200. Further, the processor 202 of the player device 1120B outputs the generated game image to the HMD 110 and the attendant device 1140 of the player device 1120B as in Step 1210 of the method 1200, and the game image is output to the player device 1120A of the first player 1132A. Also output. The second player 1132B can continue his / her game play after watching the game image displayed on the HMD 110 even after the game play of the first player 1132A is completed.

  After step 1310, steps 1312 and 1314 are performed in the attendant device 1140. Each processing of step 1312 and step 1314 is the same as the processing of step 1212 and step 1214 described above, respectively. The attendant 1134 can see the game image of the second player 1132B displayed on the display unit 1142 as before the first player 1132A finishes the game play (first embodiment), and the voice input unit 1144. It is also possible to input a voice instruction or a voice comment to the second player 1132B.

  After step 1310, the process proceeds to step 1316 in the player device 1120A of the first player 1132A, and the player device 1120A acquires the game image output from the player device 1120B of the second player 1132B, and the HMD 110 of the player device 1120A. To display. Thereby, the first player 1132A can view the game image of the second player 1132B after the game play of the first player 1132A is completed.

  Processing continues at step 1318. The first player 1132A can issue a voice instruction or a voice comment to the second player 1132B by looking at the game image of the second player 1132B displayed on the HMD 110. As shown in FIG. 11, the player device 1120A of the first player 1132A includes a sound input unit 1138 for acquiring sound uttered by the first player 1132A. In step 1318, the voice input unit 1138 acquires a voice instruction or voice comment from the first player 1132A, and transmits the acquired voice instruction or voice comment to the player device 1120B of the second player 1132B. As described above, the first player 1132A can act as an attendant who supports the game play of the second player 1132B after finishing his game play.

  The process proceeds to step 1320, where the player device 1120B of the second player 1132B acquires the audio information transmitted from the attendant device 1140 and the player device 1120A of the first player 1132A, and outputs the audio information from the audio output unit 1126. . In addition to the voice instruction or voice comment from the attendant 1134, the second player 1132B can also listen to the voice instruction or voice comment from the first player 1132A that finished the game play first.

  The process proceeds to step 1322, and the player device 1120B of the second player 1132B performs voice recognition of the audio information acquired from the attendant device 1140 or the player device 1120A, or the attendant 1134 or the second device in the same manner as the step 1210 of the method 1200. The movable doll 1128 is driven in accordance with a manual operation by one player 1132A. When the movable doll 1128 is driven in accordance with a voice instruction or voice comment from the first player 1132A, the second player 1132B is as if the first player 1132A is right next to him and provides guidance and support for game operations. You can feel as if you are doing it.

  The process proceeds to step 1324, and it is determined whether or not the game play of the second player 1132B is completed in the same manner as in step 1210 of the method 1200. When the game play of the second player 1132B has not ended, the process returns to step 1304, and when the game play of the second player 1132B ends, the process ends at step 1326.

  According to the present embodiment, the first player 1132A who has finished the game play first can transmit a voice instruction or a voice comment to the second player 1132B who has not finished the game play. Even after the play is over, the game can be enjoyed together with the second player 1132. In particular, when the first player 1132A and the second player 1132B are friends, the second player 1132B is connected to the friend by the sound from the sound output unit 1126 and the physical contact by the movable doll 1128 according to the sound. Since it is possible to obtain a sense that the game player is receiving support from the first player 1132A, a game with higher palatability can be realized.

  Embodiments of the present disclosure have been described primarily as being implemented as the processor 202 (or computer 200) or method 1200 or 1300. However, it will be apparent to those skilled in the art that embodiments of the present disclosure may be implemented as a computer program that causes processor 202 to perform method 1200 or 1300.

  While embodiments of the present disclosure have been described, it should be understood that these are only examples and are not intended to limit the scope of the present disclosure. It should be understood that changes, additions, improvements, and the like of the embodiments can be made as appropriate without departing from the spirit and scope of the present disclosure. The scope of the present disclosure should not be limited by any of the above-described embodiments, but should be defined only by the claims and their equivalents.

  In the above-described various embodiments, an example in which a virtual space in which a user is immersed is provided by a non-transparent HMD device has been described. However, a transmissive HMD device may be employed as the HMD device. In such an embodiment, an augmented reality (AR) space or a mixed reality (MR) space is displayed by superimposing and displaying an image including a virtual object in a real space visually recognized by a user via a transmissive HMD device. : User experience in a mixed reality) space may be provided.

DESCRIPTION OF SYMBOLS 100 ... HMD system, 110 ... HMD, 112 ... Display part, 114 ... Sensor, 120 ... Tracking sensor, 130 ... Motion sensor, 140 ... Gaze sensor, 150 ... Server, 160 ... Controller, 190 ... User, 192 ... Network, 200 ... Computer, 202 ... Processor, 204 ... Memory, 205 ... Communication controller, 206 ... Storage, 208 ... I / O interface, 210 ... Communication interface, 212 ... Bus, 400 ... Virtual space, 402 ... Virtual space image, 404 ... Virtual Camera, 406 ... Center, 408 ... Reference line of sight, 410 ... Viewing area, 602, 702 ... Area, 800 ... Controller, 802 ... Grip, 804 ... Frame, 806 ... Top, 808, 810, 814, 816 ... Button, 818 ... Analog stick 902 ... Virtual space specifying unit, 903 ... HMD motion detection unit, 904 ... Gaze detection unit, 906 ... Reference gaze determination unit, 908 ... Visual field region determination unit, 912 ... Visual field image generation unit, 926 ... Visual field image output unit, 928 ... Virtual space data, 930... Object data, 932... Application data, 934 .. other data, 1100. DESCRIPTION OF SYMBOLS 1132 ... Player, 1134 ... Attendant, 1138 ... Voice input part, 1140 ... Attendant device, 1142 ... Display part, 1144 ... Voice input part, 1146 ... Switching apparatus

Claims (10)

A seating section for the player to sit down ;
A head mounted device that can be worn on the player's head;
A controller for generating a game input in accordance with the movement of the body excluding the player's head;
A processor that causes the head mounted device to display a field-of-view image corresponding to the posture of the head mounted device and a game input from the controller;
An audio output unit for outputting audio information when audio information for attending the player is received;
A movable doll arranged near the seating portion and configured to physically contact the player's body when the audio information is received;
A game device comprising: The game apparatus according to claim 1 , wherein the movable doll is configured to change a position in contact with the player in accordance with the content of the audio information. The game device according to claim 1 , wherein the voice information is a voice instruction issued by an attendant to guide the player to perform a game operation. The audio information includes first audio information and second audio information,
The first audio information is an audio instruction that is generated or reproduced by a computer according to the game play of the player and that guides the player to operate the game.
The second audio information is an audio comment issued by an attendant to a game play performed by the player.
The game device according to claim 1 or 2 . The audio information includes first audio information and second audio information,
The first voice information is a voice instruction issued by a secondary attendant to guide the player to operate the game,
The second audio information is an audio comment issued by an attendant to a game play performed by the player.
The game device according to claim 1 or 2 . The movable doll physically contacts the player's body when the second voice information addressed to the player is received, and when the first voice information addressed to the player is received, The game device according to claim 4 , wherein the game device is configured not to physically contact the player's body . Emitted by A attendant, and voice recognition unit that recognizes a voice comment for the voice instruction or game play the player performs guides the game operation with respect to the player,
A control unit for controlling the movement of the movable doll based on a result of voice recognition by the voice recognition unit;
The game apparatus according to claim 1 , further comprising: The game apparatus according to claim 3 , wherein the movable doll is configured to move according to a manual operation by the attendant. A plurality of game devices according to any one of claims 3 to 8 ;
A display device that is communicably connected to each of the plurality of game devices and presents, to the attendant, a field-of-view image for each player generated according to the game play of the player in each game device;
A voice input device for inputting voice instructions or voice comments from the attendant;
A switching device capable of selecting at least one of the plurality of game devices as an output destination of the voice instruction or voice comment by the operation of the attendant;
A game system comprising: The head mounted device in each of the plurality of game devices is configured to display a view field image in another game device after the game play in the game device is completed,
Each of the plurality of game device acquires a voice instruction or voice annotation from a player of the game device after the game play has ended for the player of the other game devices in the game apparatus, the voice instruction or voice annotation obtained The game system of Claim 9 provided with the audio | voice input part transmitted to said other game device .

Images