JP2022169526A - Application control program, application control method and application control system - Google Patents

Abstract

The present invention provides an application that allows a user wearing a wearable display and a user not wearing a wearable display to play at the same time. A first program that causes a control unit 16 to output an image of a virtual space to a wearable display worn by a first user and a display used by a second user. and a second display function that outputs an image of the virtual space to the display used by the second user based on the second user's operation, and the first display function is configured such that a character corresponding to the second user is arranged. outputs an image of the virtual space to the display worn by the second user, displays the display range displayed on the display used by the second user in an identifiable manner, and the second display function is specified by the second user based on the input operation by the second user. The image of the displayed range is output to the display as an image seen from the position of the character. [Selection diagram] Figure 1

Description

The present invention relates to an application control program, an application control method, and an application control system for allowing multiple users to share a virtual space.

2. Description of the Related Art A virtual reality game is known in which an image of a virtual space is displayed and played on a wearable display (hereinafter referred to as HMD: head-mounted display) worn on the head of a user. The HMD is equipped with a tracking sensor such as a gyro sensor, and can detect movement of the user's head. When the user moves his/her head, the HMD changes the image displayed on the display following the movement of the head. In this way, the game image follows the movement of the head, allowing the user to concentrate more on the game. That is, it is possible to enhance the sense of immersion in the game.

A game played by a plurality of users has also been proposed as one of such virtual reality games. Patent Literature 1 describes a game played by a plurality of users wearing HMDs. In this game, the position information of each user wearing the HMD is reflected in the position information of the avatar in the virtual space, and the avatar of each user is drawn in the virtual space.

JP 2014-17776 A

On the other hand, when a plurality of users want to play a virtual reality game, it is not always possible to prepare the same number of HMDs as the number of players. Moreover, there is a possibility that some users are reluctant to wear an HMD. Although various situations are assumed as described above, a game in which both a user without an HMD and a user with an HMD can participate, and in which the contribution of the user without an HMD to the application is enhanced. Not found.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and its object is to provide an application control program, an application control method, and an application control system that enable a user wearing a wearable display and a user not wearing the wearable display to play at the same time. to do.

An application control program for solving the above problem is an application control program for outputting an image of a virtual space to a wearable display worn by a first user and a display used by a second user, wherein one or more computers are provided with the first A first display function for outputting an image of the virtual space on the wearable display according to the posture of one user, and outputting the image of the virtual space on the display used by the second user based on the operation of the second user. a second display function, wherein the virtual space displayed by the first display function is the virtual space in which a character corresponding to the second user is placed, wherein the first display function is An image of the virtual space including the character is output to the wearable display according to the posture of the first user, and among the images of the virtual space displayed on the wearable display, the image is displayed on the display used by the second user. The displayed display range is identifiably displayed, and the second display function determines the display range of the virtual space specified by the second user based on the second user's input operation, and determines the display range. is output to the display as an image viewed from the position of the character.
According to this, the user who wears the wearable display and the user who does not wear the wearable display can play at the same time.

In the application control program, it is preferable that the computer further implements a position changing function of changing the position of the character based on the second user's input operation.

In the application control program, it is preferable that the first display function displays the display range as a fan-shaped range that spreads around the character.

In the application control program, it is preferable that the second display function causes the display to output a bird's-eye view image indicating the position of the character in the virtual space.

In the application control program, it is preferable that the first display function outputs to the wearable display an image of the virtual space in which the character is arranged above the position of the first user.

With respect to the application control program, it is preferable that the image of the virtual space displayed on the display used by the second user includes an object.

With respect to the application control program, the first display function, when the display range and the range of the image displayed on the wearable display overlap, displays an object included in the overlapping portion, and If there is an object that is not included in the overlapping portion, it is preferable to output the predetermined object to the wearable display as a non-display state.

The application control program further causes the computer to implement a first action reflection function for reflecting an action on an object according to an operation by the first user on an image of the virtual space, wherein the first action reflection function is the It is preferable to validate only the action for the displayed object.
The application control program further causes the computer to implement a second action reflection function for reflecting an action on an object according to an operation by the second user on an image of the virtual space, and the first display function is the application control program. With respect to the control program, it is preferable that an image of the virtual space reflecting an action by the operation of the second user is output to the wearable display.

An application control method for solving the above problem is an application control method for outputting an image of a virtual space on a display worn by a first user and a display used by a second user, wherein one or more computers a first display step of outputting the image of the virtual space on the wearable display according to the posture of one user; and outputting the image of the virtual space on the display used by the second user based on the operation of the second user. a second display step, wherein the virtual space displayed in the first display step is the virtual space in which a character corresponding to the second user is arranged, and in the first display step, An image of the virtual space including the character is output to the wearable display according to the posture of the first user, and among the images of the virtual space displayed on the wearable display, the image is displayed on the display used by the second user. displaying the displayed display range in an identifiable manner, and determining the display range of the virtual space specified by the second user based on the second user's input operation in the second display step; is output to the display as an image viewed from the position of the character.

According to this, the user who wears the wearable display and the user who does not wear the wearable display can play at the same time.

An application control system for solving the above problems is an application system that includes a wearable display worn by a first user and a control unit that outputs an image to the display used by a second user, and outputs an image of a virtual space, a first display control unit for outputting an image of the virtual space on the wearable display according to the posture of the first user; and an image of the virtual space on the display used by the second user based on the operation of the second user. the virtual space displayed by the first display control unit is the virtual space in which a character corresponding to the second user is arranged, wherein the first The display control unit outputs an image of the virtual space including the character to the wearable display in accordance with the posture of the first user, and out of the images of the virtual space displayed on the wearable display, the image of the second user. identifiably display the display range displayed on the display used by the second display control unit, the display range of the virtual space specified by the second user based on the second user's input operation Then, the image of the display range is output to the display as an image viewed from the position of the character.

According to this, the user who wears the wearable display and the user who does not wear the wearable display can play at the same time.

According to the present invention, a user wearing a wearable display and a user not wearing a wearable display can play at the same time.

FIG. 2 is a conceptual view of the application control system according to the first embodiment; FIG. 2 is a block diagram functionally showing the application control system of the same embodiment; FIG. 4 is a conceptual diagram illustrating a virtual space and an image visually recognized by a user in the same embodiment; FIG. 4 is an explanatory diagram of a second virtual space image output to the touch panel display of the same embodiment; FIG. 4 is an explanatory diagram of a bird's-eye view image output to the touch panel display of the same embodiment; FIG. 4 is an explanatory diagram of a second virtual space image output to the touch panel display of the same embodiment; FIG. 4 is an explanatory diagram of a first virtual space image output to the wearable display of the embodiment; FIG. 4 is an explanatory diagram of a first virtual space image output to the wearable display of the embodiment; FIG. 4 is an explanatory diagram of a processing procedure for outputting an image to a wearable display in the same embodiment; FIG. 4 is an explanatory diagram of a processing procedure for stopping a moving object in the same embodiment; FIG. 9 is an explanatory diagram of a second virtual space image output to the touch panel display of the second embodiment; FIG. 11 is an explanatory diagram of a second virtual space image output to the touch panel display of the third embodiment; FIG. 11 is an explanatory diagram of a first virtual space image output to the wearable display of the fourth embodiment; FIG. 5 is an explanatory diagram of a first virtual space image output to a touch panel display of a modified example; FIG. 9 is an explanatory diagram of a second virtual space image output to a touch panel display of a modified example; FIG. 11 is an explanatory diagram of a virtual space image output to a wearable display according to a modification, where (a) is an image output to the wearable display and (b) is an explanatory diagram of an image output to a touch panel display;

(First embodiment)
An embodiment of an application control program, an application control method, and an application control system will be described below with reference to FIGS. 1 to 10. FIG. In this embodiment, it is assumed that a game application is provided as an application for sharing a virtual space among multiple users.

As shown in FIG. 1 , the application control system includes a head mounted display (HMD) 11 as a wearable display, an operation unit (controller) 14 , a main display 12 and a touch panel display 13 . The HMD 11 is worn on the head of the first user 101 . An image of the virtual space is displayed on the HMD 11 . The first user 101 searches for an object that moves in the virtual space (hereinafter referred to as a moving object), and performs an action (second action) on the found moving object by operating the operation unit 14 . In this embodiment, the first user 101 searches for a moving object such as a “ghost” and shoots the moving object by turning on the operation unit 14 .

Images output to the main display 12 and the touch panel display 13 are viewed by the second user 102 . The second user 102 supports the first user 101 by operating the touch panel display 13 while viewing images output to the main display 12 and the touch panel display 13 . It should be noted that the second user 102 may be able to tell the first user 101 the position of the moving object with his/her own voice or voice input through a microphone.

A schematic configuration of the application control system 10 will be described with reference to FIG. The application control system 10 includes an HMD 11, an operation unit 14, a main display 12, a touch panel display 13, and a control device 15 connected thereto.

The HMD 11 has an HMD control section 30 , a tracking sensor 31 and a display section 32 . The HMD control unit 30 outputs a signal corresponding to the detection result of the tracking sensor 31 to the control device 15 and inputs various signals such as an image signal from the control device 15 . The tracking sensor 31 is a sensor for identifying the attention point of the first user 101 and detects the position and orientation of the first user's 101 head. The tracking sensor 31 has at least one sensor among, for example, a gyro sensor, an acceleration sensor, a geomagnetic sensor, and the like. As another embodiment, a tracking sensor 31 that directly detects the movement of the eyeball of the first user 101, for example, a line-of-sight detection sensor that detects near-infrared rays entering the iris and detecting the reflected light, is used as the tracking sensor 31. may be The display unit 32 of the HMD 11 displays an image of a predetermined range centering on the attention point of the first user 101 in the virtual space. That is, the image of the predetermined range is displayed so as to be visible to the first user 101 . The virtual space image output to the display unit 32 of the HMD 11 is hereinafter referred to as a "first virtual space image".

The operation unit 14 includes, for example, an operation button and an operation lever, and is configured so that the first user 101 can operate it without looking. The operation unit 14 has an acceleration sensor or the like, and can detect a change in direction from the initial position, the amount of change, and the like. The operation unit 14 outputs to the control device 15 a signal indicating ON/OFF of the operation by the second user 102 , the amount of change in the direction of the operation unit 14 , and the like.

The main display 12 is a display that can be visually recognized by the second user 102 . A first virtual space image is displayed on the main display 12 in synchronization with the display unit 32 of the HMD 11 .

The touch panel display 13 may be a touch panel display provided on a housing, or may be a mobile information terminal such as a tablet or a smart phone. An image obtained by projecting a part of the virtual space (hereinafter referred to as a second virtual space image) is output to the touch panel display 13 . In this embodiment, as the second virtual space image, an image of the inside of the virtual space viewed from a viewpoint set at a position above the first user 101 in the vertical direction is used. The initial screen of the second virtual space image displayed on the touch panel display 13 when the game is started is the same as the position of the virtual space projected onto the first virtual space image displayed on the display unit 32 of the HMD 11. . The viewpoint position of the second virtual space image on the initial screen may be the same as or different from the viewpoint position of the first virtual space image on the initial screen. The position at which the virtual space is projected onto the second virtual space image is changed and specified according to a predetermined touch operation such as a swipe operation by the second user 102 . For example, a swipe operation by the second user 102 changes the projection range of the virtual space and changes the second virtual space image. A display range of the second virtual space image is specified. Note that the “touch operation” includes, for example, a tap operation, a swipe operation, a flick operation, a pinch-in operation, a pinch-out operation, a touch-and-hold operation, and the like. Furthermore, the touch panel display 13 outputs a signal corresponding to the touch operation by the second user 102 to the control device 15 . For example, the touch panel display 13 outputs a signal according to the trajectory of the swipe operation, a signal according to the contact position, and the like.

The first virtual space image output to the display unit 32 of the HMD 11 and the second virtual space image output to the touch panel display 13 are images in the same virtual space. The difference is that the display range of the virtual space is changed according to the touch operation of the second user 102, whereas the latter changes the display range of the virtual space according to the second user 102's touch operation. Further, the touch panel display 13 displays a bird's-eye view image of the virtual space viewed from a predetermined position.

The control device 15 acquires signals output from the HMD 11, the operation unit 14, and the touch panel display 13, and manages game progress in the virtual space by the first user 101 and the second user 102. FIG.

The control device 15 includes a control section 16 , a storage section 17 and an image information storage section 25 . The control unit 16 has a calculation unit, a volatile storage unit, and a nonvolatile storage unit, and executes a game according to an application control program recorded in the nonvolatile storage unit. The storage unit 17 is a storage unit for image processing, and may be a volatile storage unit or a nonvolatile storage unit. Game management information 18 , attention point information 19 , specified display range information 20 , object position information 21 and contact position information 22 are recorded in this storage unit 17 .

The game management information 18 is information including a user ID, progress of the game, etc., and is updated as the game progresses. The point-of-interest information 19 is information on the position and orientation of the head of the first user 101 detected by the tracking sensor 31, or information indicating the position of the point of interest specified from the position and orientation of the head. The attention point information 19 is updated according to the position and orientation of the head of the first user 101 . The specified display range information 20 is information indicating the range displayed on the touch panel display 13 by the second user 102 by the touch operation, and is updated in accordance with the second user 102's touch operation. The object position information 21 is information indicating the position of a moving object that moves within the virtual space. The control unit 16 randomly moves each of the moving objects according to the application control program. The object position information 21 is updated as the position of the moving object changes. The position of the moving object is synchronized between the first virtual space image output to HMD 11 and main display 12 and the second virtual space image output to touch panel display 13 . Furthermore, the contact position information 22 indicates the contact position on the screen of the touch panel display 13, and is updated every time the second user 102 performs a touch operation. Information for drawing an image of the virtual space is recorded in the image information storage unit 25 .

The control unit 16 records attention point information 19 based on the signal input from the HMD 11 in the storage unit 17 . Further, the control unit 16 records a signal corresponding to the designated display range input from the touch panel display 13 in the storage unit 17 as the designated display range information 20 .

The control unit 16 acquires image information of a predetermined range centering on the point of interest based on the point-of-interest information 19 from among the image information recorded in the image information storage unit 25, and obtains the HMD 11 and the main display as a first virtual space image. output to 12. Further, the control unit 16 records the specified display range specified by the touch operation of the second user 102 in the storage unit 17 as the specified display range information 20 . Furthermore, the control unit 16 acquires image information from the image information storage unit 25 based on the designated display range information 20, and outputs the image information to the touch panel display 13 as a second virtual space image. In addition, the control unit 16 outputs to the touch panel display 13 a bird's-eye view image of part or all of the virtual space.

The virtual space and the first virtual space image output to the HMD 11 and the main display 12 will be described with reference to FIG. The HMD control unit 30 identifies the position and orientation of the head of the first user 101 . For example, the position and orientation of the head are based on the initial position. Yaw θy indicating the direction of rotation, Z-axis along the depth direction, and roll θr indicating the direction of rotation about the Z-axis in three axial directions and angles around the respective axes. The HMD control unit 30 outputs the positions in the three axial directions and the angles around the respective axes to the control unit 16 as point-of-interest information 19 .

The control unit 16 identifies the point of interest 114 or the line-of-sight direction D in the virtual space 110 based on the point of interest information 19 . As a method for specifying the point of interest or the line-of-sight direction, a known method according to the type of the tracking sensor 31 or the like can be used. For example, the control unit 16 virtually sets a straight line connecting the left eye position and the right eye position, and defines the line of sight direction D as a direction extending in a direction perpendicular to the straight line and passing through the midpoint of the left eye position and the right eye position. . Furthermore, the control unit 16 can specify the intersection of the line-of-sight direction D and the object (or background) as the point of interest 114 . If a tracking sensor that detects the movement of the eyeball or detects reflected light from within the iris is used, the line-of-sight direction may be specified based on the movement of the eyeball or the reflected light.

The control unit 16 acquires the image information of the attention point 114 and the attention range 111, which is the range based on the line-of-sight direction D, from the image information storage unit 25, and outputs it to the display unit 32 of the HMD 11 as a first virtual space image. For example, when the direction from the position of the viewpoint of the first user 101 to the point of interest 114 is defined as a line-of-sight direction D, the attention range 111 has a horizontal viewing angle α in the horizontal direction centered on the line-of-sight direction D, and a vertical direction of It is a range specified by a vertical viewing angle (not shown), which is an angle. The horizontal viewing angle α and the vertical viewing angle are not the physical viewing angles of the first user 101 but the rendering angles that generate the image of the virtual space 110 . The control unit 16 also outputs an image corresponding to the acquired image information to the main display 12 as well.

In the first virtual space image displayed on the HMD 11 and the first virtual space image displayed on the main display, the entire virtual space is displayed in an initial state when a predetermined action is not performed on the image. For example, the initial state is a "dark" state, ie, a state of low brightness and saturation of the image. In the first virtual space image in this initial state, the moving object 115 moving in the virtual space is displayed in a non-display state. The non-display state is an ambiguous state in which the moving object is not displayed at all or is difficult to see. Examples of an unclear display state include a state in which only a portion of the moving object 115 is displayed and the entirety is not displayed, or a state in which at least one of the brightness, saturation, and hue of the moving object 115 is brought closer to the background to make it difficult to see. is. Also, for example, the second user 102 is displayed as a character 116 such as a drone or an avatar in the virtual space. In this embodiment, the position of the character 116 maintains the position vertically above the first user 101 .

Further, when a display range is designated by second user 102's touch operation on touch panel display 13, control unit 16 acquires the designated display range from touch panel display 13 and stores it as designated display range information 20 in storage unit 17. to record. Further, the control unit 16 outputs the specified display range 112 according to the specified display range information 20 to the touch panel display 13 as the second virtual space image in a display mode different from the initial state of the first virtual space image. Specifically, the display mode at this time is a changed state in which at least one of brightness, saturation, and hue is different from the initial state. , and the saturation and hue are different from the initial state. Based on the object position information 21, the control unit 16 visibly displays the moving object in the second virtual space image. Thereby, the second user can visually recognize the moving object on the second virtual space image.

Furthermore, when at least a part of the attention range 111 of the first user 101 and the specified display range 112 specified by the second user 102 overlap, the control unit 16 controls the first virtual space image to The overlapping portion is displayed in a modified state different from the initial state. Specifically, the overlapping portion is displayed with at least part of the brightness, saturation, and hue different from those of the other portions. Note that FIG. 3 shows a state in which the entire specified display range 112 is included in the attention range 111 . In the first virtual space image, the light is displayed as if it were emitted from the character 116 itself or a light source possessed by the character 116 .

The control unit 16 designates the designated display range 112 designated by the second user 102 in the virtual space 110 (first action), and reflects the first action in the first virtual space image. Specifically, in the first virtual space image, the control unit 16 causes only the moving object 115 included in the overlapping portion (overlapping range in the virtual space) between the attention range and the specified display range 112 to be displayed (visible state). , and the moving object 115 existing outside the overlapping portion is put into a non-display state (invisible state). Therefore, the first user 101 can visually recognize the moving object 115 only when the attention range 111 and the specified display range 112 overlap and the moving object 115 is included in the overlapping portion.

Further, in the present embodiment, the specified display range 112 specified by the second user 102, in other words, the horizontal viewing angle β of the projection range in the virtual space for generating the second virtual space image is the first virtual space. It is smaller than the horizontal viewing angle α of the projection range in the virtual space for generation. That is, when the horizontal viewing angle β of the second virtual space image is greater than or equal to the horizontal viewing angle α of the first virtual space image, the first user 101 can easily view the specified display range 112 reflected in the first virtual space image. can be found in On the other hand, if the horizontal viewing angle β of the second virtual space image is smaller than the horizontal viewing angle α of the first virtual space image, the first user 101 finds the specified display range 112 reflected in the first virtual space image. become difficult. The ratio of the horizontal viewing angle β of the second virtual space image to the horizontal viewing angle α of the first virtual space image may be determined according to the target game balance. In this embodiment, by making the horizontal viewing angle β of the second virtual space image smaller than the horizontal viewing angle α of the first virtual space image, the degree of contribution of the second user 102 to the game is increased, and the first user 101 and the second user 102 cooperate to advance the game.

Further, the second user 102 can stop the movement of the moving object 115 near the contact position by performing a predetermined touch operation on the touch panel display 13 . At this time, the second user 102 performs a touch-and-hold operation to keep the finger in contact with an arbitrary position on the screen. When a touch-and-hold operation is performed, as a first action, control unit 16 identifies a moving object within a predetermined distance from the contact position, and stops movement (movement) of identified moving object 115 . In other words, one nearest moving object 115 stops moving with respect to one contact position. For example, if five moving objects 115 exist in the virtual space 110 and it is desired to stop their movements, a touch-and-hold operation is performed using five fingers.

Furthermore, the first user 101 can perform the second action only on the moving object 115 that is included in the specified display range 112 and becomes visible. In the present embodiment, shooting the moving object 115 is performed as the second action. At this time, if the movement of the moving object 115 is stopped by the touch operation of the second user 102, it becomes easier to aim at the moving object 115 and to perform the effective second action. Specifically, when the control unit 16 detects the ON operation of the operation unit 14 , the light ray 118 is projected from the virtual position 117 of the operation unit 14 along the direction of the operation unit 14 into the first virtual space image and the second virtual space image. display on the spatial image. Then, when the direction of light ray 118 and the position of moving object 115 match, control unit 16 determines that the second action has been effectively performed. Specifically, it is determined that the moving object 115 has been shot, and a score or the like is given to the first user 101 . On the other hand, the controller 16 does not display the action light ray 118 for the moving object 115 that is not included in the specified display range 112 . Alternatively, even if the action ray 118 is emitted toward the moving object 115 that is not included in the specified display range 112, it may be determined to be invalid. The control unit 16 causes the display unit 32 of the HMD 11, the main display 12, and the touch panel display 13 to display the light beam 118 for action.

Next, screens displayed on the display unit 32 of the HMD 11, the main display 12, and the touch panel display 13 will be described with reference to FIGS. 4 to 8. FIG.
As shown in FIG. 4, the control unit 16 renders the image information of the virtual space recorded in the image information storage unit 25 according to the specified display range 112, and renders an image for displaying the second virtual space image 121. Information is generated, the brightness, saturation and hue of the generated image information are adjusted for the second virtual space image 121 and output to the touch panel display 13 . Even if the second virtual space image 121 displays the same range as the first virtual space image, at least part of the brightness, saturation and hue are different from the initial state of the first virtual space image. Specifically, the control unit 16 causes the entire second virtual space image 121 to be displayed in a state of high brightness as if illuminated with light. Also, the control unit 16 causes the second virtual space image 121 to display the moving object 115 that moves based on the object position information 21 . However, since the moving object 115 moves randomly, if the specified display range 112 is not changed, the initially displayed moving object 115 may move outside the screen. The second user 102 performs, for example, a swipe operation to change the designated display range 112 to transition the screen, and searches for the moving object 115 in order to inform the first user 101 of the position of the moving object 115 .

As shown in FIG. 5, the control unit 16 acquires image information for a bird's-eye view image recorded in the image information storage unit 25 or renders image information of a virtual space to output a bird's-eye view image 122 to the touch panel display 13. . The bird's-eye view image 122 is a bird's-eye view of the entire virtual space, and displays, for example, the position of the character 116 . The control unit 16 displays the designated display range 112 designated by the second user 102 in the bird's-eye view image 122 in a modified state such as illuminated with light, and displays the other portions in an initial state such as darkness. . Further, the control unit 16 causes the moving object 115 to be displayed in the overhead image 122 .

As shown in FIG. 6, when the second user 102 performs a touch-and-hold operation on the second virtual space image 121 displayed on the touch panel display 13, the control unit 16 causes the finger to move within a predetermined distance from the contact position of the finger. The object 115 is specified and the movement of the specified moving object 115 is stopped. In FIG. 6, the control unit 16 stops the movement of the moving object 115 whose relative distance to each of the contact positions P1, P3, and P4 is within a predetermined distance. On the other hand, control unit 16 does not stop moving object 115 because it cannot identify moving object 115 in the vicinity of contact position P2. , does not stop its movement.

As shown in FIG. 7 , the control unit 16 outputs the first virtual space image 120 to the display unit 32 of the HMD 11 and changes the display range according to the posture of the first user 101 . If the control unit 16 determines that the attention range 111 of the first user 101 and the designated display range 112 designated by the second user 102 overlap, the control unit 16 displays the overlapped portion 125 as if illuminated with light. Display in changed state. In addition, the control unit 16 displays the portion other than the overlapping portion 125 in the first virtual space image 120 in an initial state such as darkness. Further, when the moving object 115 is included in the overlapping portion 125, the control unit 16 displays only the moving object 115 included in the overlapping portion 125, and does not display the moving object 115 not included in the overlapping portion 125.

As shown in FIG. 8 , when a signal corresponding to the ON operation is input from the operation unit 14 , the control unit 16 causes the HMD 11 to display the first virtual space image 120 reflecting the second action on the first virtual space image 120 . Output to unit 32 . For example, the control unit 16 directs a light ray 118 from a virtual position 117 of the operation unit 14 in the virtual space along the direction of the operation unit 14 based on a signal that is output from the operation unit 14 and detects the direction of the operation unit 14 . display. When the moving object 115 exists in the direction of the light ray 118, the control unit 16 validates the second action, ie, sniping. At this time, as described above, if the movement of the moving object 115 is stopped, it becomes easier to aim at the moving object 115, but it is also possible to shoot at the moving object 115 whose movement is not stopped. Also, for example, the control unit 16 calculates a score according to the number of times effective actions are performed within the time limit, and adds the score to the score information included in the game management information 18 .

Next, with reference to FIG.9 and FIG.10, the procedure of each process for providing a game to the 1st user 101 and the 2nd user 102 is demonstrated.
First, referring to FIG. 9, a procedure of processing for outputting the first virtual space image 120 to the HMD 11 and the main display 12 will be described. The process of outputting the first virtual space image 120 is started when a start condition is satisfied, such as when the operation unit 14 or another device is operated to start the game. In parallel with this process, the control unit 16 acquires the coordinates of the specified display range 112 of the touch panel display 13 and performs the process of recording the coordinates in the storage unit 17 as the specified display range information 20 .

The control unit 16 acquires attention point information 19 recorded in the storage unit 17 (step S1). Then, the control unit 16 identifies the attention range 111 based on the attention point information 19 . Also, the control unit 16 acquires the designated display range information 20 recorded in the storage unit 17 (step S2). Then, the control unit 16 specifies the designated display range 112 based on the designated display range information 20 .

After specifying the attention range 111 and the designated display range 112, the control unit 16 determines whether or not the attention range 111 and the designated display range 112 overlap (step S3). For example, based on the range of the attention range 111 in the virtual space coordinate system and the range of the designated display range 112 in the virtual space coordinate system, the control unit 16 determines whether or not these ranges overlap. When the control unit 16 determines that the attention range 111 and the specified display range 112 do not overlap (step S3: NO), it selects image information corresponding to the attention range 111 from among the image information recorded in the image information storage unit 25. The first virtual space image read out and set to the initial state is output to the display unit 32 of the HMD 11 and the main display 12 (step S8).

In step S3, when the control unit 16 determines that the attention range 111 and the specified display range 112 overlap (step S3: YES), based on the object position information 21, whether or not the moving object 115 is included in the overlapping portion is determined. (step S4). When the control unit 16 determines that the overlapping portion does not include the moving object 115 (step S4: NO), the overlapping portion is changed (step S7), and the first virtual space image is displayed on the display unit 32 of the HMD 11 and the main display. 12 (step S8). As a result, the first virtual space image 120 is displayed on the display unit 32 of the HMD 11 and the main display 12 with the overlapping portion changed and the moving object 115 hidden.

In step S4, when the control unit 16 determines that the moving object 115 is included in the overlapping portion between the attention range 111 and the designated display range 112 (step S4: YES), the overlapping portion is changed (step S5), and the overlapping portion is changed. The moving object 115 included in the portion is displayed (step S6). Then, the first virtual space image 120 is output to the display section 32 of the HMD 11 and the main display 12 (step S8). As a result, the overlapping portion is changed, and the first virtual space image 120 in which the moving object 115 is displayed in the overlapping portion is displayed on the display section 32 of the HMD 11 and the main display 12 .

After outputting the first virtual space image 120 in this manner, the control unit 16 returns to step S1, and repeats steps S1 to S8 until the game ends. The control unit 16 causes the light beam 118 to be displayed on the first virtual space image 120 when the operation unit 14 is turned on. Further, when the position of moving object 115 in the displayed state and the direction of operation unit 14 match, control unit 16 determines that the second action has been effectively performed.

Next, with reference to FIG. 10, a process of accepting a touch operation by the second user 102 and stopping the movement of the moving object 115 will be described. The process of stopping the movement of the moving object 115 is started when a start condition is met, such as when the control unit 16 detects a touch-and-hold operation. In parallel with this process, the control unit 16 acquires the coordinates of the specified display range 112 of the touch panel display 13 and performs the process of recording the coordinates in the storage unit 17 as the specified display range information 20 .

The control unit 16 acquires the contact position information 22 recorded in the storage unit 17 and identifies the contact position (step S10). Also, the control unit 16 determines whether or not at least one moving object 115 near the contact position has been specified (step S11). Specifically, the control unit 16 identifies the position of the moving object 115 based on the object position information 21 . Then, the relative distance between the specified contact position and the position of the moving object 115 is calculated, and it is determined whether or not there is a moving object 115 whose relative distance is within a predetermined distance.

When the control unit 16 determines that there is no moving object 115 whose relative distance from the contact position is within the predetermined distance (step S11: NO), the process ends without stopping the movement of the moving object 115 . When the control unit 16 determines that there is at least one moving object 115 whose relative distance from the contact position is within the predetermined distance (step S11: YES), the control unit 16 stops moving the specified moving object. (Step S12).

After stopping the movement of the identified moving object 115, the control unit 16 determines whether or not the finger of the second user 102 has been released from the screen of the touch panel display 13 (step S13). When the control unit 16 determines that the finger of the second user 102 is not released (step S13: NO), it continues to stop the movement of the specified moving object 115 (step S12). When determining that the finger of the second user 102 has been released (step S13: YES), the control unit 16 resumes movement of the moving object 115 (step S14), and ends the processing of the touch-and-hold operation. Then, when the start condition is satisfied, the control unit 16 repeats the processing from step S1 to step S13.

As described above, according to the first embodiment, the following effects can be obtained.
(1) In the above embodiment, the control unit 16 of the control device 15 outputs virtual space images to both the HMD 11 and the touch panel display 13 . The second user 102 specifies the specified display range 112 by touch operation without wearing the HMD 11, and the control unit 16 specifies the range according to the touch operation in the virtual space as a first action. Further, the first user 101 uses the operation unit 14 to operate the moving object 115 displayed in the overlapping portion of the attention range 111 and the designated display range 112, and the control unit 16 responds to the operation. , performs a second action on the moving object 115 . Therefore, in order for first user 101 to perform the second action, it is a prerequisite that second user 102 has performed the first action. You can work together to advance the game.

(2) In the above embodiment, the control unit 16 identifies the specified display range 112 specified by the second user 102 . Then, the control unit 16 reflects the designated display range 112 on the first virtual space image 120 . Therefore, the second user 102 can point the arbitrary range to the first user 101 without wearing the HMD 11 .

(3) In the above embodiment, when the specified display range 112 and the attention range 111 do not overlap, the control unit 16 outputs the moving object 115 to the HMD 11 in a non-display state, and displays the specified display range 112 and the attention range. 111, the moving object 115 included in the overlapping portion is output to the HMD 11 as a display state. That is, by specifying an arbitrary range including the moving object 115 by the second user 102 and including the range specified by the first user 101 in the attention range 111, the moving object 115 can be visually recognized by the first user. . Therefore, the first user 101 and the second user 102 can cooperatively advance the game.

(4) In the above embodiment, the control unit 16 replaces at least one of the brightness, saturation, and hue of the designated display range 112 with the brightness, saturation, and hue of the other ranges in the first virtual space image. Output to the HMD 11 in different states. According to this, the first user 101 wearing the HMD 11 can distinguish between the range designated on the touch panel display 13 and the other range. In addition, since the control unit 16 displays the range specified by the second user 102 on the touch panel display 13 as if it were illuminated with light, compared to the case where the range is surrounded by a line and pointed to the first user 101, the It is possible to suppress a decrease in the immersive feeling of the first user 101 .

(5) In the above embodiment, the control unit 16 validates the second action only for the moving object 115 included in the overlapping portion between the attention range 111 and the designated display range 112 . Therefore, in order to effectively perform the second action, it is premised that the moving object 115 is included in the designated display range 112 designated by the second user 102. to progress the game.

(6) In the above embodiment, the control unit 16 made the horizontal viewing angle β of the second virtual space image 121 smaller than the horizontal viewing angle α of the first virtual space image 120 . Therefore, since the first user 101 needs to perform an operation for the second action within a narrow range in the virtual space, the second action becomes more difficult and requires the assistance of the second user 102. . Therefore, the first user 101 and the second user 102 can cooperate to advance the game.

(7) In the above embodiment, the control unit 16 causes the touch panel display 13 to always display the moving object 115 . Accordingly, in order for the first user 101 to find the moving object 115, the second user 102 must specify the specified display range 112 to provide the position information of the moving object 115. FIG. Therefore, the first user 101 and the second user 102 can cooperate to advance the game.

(8) In the above embodiment, the control unit 16 moves the moving object 115 in the first virtual space image 120 output to the HMD 11 . Further, as a first action, the control unit 16 stops the movement of the moving object 115 in the first virtual space image 120 and the second virtual space image 121 output to the touch panel display 13 based on the touch operation on the touch panel display 13 . . This makes it easier for the first user 101 to shoot the moving object 115 whose movement has been stopped, which is the second action. Thereby, the 1st user 101 and the 2nd user 102 can cooperate and advance a game.

(9) In the above embodiment, the control unit 16 identifies the moving object 115 in the vicinity of the contact point on the screen of the touch panel display 13, and causes only the moving object 115 to stop moving. That is, the movement of one moving object 115 can be stopped at one contact point. The second user 102 only has to touch the vicinity of the moving object 115 whose movement is to be stopped, and it is intuitively easy to understand the action for stopping the movement. Also, when stopping the movement of a plurality of moving objects 115, the same number of contact points as the number of moving objects 115 is required. Therefore, the difficulty of stopping moving objects can be increased according to the number of moving objects 115 to be stopped.

(10) In the above embodiment, the control unit 16 outputs to the touch panel display 13 the bird's-eye view image 122 in which the moving object 115 is displayed and the virtual space is viewed from above. Therefore, the second user 102 can grasp the position and state of the moving object 115 included in the virtual space. Therefore, the moving object 115 targeted for sniping, which is the second action, can easily be included in the specified display range 112 . Also, the second user 102 can easily grasp the positional relationship between the specified display range 112 and the moving object 115 .

(Second embodiment)
Next, a second embodiment embodying the present invention will be described. In addition, since the second embodiment has a configuration in which the first action in the first embodiment is changed, the same parts are denoted by the same reference numerals, and detailed description thereof will be omitted.

In the first embodiment, specifying the specified display range 112 is defined as the first action, but in this embodiment, the first action is selection of the moving object 115 based on the operation of the second user 102 . Note that, in the present embodiment, the initial state of the first virtual space image 120 output to the HMD 11 may be at least the state in which the moving object 115 is not displayed. In other words, the first virtual space image 120 of the present embodiment does not have to be in a display state with low brightness such as "dark".

As shown in FIG. 11 , a moving object 115 is displayed in the second virtual space image 121 displayed on the touch panel display 13 . The second user 102 designates the moving object 115 by touch operation. Note that FIG. 11 shows an example in which there is one contact position P10, but there may be a plurality of contact positions.

The control unit 16 acquires the coordinates of the contact position P10 from the touch panel display 13, and based on the object position information 21, determines whether or not there is a moving object 115 whose relative distance from the coordinates of the contact position P10 is a predetermined distance or less. do.

When the control unit 16 determines that there is a moving object 115 whose relative distance from the contact position P10 is equal to or less than the predetermined distance, the control unit 16 displays the moving object 115 on the first virtual space image 120, and displays the display unit 32 of the HMD 11 and the display unit 32 of the HMD 11. Output to the main display 12 . Note that in the present embodiment, unlike the first embodiment, processing for stopping the movement of the moving object 115 in the vicinity of the contact position P10 is not performed.

Control unit 16 continues to display moving object 115 on the first virtual space image while the relative distance between contact position P10 and moving object 115 is maintained at a predetermined distance or less. On the other hand, even if the second user 102 continues the touch operation, when the relative distance between the contact position P10 and the moving object 115 exceeds the predetermined distance as the moving object 115 moves, the control unit 16 causes the moving object 115 to move. Hide.

As described above, in the present embodiment, the moving object 115 selected by the second user 102 is displayed in the first virtual space image 120, so the task of designating the moving object 115 becomes a finer task. As a result, close cooperation between the first user 101 and the second user 102 is required, and the degree of difficulty of the game can be made suitable to enhance interest.

As described above, according to the second embodiment, in addition to the effect (1) described in the first embodiment, the following effect can be obtained.
(11) In the above embodiment, the control unit 16 of the control device 15 identifies the moving object 115 selected by the second user 102 . Then, the control unit 16 causes the moving object 115 selected by the second user 102 to be displayed on the first virtual space image 120 . Therefore, the second user 102 can point the selected moving object 115 to the first user 101 without wearing the HMD 11 .

(Third Embodiment)
Next, a third embodiment embodying the present invention will be described. Since the third embodiment has a configuration in which the specified display range in the first embodiment is variable, the same reference numerals are given to the same parts, and detailed description thereof will be omitted.

In the first embodiment, the designated display range 112 is the entire range displayed on the touch panel display 13 by the second user 102 by the touch operation. The size and position of can be changed.

As shown in FIG. 12, the second user 102 performs a pinch-in operation by tracing the screen with two fingers close together, a pinch-out operation by tracing the screen with two fingers away from each other, or the like. Change its size or position by doing something. The designated display range 112 is the entire screen in the initial state. When a pinch-in operation is performed, the control unit 16 reduces the designated display range 112 based on the trajectory of the pinch-in operation, and when a pinch-out operation is performed, the pinch-out operation is performed. The specified display range 112 that has been reduced is enlarged based on the trajectory. Also, after performing a pinch-in operation or a pinch-out operation, when two fingers are moved while being in contact with the screen, the control unit 16 maintains the size of the designated display range 112 based on the input trajectory of the operation. The specified display range 112 is moved within the screen. Note that the light source position (origin) may be moved along with the movement of the designated display range 112 .

Also, a special effect may be generated for the moving object 115 by reducing the designated display range 112 . For example, the moving object 115 included in the designated display range 112 may not be moved outside the designated display range 112 at the time the reduction of the designated display range 112 is started. As a result, by reducing the specified display range 112 , the moving objects 115 included in the specified display range 112 can be collected in the reduced specified display range 112 . When the moving objects 115 are gathered in such a narrow range, the first user 101 tends to shoot many moving objects 115 at once.

When a pinch-in operation is performed, the control unit 16 reduces the specified display range 112 according to the length and angle of the input trajectory based on the input trajectory of the touch operation. Further, the control unit 16 outputs to the touch panel display 13 the second virtual space image 121 in which the brightness, chroma, and hue of the reduced designated display range 112 are changed differently from those of the other ranges. At this time, the second virtual space image 121 also displays a portion in the initial state, for example, a portion with low brightness or saturation expressing "darkness". At this time, the control unit 16 may display or hide the moving object 115 included in the initial state portion.

In addition, the control unit 16 determines whether or not there is an overlapping portion between the attention range 111 of the first user 101 and the specified display range 112, and if there is an overlapping portion, changes the overlapping portion, and changes the other portions. The first virtual space image 120 with the portion of .

When the number of moving objects 115 included in the designated display range 112 disappears or decreases due to effective sniping by the first user 101, the second user 102 performs a pinch-out operation to expand the designated display range 112 via the touch panel. This is done on the screen of the display 13. When the pinch-out operation is performed, the control unit 16 inputs the input trajectory of the touch operation, and expands the specified display range 112 according to the length and angle of the input trajectory. In addition, the control unit 16 outputs to the touch panel display 13 the second virtual space image 121 in which the enlarged designated display range 112 is changed and other portions are in the initial state. As a result, the designated display range 112 includes a new moving object 115 .

As described above, according to the third embodiment, in addition to the effects (1) to (10) described in the first embodiment, the following effects can be obtained.
(12) In the above embodiment, the control unit 16 changes the size and position of the specified display range 112 based on a touch operation on the screen of the touch panel display 13, so that only a part of the screen can be pointed, for example.

(Fourth embodiment)
Next, a fourth embodiment embodying the present invention will be described. In the fourth embodiment, the moving object is hidden on the touch panel display, and the moving object is displayed in the second virtual space image according to the operation of the operation unit. are denoted by the same reference numerals, and detailed description thereof will be omitted.

In the first embodiment, the first action is specifying the designated display range 112 based on the operation of the second user 102, and the second action is shooting the moving object 115 based on the operation of the first user. In this embodiment, the first action is the selection of the moving object 115 by the first user 101 , and the second action is the attack on the moving object 115 based on the touch operation of the touch panel display 13 by the second user 102 .

As shown in FIG. 13 , the control unit 16 outputs the first virtual space image 120 in which the moving object 115 is always displayed to the display unit 32 of the HMD 11 and the main display 12 . The operation unit 14 operated by the first user 101 is a controller capable of changing the direction, turning on and turning off, or a glove-type controller. The first user 101 operates the operation unit 14 to select the moving object 115 in the virtual space. Specifically, the control unit 16 changes the direction of the light source 130 such as a flashlight displayed in the first virtual space image 120 by operating the operation unit 14, and irradiates the moving object 115 with light to move it. Select object 115 . Alternatively, the moving object 115 is captured using tools such as nets and containers in the virtual space.

The control unit 16 receives the signal output from the operation unit 14 and determines whether or not the moving object 115 has been selected based on the object position information 21 . When determining that moving object 115 has been selected, control unit 16 outputs to touch panel display 13 second virtual space image 121 in which selected moving object 115 is displayed.

In the second virtual space image 121 output to the touch panel display 13, the moving object 115 is in a non-display state in the initial state. When the moving object 115 selected by the first user 101 is displayed on the second virtual space image 121, the second user touches the displayed moving object 115 to attack. When control unit 16 determines that moving object 115 has been attacked based on a signal input from touch panel display 13 , control unit 16 reflects the attack against moving object 115 in first virtual space image 120 and displays HMD 11 and main display 12 . output to

As described above, according to the fourth embodiment, the following effects can be obtained.
(13) In the above embodiment, the first user 101 selects the moving object 115 and causes the selected moving object 115 to be displayed on the second virtual space image 121 . The second user 102 can attack the moving object 115 displayed in the virtual space without wearing the HMD 11 . Therefore, the first user 101 and the second user 102 can cooperatively proceed with the game.

(Other embodiments)
It should be noted that each of the above-described embodiments can be modified as appropriate and implemented as follows.
- In each of the above embodiments, the control unit 16 sets the position of the character 116 corresponding to the second user 102 to the position above the first user 101 in the vertical direction. Instead of this, for example, the position of the character 116 may be changed based on a touch operation on the screen of the touch panel display 13 by the second user 102 .

- In each of the above embodiments, the horizontal viewing angle β of the second virtual space image is made smaller than the horizontal viewing angle α of the first virtual space image. Alternatively, the horizontal viewing angle β of the second virtual space image is set to be the same as the horizontal viewing angle α of the first virtual space image 120 or larger than the horizontal viewing angle α of the first virtual space image. good too. This makes it easier to find the designated display range 112, which is effective when, for example, it is desired to reduce the difficulty of finding an object. Similarly, the vertical viewing angle of the second virtual space image may be smaller than the vertical viewing angle of the second virtual space image. Alternatively, it may be equal to or larger than the vertical viewing angle of the second virtual space image.

- In each of the above embodiments, the control unit 16 causes the main display 12 to display the attention range of the first user 101 as the first virtual space image. Instead of or in addition to this, the control unit 16 may display the attention range 111 of the first user 101 on the second virtual space image 121, for example. The attention range 111 may be indicated by encircling it with a line, or may be displayed with at least one of brightness, saturation, and hue different from the background of the second virtual space image 121 . In this case, the main display 12 viewed by the second user 102 may be omitted.

- In each of the above-described embodiments, the control unit 16 outputs to the touch panel display 13 the bird's-eye view image 122 that is a bird's-eye view of the virtual space. However, the display of the bird's-eye view image 122 may be omitted when it is easy to grasp the position of the object in the virtual space.

- In each of the above embodiments, the control unit 16 causes at least one of the brightness, saturation, and hue of the designated display range 112 to be displayed differently from the brightness, saturation, and hue of portions other than the designated display range 112. . In addition to this, as the first action, the specified display range 112 may be displayed by enclosing it with a line displayed in the image, or by displaying a mark such as an arrow.

In each of the above-described embodiments, the control unit 16 makes at least one of the brightness, saturation, and hue of the designated display range 112 different from the brightness, saturation, and hue of the portion other than the designated display range 112 to make the first virtual It was displayed on the spatial image and the second virtual spatial image. In addition to this, the control unit 16 may change at least one of the brightness, saturation, and hue of the designated display range 112 over time to display. Specifically, when the control unit 16 displays the designated display range 112 with brightness, saturation, and hue as if it were illuminated with light, the designated display range 112 is darkened over time. Alternatively, at least one of lightness, saturation, and hue may be changed so that light of a color different from the light in the initial state is emitted.

- In each of the above-described embodiments, the control unit 16 stops the movement of the moving object 115 as the first action based on the touch position on the touch panel display 13 by the second user 102 . Instead of or in addition to this, the control unit 16 may stop the movement of the moving object 115 based on an item selection operation displayed on the screen of the touch panel display 13 . For example, when a special item displayed on the screen of the touch panel display 13 is selected, movement of all moving objects 115 displayed on the screen of the touch panel display 13 may be stopped. Also, when an item is dragged and dropped to the vicinity of the moving object 115, the movement of the moving object 115 may be stopped.

- In each of the above-described embodiments, the control unit 16 stops the movement of the moving object 115 as the first action based on the touch position on the touch panel display 13 by the second user 102 . Instead of or in addition to this, the movement of the moving object included within a predetermined range corresponding to one contact position may be stopped. The predetermined range corresponding to one contact position is, for example, a circular range centered on the contact position, or a rectangular or fan-shaped range with the contact position as a starting point (corner). Alternatively, a predetermined range may be selected by a swipe operation, tapping with two fingers, or the like.

- In each of the above embodiments, the second action is performed on the moving object 115 that is constantly moving. Alternatively or additionally, the second action may be performed on an object placed at a predetermined fixed position in the virtual space. Such objects are, for example, boxes, doors, musical instruments, etc. placed in the virtual space. In this case, the second action is, for example, an action such as opening a lid of a box, opening a door, or playing a musical instrument.

In each of the above embodiments, the moving object 115 that is not included in the overlapping portion of the attention range 111 and the specified display range 112 is not allowed to perform the second action, or is invalid even if the second action is performed. and Alternatively, the second action may also be performed on objects not included in the overlap. In this case, for example, the objects included in the overlapping portion may be displayed clearly so as to facilitate aiming, and the objects not included in the overlapping portion may be displayed indistinctly so as to facilitate aiming. Also, if an effective action is performed on an object that is not included in the overlapping portion, the score may be higher than if an effective action is performed on the object that is included in the overlapping portion.

- In each of the above-described embodiments, the predetermined object (moving object 115) is set to the non-display state in the first virtual space image output to the display unit 32 of the HMD 11 and the main display 12 . Alternatively, the object to be the target of the second action may be displayed in the first virtual space image. In this case, for example, the overlapping portion of the attention range 111 and the specified display range 112 is displayed in a different manner from the other portions, and only the moving object 115 included in the overlapping portion is enabled for the second action. can be Even in this case, the second action is valid only for a predetermined object included in the overlapped portion, so that the first user 101 and the second user 102 can cooperatively advance the game.

Alternatively, the display mode of the object or the virtual space image including the object may be different between the first virtual space image and the second virtual space image. For example, the second virtual space image can be magnified and scaled, or the object can be displayed at any angle, while the first virtual space image cannot be magnified and scaled, or can be displayed at a certain angle. can't see the object. Alternatively, the second virtual space image may be a bird's-eye view image of the virtual space including the object, and the bird's-eye view image may not be displayed in the first virtual space image. Alternatively, predetermined image processing such as transparency and edge extraction can be performed on the second virtual space image, but predetermined image processing cannot be performed on the first virtual space image.

14 and 15 are examples in which the second virtual space image is a bird's-eye view image of a virtual space including an object, and the first virtual space image is an image other than the bird's-eye view image. For example, the game played by the first user 101 and the second user 102 is a puzzle game that combines solid figures. As shown in FIG. 14 , the control unit 16 outputs to the HMD 11 or the like a first virtual space image 120 that is a virtual space image including a solid figure 150 that is an object, according to the posture of the first user 101 . The three-dimensional figure 150 is not particularly limited to a cube, a rectangular parallelepiped, a triangular prism, a cone, a sphere, or the like. The first user 101 operates the operation unit 14 to combine solid figures. Note that when a predetermined number or more of solid figures 150 are combined, it is difficult to include the entire pattern formed by combining the solid figures 150 in the viewing angle of the first virtual space image 120 . Therefore, when the first user 101 forms a target pattern, it is difficult to determine at least one of the position where the solid figure 150 should be placed and the type of the solid figure 150 to be placed.

As shown in FIG. 15 , the control unit 16 outputs to the touch panel display 13 a bird's-eye view image 122 that is a bird's-eye view of a pattern formed by combining solid figures 150 . Since the second user 102 can visually recognize the entire pattern in the bird's-eye view image 122, it is possible to easily determine which solid figure 150 should be placed at which position.

Further, as shown in FIG. 15, for example, when the second user 102 touches a predetermined position on the screen of the touch panel display 13, as shown in FIG. The position at which 150 should be placed is displayed with an indicated position mark 151 (first action). Alternatively, other than the indicated position mark 151, the position where the solid figure 150 should be placed may be displayed in a manner different from the display manner of the other portions. The control unit 16 arranges the solid figure 150 at a position based on the operation of the operation unit 14 by the first user 101 (second action).

In addition, in a puzzle game in which actions are performed based on the arrangement of the types of solid figures 150, for example, the second user 102 selects any solid figure 150 from the display area 153 for selecting the type of the solid figure 150, and selects the solid figure 150. Touch the position where the solid figure 150 should be placed (see FIG. 15). As shown in FIG. 14, the control unit 16 displays the type of solid figure 150 to be arranged with a type identification mark 152 (first action). Furthermore, the position where the three-dimensional figure 150 should be placed may be indicated by an indicated position mark 151 or the like. The first user 101 operates the operation unit 14 to place the solid figure 150 indicated by the type identification mark 152 at the position indicated by the indicated position mark 151 .

- In each of the above embodiments, as the first action, a range within the virtual space is specified as the specified display range 112 based on the touch operation on the touch panel display 13 . Alternatively, as the first action, a direction or arbitrary position within the virtual space may be indicated based on a touch operation on the touch panel display 13 .

FIG. 16 shows an example of a screen output in the maze game. As shown in FIG. 16A , the control unit 16 causes the display unit 32 of the HMD 11 or the like to display an image of a three-dimensional maze 160 provided in the virtual space as the first virtual space image 120 . The control unit 16 outputs the first virtual space image 120 according to the posture of the user, and controls the first virtual space image 120 so as to proceed through the three-dimensional maze 160 based on the operation of the operation unit 14 or the user position detected by the sensor. Modify the aerial image 120 . As shown in FIG. 16B , the control unit 16 causes the touch panel display 13 to display a bird's-eye view image of the three-dimensional maze 160 as the second virtual space image 121 . The control unit 16 displays the position 161 of the first user 101 on the second virtual space image 121 . While viewing the second virtual space image 121 , the second user 102 traces, for example, the direction in which the first user 101 should move along with the movement of the first user 101 . The control unit 16 acquires the input locus corresponding to the touch operation from the touch panel display 13, and displays the indicated direction on the first virtual space image 120 along the input locus (first action). For example, the direction indicated by the second user 102 may be displayed on the first virtual space image 120 as light with different brightness, saturation, and hue with respect to the background, or may be displayed as an indicator such as an arrow. You may The first user 101 proceeds through the three-dimensional maze 160 by operating the operation unit 14 or the like, referring to the direction indicated on the first virtual space image 120 .

- In each of the above embodiments, in addition to the HMD 11 that detects the position and orientation of the head of the first user 101, a position detection sensor that detects the position of the first user 101 in the physical space may be provided. The control unit 16 detects not only the position and orientation of the head of the first user 101 but also the position where the first user 101 stands by the position detection sensor, and generates image information of the virtual space according to the position where the first user 101 stands. to display the first virtual space image. In this way, the first virtual space image is changed as the first user 101 moves, so that the sense of immersion can be further enhanced.

- In each of the above-described embodiments, the game in which the image of the virtual space is output to the HMD 11 and the touch panel display 13 is embodied as a game in which a predetermined object is shot. Alternatively, it may be embodied in other games such as exploration games, card games, racing games, fighting games, simulation games, and role-playing games. Also, the action on the object is not limited to "sniping", and may be other actions such as transforming, throwing, hitting, scattering, splitting, rotating, moving, etc.

- In each of the above-described embodiments, the application that outputs the image of the virtual space to the HMD 11 and the touch panel display 13 is embodied as a game. Alternatively, an application that outputs an image of a virtual space to the HMD 11 and the touch panel display 13 may be used as an application other than a game, such as a simulation application used in fields other than games, such as the medical field and the education field, and an application for training. good too.

Next, technical ideas that can be grasped from the above embodiment and another example will be added below.
[1]
An application control program for outputting an image of a virtual space to a wearable display worn by a first user and a display used by a second user,
on one or more computers,
a first display function that causes the wearable display to output an image of the virtual space according to the posture of the first user;
realizing a second display function for outputting an image of the virtual space to a display used by the second user based on the operation of the second user;
The virtual space displayed by the first display function is the virtual space in which a character corresponding to the second user is arranged,
The first display function is
outputting an image of the virtual space including the character to the wearable display according to the posture of the first user;
identifiably displaying a display range displayed on the display used by the second user in the image of the virtual space displayed on the wearable display;
The second display function is
An application that determines the display range of the virtual space specified by the second user based on the second user's input operation, and outputs an image of the display range to the display as an image viewed from the position of the character. control program.
[2]
to the computer;
The application control program according to [1], further realizing a position change function of changing the position of the character based on the second user's input operation.
[3]
The application control program according to [1] or [2], wherein the first display function displays the display range as a fan-shaped range extending around the character.
[4]
The application control program according to any one of [1] to [3], wherein the second display function causes the display to output an overhead image showing the position of the character in the virtual space.
[5]
The application control according to any one of [1] to [4], wherein the first display function outputs to the wearable display an image of the virtual space in which the character is arranged above the position of the first user. program.
[6]
The application according to any one of [1] to [5], wherein an image of the virtual space displayed on the display worn by the first user and the display used by the second user includes an object. control program.
[7]
When the display range and the range of the image displayed on the wearable display overlap, the first display function sets an object included in the overlapping portion to a display state so that the object is not included in the overlapping portion. The application control program according to [6], which, if there is an object, outputs the predetermined object to the wearable display as a non-display state.
[8]
to the computer;
further realizing a first action reflecting function for reflecting an action on an object according to an operation by the first user on an image of the virtual space;
The application control program according to [7], wherein the first action reflecting function validates only an action for the object set in the display state.
[9]
to the computer;
further realizing a second action reflection function for reflecting an action on an object according to an operation by the second user on an image in the virtual space;
The first display function is
The application control program according to any one of [6] to [8], which outputs an image of the virtual space reflecting an action by the second user's operation to the wearable display.
[10]
An application control method for outputting an image of a virtual space to a wearable display worn by a first user and a display used by a second user,
one or more computers
a first display step of causing the wearable display to output an image of the virtual space according to the posture of the first user;
a second display step of outputting the image of the virtual space to the display used by the second user based on the operation of the second user;
The virtual space displayed in the first display step is the virtual space in which a character corresponding to the second user is arranged,
In the first display step,
outputting an image of the virtual space including the character to the wearable display according to the posture of the first user;
identifiably displaying a display range displayed on the display used by the second user in the image of the virtual space displayed on the wearable display;
In the second display step,
An application that determines the display range of the virtual space specified by the second user based on the second user's input operation, and outputs an image of the display range to the display as an image viewed from the position of the character. control method.
[11]
An application system that includes a wearable display worn by a first user and a control unit that outputs an image to the display used by a second user, and outputs an image of a virtual space,
a first display control unit that causes the wearable display to output an image of the virtual space according to the posture of the first user;
a second display control unit that outputs an image of the virtual space to a display used by the second user based on the operation of the second user;
The virtual space displayed by the first display control unit is the virtual space in which a character corresponding to the second user is arranged,
The first display control unit
outputting an image of the virtual space including the character to the wearable display according to the posture of the first user;
identifiably displaying a display range displayed on the display used by the second user in the image of the virtual space displayed on the wearable display;
The second display control unit
An application that determines the display range of the virtual space specified by the second user based on the second user's input operation, and outputs an image of the display range to the display as an image viewed from the position of the character. control system.

DESCRIPTION OF SYMBOLS 11... Head mounted display (HMD) as a wearable display, 12... Main display, 13... Touch panel display, 14... Operation part, 15... Control apparatus, 16... Control part, 17... Storage part, 18... Game management information, 19 Point of interest information 20 Designated display range information 21 Object position information 22 Contact position information 24 Gaze position specifying unit 25 Image information storage unit 30 HMD control unit 31 Tracking sensor 32 Display section 101 First user 102 Second user 110 Virtual space 111 Attention range 112 Specified display range 114 Attention point 115 Moving object 116 Character 118 Light ray 120... First virtual space image, 121... Second virtual space image, 122... Bird's-eye view image, 125... Overlapping part.

Claims (11)

An application control program for outputting an image of a virtual space to a wearable display worn by a first user and a display used by a second user,
on one or more computers,
a first display function that causes the wearable display to output an image of the virtual space according to the posture of the first user;
realizing a second display function for outputting an image of the virtual space to a display used by the second user based on the operation of the second user;
The virtual space displayed by the first display function is the virtual space in which a character corresponding to the second user is arranged,
The first display function is
outputting an image of the virtual space including the character to the wearable display according to the posture of the first user;
identifiably displaying a display range displayed on the display used by the second user in the image of the virtual space displayed on the wearable display;
The second display function is
An application that determines the display range of the virtual space specified by the second user based on the second user's input operation, and outputs an image of the display range to the display as an image viewed from the position of the character. control program. to the computer;
2. The application control program according to claim 1, further realizing a position change function of changing the position of said character based on said second user's input operation. 3. The application control program according to claim 1, wherein the first display function displays the display range as a fan-shaped range that spreads around the character. 4. The application control program according to any one of claims 1 to 3, wherein the second display function causes the display to output a bird's-eye view image showing the position of the character in the virtual space. 5. The application control program according to any one of claims 1 to 4, wherein the first display function outputs to the wearable display an image of the virtual space in which the character is arranged above the position of the first user. 6. The application control program according to any one of claims 1 to 5, wherein an image of said virtual space displayed on said display worn by said first user and said display used by said second user includes an object. . When the display range and the range of the image displayed on the wearable display overlap, the first display function sets an object included in the overlapping portion to a display state so that the object is not included in the overlapping portion. 7. The application control program according to claim 6, wherein if there is an object, the predetermined object is output to the wearable display as a non-display state. to the computer;
further realizing a first action reflecting function for reflecting an action on an object according to an operation by the first user on an image of the virtual space;
8. The application control program according to claim 7, wherein said first action reflection function validates only an action for said displayed object. to the computer;
further realizing a second action reflection function for reflecting an action on an object according to an operation by the second user on an image in the virtual space;
The first display function is
9. The application control program according to any one of claims 6 to 8, wherein an image of said virtual space reflecting an action by said second user's operation is output to said wearable display. An application control method for outputting an image of a virtual space to a wearable display worn by a first user and a display used by a second user,
one or more computers
a first display step of causing the wearable display to output an image of the virtual space according to the posture of the first user;
a second display step of outputting the image of the virtual space to the display used by the second user based on the operation of the second user;
The virtual space displayed in the first display step is the virtual space in which a character corresponding to the second user is arranged,
In the first display step,
outputting an image of the virtual space including the character to the wearable display according to the posture of the first user;
identifiably displaying a display range displayed on the display used by the second user in the image of the virtual space displayed on the wearable display;
In the second display step,
An application that determines the display range of the virtual space specified by the second user based on the second user's input operation, and outputs an image of the display range to the display as an image viewed from the position of the character. control method. An application system that includes a wearable display worn by a first user and a control unit that outputs an image to the display used by a second user, and outputs an image of a virtual space,
a first display control unit that causes the wearable display to output an image of the virtual space according to the posture of the first user;
a second display control unit that outputs an image of the virtual space to a display used by the second user based on the operation of the second user;
The virtual space displayed by the first display control unit is the virtual space in which a character corresponding to the second user is arranged,
The first display control unit
outputting an image of the virtual space including the character to the wearable display according to the posture of the first user;
identifiably displaying a display range displayed on the display used by the second user in the image of the virtual space displayed on the wearable display;
The second display control unit
An application that determines the display range of the virtual space specified by the second user based on the second user's input operation, and outputs an image of the display range to the display as an image viewed from the position of the character. control system.

Images