JP2025070872A - Information processing device, control method for information processing device, and control program for information processing device - Google Patents

Abstract

To provide an information processor that enables a user to further easily understand a point and a trigger at which an event occurs in a virtual space.SOLUTION: An information processor includes: a display unit that displays a virtual space image; a display control unit that displays a virtual space on the display unit according to the orientation of the display unit; and a determination unit that determines whether a predetermined first position on the display unit overlaps with a predetermined second position in the virtual space. The display control unit causes the second position on the virtual space image to be displayed in an identifiable manner, and causes the display unit to display a predetermined image corresponding to the second position in response to the determination determined by the determination unit that the first position and the second position overlap.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, a control method for an information processing device, and a control program for an information processing device.

Conventionally, in entertainment facilities, there is a video providing system that provides a user with a display device (head mounted display (HMD)) worn on the user's head according to the user's location (for example, Patent Document 1). In recent years, in addition to VR (Virtual Reality) as in Patent Document 1, XR (Cross Reality) technology including AR (Augmented Reality) and MR (Mixed Reality) has been spreading in various fields.

JP 2018-121132 A

When providing users with an experience using XR technology, there was a need to make it easier for users to understand the points and triggers at which events occur in the virtual space.

An information processing device according to one embodiment of the present invention includes a display unit that displays a virtual space image, a display control unit that causes the display unit to display a virtual space corresponding to the orientation of the display unit, and a determination unit that determines whether a predetermined first position on the display unit and a predetermined second position on the virtual space image overlap, and the display control unit displays the second position on the virtual space image in an identifiable manner, and causes the display unit to display a predetermined image corresponding to the second position in response to the determination by the determination unit that the first position and the second position overlap.

The information processing device according to one embodiment of the present invention may further include a generation unit that generates a virtual space image, and the display control unit may cause the display unit to display the virtual space image generated by the generation unit.

An information processing device according to one embodiment of the present invention may further include a communication unit that receives a virtual space image, and the display control unit may cause the virtual space image received by the communication unit to be displayed on the display unit.

In an information processing device according to one embodiment of the present invention, the display control unit may cause the display unit to display a time gauge for a period of time set according to the second position, the time gauge changing according to the period of time that the first position and the second position overlap.

In an information processing device according to one embodiment of the present invention, the display control unit may display a predetermined image when the overlap between the first location and the second location continues for a period of time that is set according to the second location.

In an information processing device according to one embodiment of the present invention, the first position may be the center of the display unit.

In an information processing device according to one embodiment of the present invention, the display control unit may change the display mode of the center of the display unit in the virtual space image displayed on the display unit.

A control method for an information processing device according to one embodiment of the present invention includes a step of displaying a virtual space image on the display unit according to the orientation of the display unit, and a step of determining whether a predetermined first position on the display unit overlaps with a predetermined second position on the virtual space image, the displaying step displays the second position on the virtual space image in an identifiable manner, and, in response to determining that the first position and the second position overlap, displays a predetermined image on the display unit according to the second position.

A control program for an information processing device according to one embodiment of the present invention causes the information processing device to realize a function of displaying a virtual space image on the display unit according to the orientation of the display unit, and a function of determining whether a predetermined first position on the display unit overlaps with a predetermined second position on the virtual space image, the display function displaying the second position on the virtual space image in an identifiable manner, and displaying a predetermined image corresponding to the second position on the display unit according to whether it is determined that the first position and the second position overlap.

According to one embodiment of the present invention, it is possible to provide an information processing device etc. that makes it easier for users to understand the points and triggers at which events occur in a virtual space.

FIG. 1 is a schematic diagram of a configuration of a video providing system according to an embodiment of the present invention. FIG. 2 is an example of a functional block diagram of an information processing apparatus according to an embodiment of the present invention. FIG. 3 is a flowchart showing an example of the operation of the information processing device according to an embodiment of the present invention. 4A to 4C are diagrams showing examples of transitions of the display unit of an information processing device according to one embodiment of the present invention. FIG. 5 is an example of an event information table stored in the video providing system according to an embodiment of the present invention.

Below, an embodiment of an information processing device according to the present invention will be described with reference to the drawings. Note that the present invention is not limited to this embodiment.

<System Configuration>
1 is a diagram showing an example of the configuration of a video providing system according to an embodiment of the present invention. The video providing system 500 is a system in which an image is provided to an information processing device 100 (100A, 100B, 100C) held by a user according to the position and orientation of the information processing device 100 by XR technology. The information processing device 100 is a display device having a communication function that can be held and moved by a user, and may be, for example, a smartphone, a tablet, a notebook computer, a wearable terminal (glasses-type device, watch-type device, smart glasses, etc.), an HMD, etc.

Hereinafter, one embodiment of the present invention will be described as an image providing system using AR technology, with a case in which a smartphone or tablet is used as the information processing device 100 as an example. Here, AR refers to a technology that extends the real world by overlaying computer-generated images or pictures onto the real world in front of the user, and the user can view an image in which virtual world content is displayed on an image of the real world via the camera of the information processing device 100. However, the present invention is not limited to this, and VR and MR may also be used.

The video providing system 500 includes an information processing device 100, a video providing server 200, and a database server 400. The video providing server 200 is connected to the information processing device 100 via a network 300, and transmits virtual video (content) to the information processing device 100 as necessary. The content provided by the video providing server 200 may be, but is not limited to, moving images, text messages, music, etc.

The information processing device 100 is a communication terminal of a user who uses the video provision service realized by the video provision system. An application for using the video provision service (hereinafter also referred to as an "AR application") is installed in the information processing device 100, and the information processing device 100 may be capable of transmitting and receiving various information to and from the video provision server 200. Note that although only three information processing devices 100 are shown in FIG. 1, there may be as many information processing devices 100 as there are users who use the video provision service.

The network 300 includes a wireless network and a wired network, and is a mobile communication system such as a wireless LAN (WLAN), LTE (long term evolution), fourth generation communication (4G), and fifth generation communication (5G). Note that the network 300 is not limited to these examples, and may be, for example, a short-range wireless communication (Bluetooth (registered trademark)) or an optical fiber line.

The database server 400 stores virtual images provided by the image providing system 500 and information about users who use the image providing system 500.

<Information processing device>
Next, an information processing device 100 according to an embodiment of the present invention will be described with reference to Fig. 2. The information processing device 100 includes a control unit 110, a communication unit 120, a display unit 130, an input/output unit 140, various sensors 150, and a storage unit 170.

The communication unit 120 is implemented as hardware such as a network adapter, communication software, or a combination of these, and transmits and receives various data to and from the video providing server 200 via the network 300. For example, the communication unit 120 receives a virtual space image from the video providing server 200.

The display unit 130 is a display that displays moving images transmitted from the control unit 110. The display unit 130 also displays images of the real world captured via a camera provided in the information processing device 100. In one embodiment of the present invention, the display unit 130 may be a touch panel, a touch display, or the like that is integrated with the input/output unit 140 described below.

The input/output unit 140 includes an input device that inputs various operations to the information processing device 100, and an output device that outputs the results of processing by the information processing device 100. The input device includes, for example, a touch panel, a touch display, a camera, and a microphone, and the output device includes, for example, a display, a touch panel, a speaker, etc.

The sensor 150 includes an acceleration sensor that detects the inclination of the information processing device 100 and a gyro sensor that detects the direction (orientation). The sensor 150 may also include a GPS (Global Positioning System) sensor that acquires position information of the information processing device 100 and a geomagnetic sensor.

The storage unit 170 stores various programs and data required for the operation of the information processing device 100. In other words, the storage unit 170 stores programs related to the application of the video providing service. The storage unit 170 includes, for example, a flash memory and a memory (such as a RAM (Random Access Memory) or a ROM (Read Only Memory)) that provides a working area for the control unit 110.

The control unit 110 is typically a processor, and is realized by a CPU, a GPU, etc. The control unit 110 reads out a program stored in the storage unit 170, and executes the code or instructions contained in the read out program, thereby realizing the functions of the acquisition unit 111, the display control unit 112, the determination unit 113, and the generation unit 114.

The acquisition unit 111 acquires information about the position and orientation of the device from the various sensors 150. For example, the acquisition unit 111 acquires latitude and longitude as position information of the device from a GPS sensor. The position information may include altitude. The acquisition unit 111 also acquires information about the inclination of the device from a gyro sensor and an acceleration sensor.

The display control unit 112 causes the display unit 130 to display a virtual space corresponding to the orientation of the device, based on the information acquired by the acquisition unit 111. The display control unit 112 also causes a predetermined second position in the virtual space, which is a point at which an event occurs, to be displayed in an identifiable manner. The determination unit 113 performs various determination processes. For example, the determination unit 113 determines whether a predetermined first position on the display unit 130 overlaps with a predetermined second position in the virtual space (details will be described later).

In one embodiment of the present invention, information regarding the virtual space image may be acquired in advance from the database server 400 according to the area in which the information processing device 100 is located, and stored in the storage unit 170. The generation unit 114 of the information processing device 100 may then generate a virtual space image according to the position and orientation of the information processing device 100 based on the information stored in the storage unit 170. Alternatively, the generation unit 114 may process an image captured by a camera to generate a virtual space image. The image displayed on the display unit 130 by the display control unit 112 may be a composite image of an image captured by the camera and an image generated by the generation unit 114.

Alternatively, the video providing server 200 may generate a virtual space image according to the position information of the information processing device 100 and transmit it to the information processing device 100.

<Examples of operation and display screen>
Next, functions of the information processing device 100 according to an embodiment of the present invention will be described based on a flowchart and an example of a display screen of the information processing device 100. Fig. 3 is a flowchart showing the operation of the information processing device 100, and Figs. 4(a) to (c) are examples of display screens of the information processing device 100. Note that the figures are merely examples, and the present invention is not limited to these.

First, the acquisition unit 111 acquires information about the position and inclination of the device from various sensors 150. As a result, the display control unit 112 causes the display unit to display a virtual space image corresponding to the orientation of the display unit 130 (step S11). Here, FIG. 4(a) is an example of a display screen of the display unit 130 on which a virtual space image is displayed. In the screen D10, an object 12 is displayed as a virtual space image on an image of the real world (in the example of the figure, trees, buildings including a passage extending in the depth direction of the screen, etc.) captured by a camera. Note that the image of the virtual space may be displayed so as to cover the image of the real world. The object 12 indicates a point where an event occurs when the user takes a predetermined action, and is set according to the position of the information processing device 100, that is, the position in the real world. Note that the object 12 may or may not be displayed depending on the position of the information processing device 100. In other words, when the information processing device 100 is within a predetermined distance from the object 12 and is within the angle of view of the information processing device 100, the object 12 may be displayed. This allows the video provider to display the virtual image in the way they want the user to see it. Information about the points is stored in the database server 400, and the communication unit 120 may receive information about the points set in the area in which the information processing device 100 is located in advance.

Figure 5 shows an example of a data table related to points at which events occur, stored in database server 400. Table TB10 may store an identifier (event ID (IDentifier)) that uniquely identifies an event occurring at each point, the position coordinates of the point, content provided in response to the occurrence of the event, and the startup time until the event occurs. For example, an event identified by event ID "action_001" displays "video A" as content when the user takes a specified action at the point at coordinates (x1, y1, z1). The specified action and the startup time until the event occurs will be described later. Also, the figure is an example, and the stored information is not limited to this. For example, the coordinates of the point may be spherical coordinates instead of orthogonal linear coordinates.

When the user recognizes that the object 12 is a trigger that generates an event, the user moves the information processing device 100 so as to overlap the marker 11 with the object 12. The determination unit 113 determines whether or not a predetermined first position on the display unit 130 overlaps with a predetermined second position in the virtual space (step S12 in FIG. 3). The predetermined first position on the display unit 130 may be, for example, the center point of the display unit 130. The predetermined second position in the virtual space may refer to the point described above. That is, the determination unit 113 determines whether or not the center point in the camera's shooting range is located on the object 12 based on the position information and tilt of the information processing device 100.

As shown in screen D10 in FIG. 4(a), the display control unit 112 may change the display mode of the center of the display unit 130 in the image displayed on the display unit 130. In the example of screen D10, the display control unit 112 causes the display unit 130 to display a marker 11 indicating the center of the display unit 130. The shape of the marker 11 is not limited to a circle, and any shape is acceptable as long as it allows the user to recognize that it is the center. For example, the display control unit 112 may cause the area near the center of the display unit 130 to blink, be colored, or be transparent. Alternatively, the center of the display unit 130 may be indicated by a beam-like line extending toward the user.

If the first position and the second position do not overlap (NO in step S12), the process returns to step S11, and the display of the virtual space image according to the orientation of the display unit 130 continues.

When it is determined that the first position (the center of the display unit 130) and the second position (point) overlap (YES in step S12 in FIG. 3), the display control unit 112 may display on the display unit 130 a time gauge for a time period set according to the second position, in which the gauge changes according to the time period during which the first position and the second position overlap. FIG. 4(b) shows an example of a display screen on which a time gauge is displayed. A time gauge 21 is displayed in the center of the screen D20. As described above, the time until an event occurs is set for each point in the table TB10 in FIG. 5, and the display control unit 112 displays the time gauge 21 so that it expires within the set time period. For example, the event identified by the event ID "action_001" may display a time gauge 21 for "2 seconds". The display form of the time gauge 21 is not limited to that shown in the figure, and may be in the shape of a time bar, and any form is acceptable as long as it allows the user to recognize the time until activation.

The display control unit 112 may cause the display unit 130 to display a predetermined image corresponding to the second position in response to the overlap between the first position and the second position extending over a period of time set according to the second position (step S13 in FIG. 3). FIG. 4(c) shows an example of a display screen on which a predetermined image is displayed. The example of screen D30 is an event identified by, for example, event ID "action_001", and "video A" in which birds 31 and 32 as virtual images jump out from the center (i.e., the position of object 12) is displayed by the display control unit 112.

Thereafter, if an operation to end the AR app by the user is detected (YES in step S14), the process ends. If an end operation is not detected, the process returns to step S11 and the display of the virtual space continues.

Thus, according to one embodiment of the present invention, a first position on the display unit 130 of the information processing device 100 is overlapped with a second position in the virtual space, and an event corresponding to the second position occurs. At this time, the second position, which is the point where the event occurs, is displayed in an identifiable manner, so that a video providing service that is easier for the user to understand and allows for greater immersion can be realized. Another advantage is that events in the virtual space can be enjoyed simply by inputting operations onto the screen, eliminating the need for a separate controller.

Furthermore, according to one embodiment of the present invention, information regarding the time until an event occurs is provided to the user in the form of a time gauge. This allows the user to enjoy the video providing service without worrying about when the event will occur.

Although the present invention has been described based on the drawings and examples, it should be noted that a person skilled in the art would easily be able to make various modifications and amendments based on the present disclosure. Therefore, it should be noted that these modifications and amendments are included within the scope of the present invention. For example, the functions included in each component, step, etc. can be rearranged so as not to cause logical inconsistencies, and multiple components, steps, etc. can be combined into one or divided. In addition, the configurations shown in the above embodiments may be combined as appropriate. For example, each function described as being provided by the information processing device 100 may be executed by the video providing server 200.

For example, in the above description, the first position is described as the center of the display unit 130, but the present invention is not limited to this and may be set at a location other than the center of the screen. For example, the first position may be the top edge of the screen, or may change over time or depending on the location. Alternatively, the user may be able to set the first position.

In addition, in the above, a manner in which a time gauge is displayed when the first position and the second position overlap has been described, but the time gauge may not be displayed and the user may be made aware that an event is occurring in another manner. For example, a countdown sound may be output or text may be displayed indicating that an event is occurring.

In the above, a case has been described in which the object is displayed in such a manner that the second position is identifiable. However, the second position does not have to be displayed in an identifiable manner. For example, the user may obtain a hint regarding the second position, and may be able to explore the second position by moving or displacing the information processing device 100 in various directions. That is, the object 12 may be transparent.

Also, in the above, an image providing system using AR technology has been described. However, the present invention can also be applied to an image providing system using VR technology or MR technology. When using VR technology, the display unit 130 in the information processing device 100 may be realized by an HMD, or the information processing device 100 itself may be realized by an HMD.

The programs of each embodiment of the present disclosure may be provided in a state stored in a storage medium readable by an information processing device. The storage medium can store the programs in a "non-transitory tangible medium." The programs include, for example, software programs and control programs. When each functional unit of the information processing device 100 is realized by software, the information processing device 100 functions as an acquisition unit 111, a display control unit 112, a determination unit 113, and a generation unit 114 by the processor executing the programs loaded on the memory.

The storage medium may, where appropriate, include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs), application specific ICs (ASICs), etc.), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable storage media, or any suitable combination of two or more of these. The storage medium may, where appropriate, be volatile, non-volatile, or a combination of volatile and non-volatile.

The program of the present disclosure may also be provided to the information processing device 100 via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves).

Furthermore, each embodiment of the present disclosure may be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission. The program of the present disclosure may be implemented using, for example, a scripting language such as JavaScript (registered trademark) or Python, C language, Go language, Swift, Koltin, Java (registered trademark), etc.

REFERENCE SIGNS LIST 100 Information processing device 110 Control unit 111 Acquisition unit 112 Display control unit 113 Determination unit 114 Generation unit 120 Communication unit 130 Display unit 140 Input/output unit 150 Sensor 170 Storage unit

Claims (9)

A display unit that displays a virtual space image;
a display control unit that causes a virtual space image corresponding to an orientation of the display unit to be displayed on the display unit;
a determination unit that determines whether a predetermined first position on the display unit and a predetermined second position in the virtual space overlap;
Equipped with
The display control unit displays the second position on the virtual space image in an identifiable manner, and, in response to the determination unit that the first position and the second position overlap, displays a predetermined image corresponding to the second position on the display unit. a generation unit for generating the virtual space image;
Further equipped with
The display control unit causes the virtual space image generated by the generation unit to be displayed on the display unit.
The information processing device according to claim 1 . A communication unit for receiving the virtual space image;
Further equipped with
The display control unit causes the virtual space image received by the communication unit to be displayed on the display unit.
The information processing device according to claim 1 . The display control unit causes the display unit to display a time gauge for a time period set according to the second position, the time gauge changing according to a time period during which the first position and the second position overlap.
The information processing device according to claim 1 . The display control unit causes the predetermined image to be displayed in response to an overlap between the first location and the second location spanning a time period set in accordance with the second location.
The information processing device according to claim 4. The first position is a center of the display unit.
The information processing device according to claim 1 . The display control unit changes a display mode of a center of the display unit in the virtual space image displayed on the display unit,
The information processing device according to claim 1 . A step of displaying a virtual space image according to an orientation of a display unit on the display unit;
determining whether a predetermined first position on the display unit overlaps with a predetermined second position in the virtual space;
Including,
The display step is a control method for an information processing device, in which the second position on the virtual space image is displayed in an identifiable manner, and a predetermined image corresponding to the second position is displayed on the display unit in response to a determination that the first position and the second position overlap. In the information processing device,
A function of displaying a virtual space image on the display unit according to the orientation of the display unit;
a function of determining whether a predetermined first position on the display unit and a predetermined second position in the virtual space overlap;
Realize this,
The display function is a control program of an information processing device that displays the second position on the virtual space image in an identifiable manner, and, when it is determined that the first position and the second position overlap, displays a predetermined image corresponding to the second position on the display unit.

Images