JP7260763B2 - Information processing device and program - Google Patents

Description

The present invention relates to an information processing apparatus and program.

2. Description of the Related Art Conventionally, a virtual reality (VR) technology has been put into practical use, in which a virtual space (virtual space) is displayed on a mobile information terminal typified by a smart phone and presented to a user. In recent years, for example, when a mobile information terminal is pointed at a specific place, an artificial character or structure, or an animal, plant, or structure captured by imaging or the like is displayed on the mobile information terminal and presented to the user. AR) technology is attracting attention. Below, an artificial character or structure, or a captured image is collectively referred to as an object.

For example, Patent Literature 1 discloses an information processing device and a program related to a technique of operating an object arranged in a virtual space by an input operation by a user's touch operation.

JP 2017-176764 A

One of the objects of the present invention is to make it possible to respond to inquiries about objects placed in virtual space based on the movement of the terminal.

A program comprising: displaying at least a portion of a virtual space in which an object is arranged on a display unit included in a terminal; outputting an inquiry regarding predetermined processing; and displaying at least a portion of the object on the display unit When displayed, the computer is caused to receive an input to an inquiry based on the movement of the terminal and execute a predetermined process based on the input to the inquiry.

causing a computer to set one of a plurality of operation modes to an object; and executing a predetermined process when a predetermined mode is set to the object and a query is output from the object good too.

A plurality of objects are arranged in the virtual space, one of a plurality of operation modes is set corresponding to each of the plurality of objects, and an input to an inquiry is accepted based on the movement of the terminal. and executing a predetermined process based on the input to the inquiry.

The motion of the terminal may be determined from an image obtained by synthesizing at least part of the virtual space in which the object is arranged and the image captured by the imaging unit.

When executing predetermined processing based on the movement of the terminal, the computer may be caused to fix the image captured by the imaging unit.

The computer may be caused to execute the predetermined processing based on the movement of the terminal when the amount of change in the parameter indicating the movement of the terminal is within a predetermined range.

An information processing device, comprising: a display control unit for displaying on a terminal at least part of a virtual space in which an object is arranged; an inquiry output unit for outputting an inquiry regarding predetermined processing; an execution unit that receives an input to the inquiry based on the movement of the terminal when the part is displayed, and executes a predetermined process based on the input to the inquiry.

According to one embodiment of the present invention, it is possible to respond to queries of objects placed in the virtual space based on the movement of the terminal.

1 is a block diagram showing an information processing system according to one embodiment of the present invention; FIG. 1A is a block diagram showing the hardware configuration of an information processing apparatus according to one embodiment of the present invention; FIG. 3B is a block diagram showing the functional configuration of the information processing apparatus according to one embodiment of the present invention; FIG. FIG. 4 is a diagram showing a virtual space in which objects are arranged according to an embodiment of the present invention; 1A, 1B, 1C and 1D are plan views showing the configuration of an information processing apparatus according to an embodiment of the present invention; FIG. (A), (B), (C) and (D) are perspective views showing the movement of the information processing apparatus according to one embodiment of the present invention. 4 is a flowchart illustrating a processing method according to one embodiment of the present invention; 4 is a flowchart illustrating a processing method according to one embodiment of the present invention; FIG. 4 is a diagram showing a virtual space in which a plurality of objects are arranged according to one embodiment of the present invention; 4 is a flowchart illustrating a processing method according to one embodiment of the present invention; (A) is a plan view showing a virtual space in which objects are arranged according to one embodiment of the present invention. (B) is a plan view showing a captured image according to one embodiment of the present invention. (C) is a plan view showing a diagram in which a virtual space and a captured image are superimposed according to one embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings and the like. However, the present invention can be implemented in many different aspects and should not be construed as being limited to the description of the embodiments exemplified below. Also, in this specification and each figure, the same reference numerals (or numerals followed by A, B, -1, -2, etc.) are attached to elements similar to those described above with respect to previous figures. Therefore, detailed description may be omitted as appropriate. The letters "first" and "second" for each element are convenient signs used to distinguish each element, and unless otherwise explained, have a further meaning. do not have.
1. First Embodiment 1-1. Information processing system

FIG. 1 is a block diagram showing an information processing system 1 according to one embodiment of the invention. The information processing system 1 includes a server 20, a plurality of information processing apparatuses 10, and a network 40 such as a communication line.

For example, when a user enters a room, the information processing system 1 according to one embodiment of the present invention can control whether or not to turn on lights in the room based on an inquiry from an object placed in the virtual space. can. An outline of the above example will be described. A standby mode and an inquiry mode are set for objects placed in the virtual space. Objects are normally set to wait mode. When the user moves the information processing device 10 in response to the object's inquiry, the information processing device 10 or the object executes whether or not to turn on the light.

In one embodiment of the present invention, the operation mode indicates a state of the information processing device 10 or an object, such as standby mode, inquiry mode, or the like. Here, the standby mode is a mode in which the information processing apparatus 10 or an object waits for an instruction (input) from the user or the server 20. It is a mode in which the user normally uses the information processing apparatus 10 . Inquiry mode is, for example, a mode in which an object inquires, questions, or interacts with a user.

In one embodiment of the present invention, the predetermined process is, for example, a series of processes for turning on the electricity. Hereinafter, the predetermined processing will be explained using an example of controlling whether or not to turn on the lights in the room.

In FIG. 1, server 20 is a computer that provides programs to information processing apparatus 10 via network 40 . Also, the server 20 is a computer that provides various services to the information processing apparatus 10 via the network 40 . Various services may include, for example, SNS (Social Networking Service) and provision of installation files for various applications. For example, when the server 20 receives an inquiry about provision of an application or program from the information processing device 10 , the server 20 may provide the information processing device 10 with an installation file of the application or the program. A user can connect to the server 20 via the network 40 by installing an application on the information processing device 10 and using the application installed on the information processing device 10 . The information processing device 10 also executes a predetermined process based on the provided program. Although the server 20 is illustrated as one device in FIG. 1, the server 20 may be composed of a plurality of devices. In addition, the server 20 stores information such as image information or video information such as objects, virtual space, imaging, inquiry output (are you sure?, yes or no), programs, and parameters required for the operation of the programs. It may have a storage device.

The information processing device 10 is, for example, a mobile information terminal such as a smart phone or a tablet, a mobile game machine, or a personal computer. Note that the plurality of information processing apparatuses 10 can communicate with each other.
1-2. Hardware configuration 100 of information processing device 10

FIG. 2A is a diagram showing the hardware configuration 100 of the information processing apparatus 10 according to one embodiment of the invention. Hardware configuration 100 is included in a personal digital assistant. A hardware configuration 100 includes a communication unit 101 , an operation unit 103 , a display unit 105 , an imaging unit 107 , a control unit 109 , a storage unit 111 and a detection unit 113 .

The communication unit 101 is a communication interface included in the information processing device 10 . In one embodiment of the present invention, the information processing device 10 is connected to the server 20 via the network 40 via the communication unit 101 . The information processing device 10 transmits and receives information to and from the server 20 . Also, the information processing device 10 may be connected to another information processing device 10 via the network 40 . In one embodiment of the invention, information may be replaced with data.

The operation unit 103 is, for example, a user interface such as buttons or a touch sensor. When the operation unit 103 detects that a user's operation is input to the interface, the operation unit 103 outputs various control signals corresponding to the input of the user's operation to the control unit 109 . In one embodiment of the invention, the operation unit 103 is, for example, a touch sensor.

The display unit 105 is provided integrally within a housing that constitutes the information processing apparatus 10 . The display unit 105 is, for example, a liquid crystal display panel or an EL (Electroluminescence) display panel. The display unit 105 displays a partial area of the virtual space in which the object is arranged. Although details will be described later, one object may be placed in the virtual space, or a plurality of objects may be placed. For example, as shown in FIG. 3, an object 300 is arranged in a partial area 310A of the virtual space 305 . Also, the object 300 is not arranged in the partial area 310B of the virtual space 305 .

The imaging unit 107 has an imaging element that converts light into an electrical signal, and includes, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor and a CCD (Charge Coupled Device) image sensor. The imaging unit 107 is, for example, a camera. The imaging unit 107 is driven based on imaging control signals output from the control unit 109 . The imaging unit 107 outputs the captured image information and video information to the control unit 109 , the storage unit 111 or the server 20 .

The control unit 109 has an arithmetic processing circuit and executes a program using the arithmetic processing circuit. In this specification and the like, an arithmetic processing circuit is, for example, a circuit capable of arithmetic processing such as a processor, a CPU (Central Processing Unit), or a GPU (Graphics Processing Unit). Note that the control unit 109 may receive the program stored in the server 20 via the communication unit 101 .

The storage unit 111 is a storage device capable of permanently holding information and rewriting information. Storage unit 111 is, for example, a non-volatile memory. The storage unit 111 stores image information or video information such as objects, virtual space, imaging, inquiry output (are you sure?, yes or no), information about the operation mode set for the object, programs, and programs necessary for the operation of the programs. Stores information such as parameters that become The storage unit 111 may have a rewritable storage device used for temporarily holding information. A rewritable storage device is, for example, a volatile memory, an SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory). The storage unit 111 may be used as a development area for the programs stored in the storage unit 111 and a storage area for storing intermediate data such as information output during the operation of the programs.

The detection unit 113 detects motion of the information processing device 10 and receives an input to an inquiry. Specifically, when the inquiry output signal is output from the inquiry output unit 205 , the detection unit 113 detects an instruction input to the information processing apparatus 10 by the user. Detecting an instruction input by the user to the information processing apparatus 10 by the detection unit 113 may be rephrased as detecting a user's response or detecting a movement given to the information processing apparatus 10 by the user. . The detection unit 113 has a sensor such as an angular velocity sensor (gyro sensor), a geomagnetism sensor, or an acceleration sensor that can detect a movement given to the information processing device 10 by the user. For example, when the detection unit 113 has an angular velocity sensor, when the inquiry output signal is output from the inquiry output unit 205 , the detection unit 113 detects the motion of the user moving the information processing apparatus 10 up and down with the angular velocity sensor.
1-3. Functional Configuration 200 of Information Processing Device 10

FIG. 2B is a diagram showing a functional configuration 200 of the information processing apparatus 10 executing the program according to one embodiment of the present invention. A functional configuration 200 is included in the control unit 109 . A functional configuration 200 includes a display control unit 201 , a setting unit 203 , a query output unit 205 and an execution unit 209 .

The setting unit 203 has a function of setting one of a plurality of operation modes for an object. Setting unit 203 receives an instruction (setting control signal) to set an operation mode from operation unit 103 by a user's touch operation on operation unit 103 . Upon receiving the setting control signal, the setting unit 203 sets the standby mode and inquiry mode to the object. Further, upon receiving the inquiry output signal from the inquiry output unit 205, the setting unit 203 causes the state of the object to transition from the standby mode to the inquiry mode. When the setting unit 203 transitions the object to the inquiry mode, the setting unit 203 outputs a transition signal to the inquiry output unit 205 .

The inquiry output unit 205 has a function of outputting an inquiry regarding predetermined processing to the display unit 105 . The inquiry output unit 205 outputs an inquiry output signal to the display control unit 201 , the setting unit 203 and the operation determination unit 207 . For example, when a user enters a room, a human sensor provided in the room reacts, and the human sensor outputs a sensor signal to the server. When the server receives the sensor signal, it outputs a detection signal to the inquiry output unit 205 . The inquiry output unit 205 receives a detection signal from the server. Further, upon receiving the detection signal from the server, the inquiry output unit 205 outputs an inquiry output signal (a signal indicating whether or not to turn on the electricity) to the display control unit 201 , the setting unit 203 and the operation determination unit 207 .

The motion determination unit 207 has a function of determining an input to an inquiry based on the motion detection result of the information processing apparatus 10 by the detection unit 113 . Specifically, the motion determination unit 207 determines whether or not an inquiry has been input based on the motion of the information processing apparatus 10 detected by the detection unit 113 , and outputs the determination result to the execution unit 209 . For example, when the motion determination unit 207 determines from the motion of the information processing apparatus 10 by the detection unit 113 that an input to the inquiry has been made, the motion determination unit 207 outputs an input detection signal to the execution unit 209 . For example, when the motion determination unit 207 determines from the motion of the information processing apparatus 10 by the detection unit 113 that an input for the inquiry has not been made, the motion determination unit 207 outputs an input non-detection signal to the execution unit 209 . For example, an input detection signal or an input non-detection signal that determines an input to an inquiry from the vertical and horizontal movements of the information processing device 10 is processed by the information processing device 10, and an input detection that determines an input to the inquiry from the tilt of the information processing device 10. Signals or input undetected signals may be processed at server 20 .

The execution unit 209 has a function of receiving an input detection signal or an input non-detection signal output from the motion determination unit 207 and executing predetermined processing. The execution unit 209 outputs a signal (execution control signal) for executing a predetermined process upon receiving an input detection signal or an input non-detection signal from the motion determination unit 207 . For example, when the execution unit 209 receives a signal (input detection signal) to turn on the lights from the action determination unit 207, it outputs an execution control signal to a switch that turns on the lights in the room. Then the lights in the room are turned on. The execution unit 209 may output an execution control signal to the server 20 . Then, upon receiving the execution control signal, the server 20 may output a signal for controlling the electricity in the room to the switch for turning on the electricity in the room, and the electricity in the room may be turned on.

The display control unit 201 has a function of displaying image information or video information on the display unit 105 . Specifically, the display control unit 201 controls image information or video information displayed on the display unit 105 . For example, when the display control unit 201 receives a display control signal from the operation unit 103, the display control unit 201 receives image information related to a partial area of the virtual space in which the object is arranged, image information captured by the imaging unit 107, or video information. is output to the display unit 105 . Further, for example, when the display control unit 201 outputs a signal indicating whether or not to turn on the light from the inquiry output unit 205, the display control unit 201 outputs an inquiry output (Are you sure?, Yes or No, etc.) to the display unit 105. .

Here, the server 20 may include a portion of the functional configuration 200 and execute a portion of the functions of the functional configuration 200 .
1-4. query output

Referring to FIG. 4, query output according to one embodiment of the present invention will be described. 4A, 4B, 4C, and 4D are plan views showing the configuration of the information processing apparatus 10 according to one embodiment of the present invention. Descriptions of configurations that are the same as or similar to those in FIGS. 1 to 3 may be omitted.

As shown in FIGS. 4A, 4B, 4C, and 4D, the information processing apparatus 10 has a display unit 105 and an imaging unit 107. FIG. 4A, 4B, 4C, and 4D, description of the hardware configuration other than the display unit 105 and the imaging unit 107 is omitted.

As shown in FIG. 4A, the display unit 105 displays a partial area 310A of the virtual space in which the object 300 is arranged. The display unit 105 may display a part of the object 300 in a partial area 310 in the virtual space. For example, the part above the waist of the object 300 may be displayed, or the face of the object 300 may be displayed.

4A shows image information related to a partial area of the virtual space 305 in which the object 300 is arranged when the display control unit 201 outputs a display control signal from the operation unit 103. FIG. 3 is a diagram output to a unit 105; FIG. FIG. 4A also shows a state in which the object 300 has been transitioned to the inquiry mode, but no inquiry has been output.

As shown in FIG. 4B, the display unit 105 may display a partial area 320B of the virtual space in which the object 300 is not arranged.

As shown in FIG. 4C, display unit 105 displays query output 330 in addition to the image in FIG. 4A. FIG. 4C is a diagram in which the display control unit 201 outputs an inquiry output (Are you sure?) to the display unit 105 when the inquiry output unit 205 outputs an inquiry output signal.

As shown in FIG. 4(D), the display unit 105 may display the query output 330-2 on top of the query output 330-1 while shifting the query output 330-2. FIG. 4D is a diagram in which the display control unit 201 outputs an inquiry output (Are you sure?) to the display unit 105 when the inquiry output unit 205 outputs the inquiry output signal twice.
1-5. Input for inquiry

5(A), 5(B), 5(C), and 5(D), input for a query according to an embodiment of the present invention will be described. 5(A), 5(B), 5(C) and 5(D) are perspective views showing the movement of the information processing apparatus that executes the program according to one embodiment of the present invention. Descriptions of configurations that are the same as or similar to those in FIGS. 1 to 4 may be omitted. In this specification and the like, an input to an inquiry is sometimes referred to as a movement of the terminal. 5A, 5B, 5C, and 5D, the display surface 105A of the display unit 105 of the information processing apparatus 10 is simply referred to as the display surface 105A.

In FIG. 5A, an input to an inquiry is made by moving the information processing apparatus 10 in a direction (vertical direction) substantially parallel to a normal line E substantially perpendicular to the horizontal direction (AB direction) of the display surface 105A. It is a figure which shows the case where it is performed. In FIG. 5B, an input to an inquiry is made by moving the information processing device 10 in a direction (horizontal direction) substantially parallel to a normal line F substantially perpendicular to the vertical direction (CD direction) of the display surface 105A. It is a figure which shows the case where it is performed. FIG. 5C is a diagram showing a case where an inquiry is input by moving the information processing apparatus 10 so as to be tilted with respect to a normal line G substantially perpendicular to the AB direction of the display surface 105A. FIG. 5D is a diagram showing a case where an inquiry is input by moving the information processing apparatus 10 so as to be tilted with respect to a normal line H substantially perpendicular to the CD direction of the display surface 105A. In this specification and the like, the horizontal direction is indicated by the direction along A and B, and the vertical direction is indicated by the direction along C and D. Descriptions of configurations that are the same as or similar to those in FIGS. 1 to 4 may be omitted.

As shown in FIG. 5A, when the user moves the information processing apparatus 10 up and down by a movement amount d1 within a range that satisfies a predetermined first condition, the detection unit 113 (FIG. 2A) is detected by the user. detects the motion of moving the information processing apparatus 10 up and down. When the detection unit 113 detects the motion of the information processing device 10 , the motion determination unit 207 (FIG. 2B) outputs an input detection signal to the execution unit 209 . On the other hand, when the user moves the information processing device 10 up and down by a movement amount d1 within a range that does not satisfy the predetermined first condition, when the detection unit 113 detects the movement of the information processing device 10, the motion determination unit 207 inputs A non-detection signal is output to execution unit 209 . The execution unit 209 again prompts for the input of the inquiry. As a result, for example, the display unit 105 displays an image similar to the query output shown in FIG. 4(C). Further, as shown in FIG. 4D, the display unit 105 may display the inquiry output 330-2 on the inquiry output 330-1 while shifting the inquiry output 330-2.

As shown in FIG. 5B , when the user moves the information processing device 10 left or right by a movement amount d2 within a range that satisfies the second condition, the detection unit 113 detects the movement of the information processing device 10 . Then, the motion determination section 207 outputs an input detection signal to the execution section 209 . On the other hand, when the user moves the information processing device 10 left or right by a movement amount d2 within a range that does not satisfy the second condition, when the detection unit 113 detects the movement of the information processing device 10, the motion determination unit 207 inputs A non-detection signal is output to execution unit 209 . The execution unit 209 again prompts for the input of the inquiry.

As shown in FIG. 5C , when the user tilts the information processing device 10 within a range where the angle θx satisfies the third condition, the detection unit 113 detects the movement of the information processing device 10 and the motion is detected. The determination unit 207 outputs the input detection signal to the execution unit 209 . On the other hand, when the user tilts the information processing device 10 in a range where the angle θx does not satisfy the third condition, when the detection unit 113 detects the movement of the information processing device 10, the motion determination unit 207 outputs an input undetected signal. to the execution unit 209 . The execution unit 209 again prompts for the input of the inquiry.

As shown in FIG. 5D , when the user tilts the information processing apparatus 10 within a range where the angle θy satisfies the predetermined fourth condition, the detection unit 113 detects the motion of the information processing apparatus 10 and the motion is detected. The determination unit 207 outputs the input detection signal to the execution unit 209 . On the other hand, when the user tilts the information processing device 10 in a range where the angle θy does not satisfy the fourth condition, when the detection unit 113 detects the movement of the information processing device 10, the motion determination unit 207 outputs an input undetected signal. to the execution unit 209 . The execution unit 209 again prompts for the input of the inquiry.

For example, when an inquiry is input, driving of the imaging unit 107 is stopped. Further, when the input for the inquiry is executed, the image captured by the imaging unit 107 displayed on the display unit 105 may be fixed, or the image captured by the imaging unit 107 may not be displayed on the display unit 105. good too. This makes it possible to distinguish between the user's normal use of the information processing device 10 and the user's input in response to an object inquiry by moving the information processing device 10 . Therefore, erroneous operation when the user inputs an object inquiry is suppressed.
1-6. Processing method for executing the program

A processing method according to an embodiment of the present invention will be described with reference to FIGS. 1, 2, 4, 5, 6 and 7. FIG. 6 and 7 are flowcharts illustrating a processing method according to one embodiment of the present invention. Descriptions of configurations that are the same as or similar to those in FIGS. 1 to 5 may be omitted.

The information processing apparatus 10 starts a predetermined process when executing the program. For example, the information processing apparatus 10 activates an application according to a user's instruction. First, in the setting unit 203 (FIG. 2(B)), the standby mode and inquiry mode are set for the object (step 501 (S501)).

In step 503 (S503), for example, when the user performs a touch operation on the operation unit 103, the information processing apparatus 10 transmits one of a plurality of operation modes from the server 20 to the display control unit 201 (FIG. 2B). is set and the image data of the virtual space is read. The display control unit 201 arranges the object in the virtual space based on the read object and the image data of the virtual space. The object is in a state (standby mode) waiting for an instruction (input) from the user or the server 20 . The above is step 503 (S503).

At step 505 (S505), if there is an input from the user or the server 20 (Yes), the setting unit 203 (FIG. 2B) causes the object to transition from standby mode to inquiry mode. If there is no input from the user or server 20 (no), step 505 (S505) is repeated and the object remains in standby mode. For example, when a user enters a room, the inquiry output unit 205 outputs an inquiry output signal to the setting unit 203 according to a signal output from the server. Upon receiving the inquiry output signal, the setting unit 203 causes the object to transition from the standby mode to the inquiry mode. When the setting unit 203 transitions the object to the inquiry mode, the setting unit 203 outputs a transition signal to the inquiry output unit 205 . The above is step 505 (S505).

If step 505 (S505) has already been performed, step 501 (S501) and step 503 (S503) may be omitted.

At step 507 (S507), the inquiry output unit 205 receives the transition signal from the setting unit 203 and outputs an inquiry output signal regarding predetermined processing. For example, the inquiry output unit 205 outputs a signal indicating whether to turn on the lights in the room to the display control unit 201 , the setting unit 203 and the operation determination unit 207 . The above is step 507 (S507).

At step 509 (S509), when the display control unit 201 receives a signal indicating whether or not to turn on the lights in the room, data of a partial area in the virtual space is output to the display unit 105 (FIG. 2A). do. A partial area of the virtual space is displayed on the display unit 105 . At this time, as shown in FIG. 4C, if the partial area 310A of the virtual space in which the object 300 is placed and the query output 330-1 are displayed on the display unit 105 (Yes), The information processing device 10 executes step 513 (S513). That is, when the object 300 performs the inquiry output 330-1 to the user, the information processing apparatus 10 executes step 513 (S513). As shown in FIG. 4B or 4C, when a partial area of the virtual space in which no object is arranged is displayed on the display unit 105 (No), the information processing apparatus 10 returns to step 505 (S505). That is, if the object 300 has not performed the inquiry output 330-1 to the user, or if the object is not displayed, the information processing apparatus 10 returns to step 505 (S505). The above is step 509 (S509). Note that the user may move the information processing apparatus 10 so that a partial area of the virtual space in which the object is arranged is displayed on the display unit 105 .

Subsequently, the user inputs the object inquiry by moving the information processing apparatus 10 . For example, when the display control unit 201 displays an inquiry output as to whether or not to turn on the light on the display unit 105, the detection unit 113 (FIG. 2A) detects the movement of the information processing apparatus 10 by the user. . The above is step 513 (S513).

When the motion of the information processing device 10 is the first motion, when the input detection signal is output from the motion determination unit 207 to the execution unit 209, the execution unit 209 executes the first predetermined process (step 521). (S521)). Further, when the motion of the information processing apparatus 10 is the second motion, when an input non-detection signal is output from the motion determination unit 207 to the execution unit 209, the execution unit 209 executes the second predetermined process. (Step 531 (S531)).

For example, when the object 300 outputs an inquiry to the user as to whether or not to turn on the light, the user moves the information processing device 10 up and down as shown in FIG. Yes" (first move) can be entered. The operation determination unit 207 determines that “yes” has been input in response to the inquiry as to whether or not to turn on the lights, and outputs a signal to turn on the lights to the execution unit 209 . Further, the execution unit 209 outputs an execution control signal to the server 20 to turn on the lights in the room. Further, when the object 300 outputs an inquiry to the user as to whether or not to turn on the light, the user moves the information processing device 10 left and right as shown in FIG. No" (second move) can be entered. The operation determination unit 207 determines that “No” is input in response to the inquiry as to whether or not to turn on the lights, and outputs a signal not to turn on the lights to the execution unit 209 . Furthermore, the execution unit 209 outputs an execution control signal to the server 20 so that the lights in the room are not turned on. The above is step 521 (S521) and step 531 (S531). Then, the operation of the information processing apparatus 10 is completed.

When the operation is completed, the information processing apparatus 10 may return to the standby mode shown in step 507 (S507) and execute step 509 (S509).

Although the input for the inquiry according to the embodiment of the present invention is shown as an example of inputting two options of "yes" and "no", it is not limited to this example. For example, the input may be three choices of "yes", "no" and "not sure", or four or more choices. Inputs for inquiries according to an embodiment of the present invention can also correspond to complicated predetermined processing by increasing options. In addition, although the input to the inquiry according to the embodiment of the present invention is illustrated as an example in which "yes" and "no" correspond to FIGS. 5(A) and 5(B), the present invention is not limited to this example. For example, "yes" and "no" may correspond to FIGS. 5(B) and 5(A).

As an example of the predetermined processing according to one embodiment of the present invention, an example in which a response to an inquiry as to whether or not to turn on the electricity of an object is made by the movement of the information processing device of the user has been shown. Predetermined processing is not limited to this example. For example, a query related to movement of an object may be responded to by movement of the user's information processing device.

As described above, by using the information processing apparatus and program according to one embodiment of the present invention, the user can respond to an inquiry about an object placed in a virtual space by simply moving the information processing apparatus. can be entered.

Further, by using the information processing device and the program according to one embodiment of the present invention, the user can input a response by moving the information processing device. Reduces transportation complexity.
2. Second embodiment

A processing method when a plurality of objects are arranged in the virtual space will be described with reference to FIG.

FIG. 8 is a diagram showing a virtual space in which a plurality of objects are arranged according to one embodiment of the present invention. Descriptions of configurations that are the same as or similar to those in FIGS. 1 to 7 may be omitted.

FIG. 8 differs from the diagram shown in FIG. 3 in that a plurality of objects (object 300A, object 300B, and object 300C) are arranged in virtual space 305. FIG. Also, FIG. 8 shows an example in which an inquiry mode is set for an object.

As shown in FIG. 8, an object 300A and a query output 330A are arranged in a partial area 310A of the virtual space 305 . Object 300B and query output 330B are arranged in partial area 310B of virtual space 305 . An object 300C is arranged in a partial area 310C of the virtual space 305 . No object is placed in a partial area 310D of the virtual space 305 .

Even when a plurality of objects (object 300A, object 300B) are arranged in the virtual space 305, the flowcharts of FIGS. 6 and 7 can be used.

For example, regarding the processing method when a plurality of objects are arranged in the virtual space of the present invention, after steps 501 (S501) to 507 are executed based on the flowcharts of FIGS. ) is executed.

If the partial area 310A of the virtual space 305 in which the object 300A is arranged or the partial area 310B of the virtual space 305 in which the object 300B is arranged is displayed on the display unit 105 of the information processing terminal (Yes), At step 513 ( S<b>531 ), the user can use the information processing device 10 to input an inquiry. On the other hand, when a partial area 310C of the virtual space 305 in which the object 300C is arranged is displayed on the display unit 105, the object 300C is in the inquiry mode but does not output an inquiry (No), so the information processing apparatus 10 returns to step 505 (S505). Further, when the partial area 310C of the virtual space 305 in which no object is arranged is displayed on the display unit 105 (No), the information processing apparatus 10 performs step 505 ( S505).

Although there are three objects in the inquiry mode, the user can use the information processing apparatus 10 to input the inquiry to the object displayed on the display unit 105 by outputting the inquiry.

For example, when both a partial area 310A of the virtual space 305 and a partial area 310B of the virtual space 305 are displayed on the display unit 105, the user performs information processing on either the object 300A or the object 300B. After inputting an inquiry using the device 10, the user inputs an inquiry using the information processing device 10 to the other of the object 300A or the object 300B. Further, for example, the information processing apparatus 10 may have a function capable of simultaneously inputting inquiries to a plurality of objects in the inquiry mode.

For example, each of a plurality of objects may be operated by different users or may be operated by the same user. The number of displayed objects is not limited to the numbers shown in FIGS. The operation of objects, the number of objects to be displayed, and the like may be appropriately determined by the user.

As described above, by using the information processing apparatus and program according to one embodiment of the present invention, the user can input responses to inquiries about a plurality of objects by simply moving the information processing apparatus. be able to.
3. Third embodiment

A processing method according to an embodiment of the present invention will be described with reference to FIGS. 9 and 10. FIG. A program according to an embodiment of the present invention superimposes an image captured by the imaging unit 107 included in the information processing apparatus 10 and an object placed in a virtual space, and responds to an inquiry about the object by displaying information. Including responding with movement of the processor 10 .

FIG. 9 is a flowchart illustrating a processing method according to one embodiment of the invention. FIG. 10A is a plan view showing a virtual space in which objects are arranged according to one embodiment of the present invention. FIG. 10B is a plan view showing a captured image according to one embodiment of the present invention. FIG. 10C is a plan view showing a superimposed view of the virtual space and the captured image according to one embodiment of the present invention. Descriptions of configurations that are the same as or similar to those in FIGS. 1 to 8 may be omitted.

Since step 501 (S501) has already been explained using the flowchart shown in FIG. 6, the explanation here is omitted. As described using the flowchart shown in FIG. 6, if step 505 (S505) has already been performed, step 501 (S501) and step 503 (S503) may be omitted.

At step 504 (S504), for example, the user reads the image data of the object 300 and the virtual space 305 set to one of a plurality of operation modes, and the image data of the captured image 410 from the server 20. FIG. The read image data of the object 300 and the virtual space 305 and the image data of the imaged image 420 are input to the display control unit 201 (FIG. 2B) via the network 40 . Subsequently, the display control unit 201 generates synthesized image data of a synthesized image 500 in which the image data of the partial area 310 in the virtual space 305 in which the object 300 is arranged and the image data of the captured image 410 are superimposed. Generate. The display control unit 201 displays on the display unit 105 a composite image 500 in which the image of the partial area 310 of the virtual space 305 in which the object 300 is arranged and the captured image 410 are superimposed. The above is step 504 (S504). Here, as in step 503 (S503), the object is in a state of waiting for an instruction (input) from the user or server 20 (standby mode).

Since the steps after step 505 (S505) have already been described using the flow charts shown in FIGS. 6 and 7, description thereof will be omitted here.

As described above, by using the information processing apparatus and the program according to the embodiment of the present invention, an image of a partial area of the virtual space in which the object is arranged and the captured image are displayed in an overlapping manner. Even in such a case, the user can input a response to an inquiry about an object by simply moving the information processing device. That is, a simple motion of the information processing device is determined from an image obtained by synthesizing an image of a partial area of the virtual space in which the object is arranged and a captured image. Therefore, by using the information processing device and the program according to the embodiment of the present invention, it is possible to input a response to an inquiry about an object including a captured image by a simple movement of the information processing device. , the discontinuity of the operation that occurs in the case of the touch operation, and the complexity of moving the finger are improved.

All the processes executed in the above-described embodiments of the present invention may be predetermined processes.

Each of the embodiments described above as embodiments of the present invention can be implemented in combination as appropriate as long as they do not contradict each other.

Even if there are other effects different from the effects brought about by the aspects of each embodiment described above, those that are obvious from the description of this specification or those that can be easily predicted by those skilled in the art are of course It is understood that it is provided by the present invention.

1: information processing system, 10: information processing apparatus, 20: server, 40: network, 100: hardware configuration, 101: communication unit, 103: operation unit, 105: display unit, 105A: display surface, 107: imaging unit , 109: control unit, 111: storage unit, 113: detection unit, 200: functional configuration, 201: display control unit, 203: setting unit, 205: output unit, 207: operation determination unit, 209: execution unit, 300: Object 300A: Object 300B: Object 300C: Object 305: Virtual space 310: Partial area 310A: Partial area 310B: Partial area 310C: Partial area 310D: Partial area 320B : partial area, 330: output, 330-1: output, 330-2: output, 330A: output, 330B: output, 410: image, 420: image, 500: composite image

Claims (9)

displaying, on a display unit included in the terminal, at least part of a virtual space in which an object in a waiting mode for waiting for reception of a signal from a first device that detects room information is placed;
In response to the reception of the signal, transitioning the object in the waiting mode to an inquiry mode for outputting an inquiry regarding predetermined processing regarding the second device installed in the room;
when at least part of the object and an inquiry about the predetermined process are displayed on the display unit, receiving an input to the inquiry based on the movement of the terminal;
A program for causing a computer to execute the predetermined processing in the second device based on the input in response to the inquiry. setting one of a plurality of operation modes for the object;
2. The program according to claim 1, for causing a computer to execute the steps of: setting a predetermined mode to said object, and executing said predetermined process when said inquiry is output from said object. placing a plurality of the objects in the virtual space;
setting one of a plurality of operation modes corresponding to each of the plurality of objects;
accepting input for the query based on movement of the terminal;
2. The program according to claim 1, for causing a computer to execute said predetermined processing based on the input to said inquiry. 13. The method for causing a computer to determine the motion of the terminal from an image obtained by synthesizing at least part of a virtual space in which the object is arranged and an image captured by an imaging unit. Item 4. The program according to any one of Item 3. 5. The program according to claim 4, for causing a computer to execute that the image captured by said imaging unit is fixed when said predetermined processing is to be performed based on the movement of said terminal. Claims 1 to 5 for causing a computer to execute the predetermined processing based on the movement of the terminal when the amount of change in the parameter indicating the movement of the terminal is within a predetermined range. A program according to any one of clauses 1 to 1. a display control unit that causes a terminal to display at least part of a virtual space in which an object in a waiting mode for receiving a signal from a first device that detects room information is placed ;
an inquiry output unit that, in response to the reception of the signal, transitions the object in the waiting mode to an inquiry mode for outputting an inquiry regarding predetermined processing relating to the second device installed in the room;
when at least part of the object displayed on the terminal and an inquiry about the predetermined process are displayed, an input to the inquiry is received based on the movement of the terminal ; 2. An information processing apparatus comprising : an execution unit that causes a device to execute the predetermined process. 2. The program to be executed by a computer according to claim 1, wherein said room information is information based on a human sensor. 2. The program to be executed by a computer according to claim 1, wherein said movement of said terminal is movement of moving said terminal in a predetermined direction by a predetermined amount or more.

Images