JP7532941B2 - Information processing device, telepresence robot, base control system, remote control system, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, a telepresence robot, a base control system, a remote control system, an information processing method, and a program.

There is known a remote control system that remotely operates a telepresence robot located at a remote site using an administrator terminal located at another site via a communication network. This remote control system allows information about the site where the telepresence robot is located to be confirmed from a remote location by displaying images captured by a camera provided on the telepresence robot on the administrator terminal.

For example, Patent Document 1 discloses a method for communicatively connecting a communication terminal used by a user who is the source of communication with a telepresence robot appropriate for the destination user from among multiple telepresence robots located in a remote location away from the source user.

However, with conventional methods, remote communication between locations was initiated in response to a request from an administrator operating a telepresence robot, and it was not possible for a user at the location where the telepresence robot was installed to call the administrator. As a result, there was an issue that there was room for improvement in order to improve the convenience of remote communication using a telepresence robot.

In order to solve the above-mentioned problem, the invention according to claim 1 is an information processing device that controls the operation of a telepresence robot installed in a specified base, and includes a notification information receiving means that receives notification information including position information of an operation means from an operation means installed in the base that accepts operations from a user, an output means that outputs a request to a specific telepresence robot determined based on the position information included in the received notification information and the position information of the telepresence robot to start operation to a specific destination, and a communication request sending means that sends a communication request to the specific telepresence robot to an administrator terminal that remotely communicates with the telepresence robot, and is an information processing device that starts the movement of the specific telepresence robot to the specific destination and starts remote communication between the administrator terminal and the specific telepresence robot.

The present invention has the effect of improving the convenience of remote communication using a telepresence robot.

FIG. 1 is a diagram illustrating an example of a system configuration of a remote control system according to an embodiment. FIG. 1 is a diagram for explaining an example of an outline of a remote control system according to an embodiment. FIG. 1 is a diagram illustrating an example of a schematic configuration of a robot according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a robot according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of an administrator terminal according to the embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing server according to the embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of a remote control system according to an embodiment. 1A is a conceptual diagram showing an example of a user information management table according to an embodiment, and FIG. 1B is a conceptual diagram showing an example of a notification pattern management table according to an embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of a remote control system according to an embodiment. FIG. 1A is a conceptual diagram showing an example of a base information management table according to an embodiment, FIG. 1B is a conceptual diagram showing an example of a location information management table according to an embodiment, and FIG. 1C is a conceptual diagram showing an example of a condition information management table according to an embodiment. (A) is a conceptual diagram showing an example of a status information management table for an embodiment, (B) is a conceptual diagram showing an example of an administrator information management table for an embodiment, and (C) is a conceptual diagram showing an example of a notification history management table for an embodiment. 1A is a conceptual diagram showing an example of an authentication management table according to an embodiment, and FIG. 1B is a conceptual diagram showing an example of a terminal management table according to an embodiment. 1A is a conceptual diagram showing an example of a destination list management table according to an embodiment, and FIG. 1B is a conceptual diagram showing an example of a session management table according to an embodiment. 11 is a sequence diagram showing an example of a call process from a base system in the remote control system according to the embodiment. FIG. 13 is a flowchart illustrating an example of a robot search process in the information processing server according to the embodiment. 11 is a flowchart illustrating an example of an operation start process in the robot according to the embodiment. 11 is a sequence diagram showing an example of a preparatory process for starting data transmission and reception between a robot and an administrator terminal according to an embodiment. FIG. FIG. 13 is a diagram illustrating an example of a destination list screen displayed on the administrator terminal according to the embodiment. 11 is a sequence diagram illustrating an example of a process from selection of a destination candidate to start of data transmission and reception according to the embodiment. FIG. 13 is a flowchart showing another example of the operation start process in the robot according to the embodiment. 13 is a flowchart illustrating an example of a communication state notification process in the robot according to the embodiment. 13A and 13B are diagrams for explaining an outline of notification of a communication status of a robot according to an embodiment. 10 is a diagram for explaining an outline of notification of a communication state of a robot according to an embodiment. FIG. 13 is a flowchart illustrating an example of a cancellation process of a communication request in the information processing server according to the embodiment. 11 is a sequence diagram showing an example of a cancellation process of a communication request in the remote control system according to the embodiment. FIG. 13 is a sequence diagram showing an example of a process for terminating remote communication between the robot and the manager terminal according to the embodiment. FIG. FIG. 11 is a diagram illustrating an example of notification history information according to the embodiment. 13 is a flowchart illustrating an example of a process at the end of remote communication in the robot according to the embodiment. FIG. 11 is a diagram illustrating an example of a functional configuration of a remote control system according to a modified example of the embodiment. FIG. 11 is a sequence diagram illustrating an example of a call process from a base system in a remote control system according to a modified example of an embodiment.

Below, we will explain the mode for implementing the invention with reference to the drawings. Note that in the explanation of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted.

●Overview●
System Configuration Fig. 1 is a diagram showing an example of the system configuration of a remote control system according to an embodiment. The remote control system shown in Fig. 1 is a system that can manage or maintain devices at a base, or check the position or movement of people at the base, by performing remote communication between a robot 10 located at each base and an administrator terminal used by an administrator at a remote location.

The remote control system 1a is composed of telepresence robots 10 (10A, 10B, 10C, hereinafter referred to as robots 10A, 10B, 10C. When there is no need to distinguish between them, they will be referred to as robot 10), an administrator terminal 50, an information processing server 70, and a communication management server 90 located at each of multiple bases (base A, base B, base C), respectively. The robots 10, the administrator terminal 50, the information processing server 70, and the communication management server 90 are communicatively connected via a communication network 9. The communication network 9 is constructed, for example, by a LAN (Local Area Network), a dedicated line, the Internet, a mobile communication network, or the like. Note that the communication network 9 may be not only wired, but may also include locations where wireless communication is performed, such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), LTE (Long Term Evolution), Wi-Fi (Wireless Fidelity (registered trademark)), or WiMAX (Worldwide Interoperability for Microwave Access). Additionally, the communication network 9 may include a network constructed using a blockchain.

The robot 10 is installed at each base (base A, base B, base C) and is a mobile object that travels autonomously within each base. The robot 10 travels within the base while capturing images of the subject around the robot 10 with the imaging device 12, and transmits the captured images captured by the imaging device 12 to the administrator terminal 50, thereby providing information (images) within the base to the administrator using the administrator terminal 50. As shown in FIG. 2, which will be described later, each base is provided with multiple robots 10 and a notification button 20 for a user located within the base to call an administrator located in a remote location. The robots 10 and notification buttons 20 installed at each base constitute a base system 2 (2A, 2B, 2C; hereinafter, when there is no need to distinguish between them, they will be referred to as the base system 2). The robot 10 may be any mobile object that can travel within the base using technology such as autonomous travel or line tracing.

The administrator terminal 50 is a terminal device such as a PC (Personal Computer) that uses the robot 10 installed at each base (base A, base B, base C) to remotely manage each base. The administrator terminal 50 displays the captured images sent from the robot 10. While viewing the images displayed on the administrator terminal 50, the administrator can remotely communicate with users in the base where the robot 10 is installed. In addition, while viewing the images displayed on the administrator terminal 50, the administrator can remotely operate the robot 10.

The administrator terminal 50 may be any device equipped with a display means for displaying images transmitted from the robot 10, and may be, for example, a tablet terminal, a mobile phone, a smartphone, a wearable terminal such as a head mounted display (HMD), a communication terminal equipped with a wide-angle screen (cylindrical, spherical, hemispherical screen, etc.), a PDA (Personal Digital Assistant), etc.

The information processing server 70 is a server computer for controlling the operation of the robot 10 located at each base. The communication management server 90 is a server computer for managing communication between the robot 10 located at each base and the administrator terminal 50. The information processing server 70 and the communication management server 90 are connected to the robot 10 and the administrator terminal 50 via a communication network 9. The information processing server 70 may be constructed by a single computer, or may be constructed by multiple computers in which each section (function, means, or memory section) is divided and assigned arbitrarily. The same applies to the communication management server 90. The information processing server 70 is an example of an information processing device.

Here, the information processing server 70 and the communication management server 90 constitute the server system 7. This server system 7 may be constituted by a single computer that includes each part (function or means) of the information processing server 70 and the communication management server 90. In addition, the base system 2 and the server system 7 constitute the base control system 3.

The bases where the robot 10 is installed include, for example, offices, schools, warehouses, factories, construction sites, etc. An administrator who remotely manages each base using the administrator terminal 50 can check the positions and movement of people within the base, and manage and maintain devices installed within the base, by checking images of the base transmitted from the robot 10. In addition, the robot 10 and the administrator terminal 50 can also carry out two-way communication (remote conferencing) by sending and receiving images taken by both.

The administrator terminal 50 may be configured to communicate with each of the robots 10 located at multiple bases, or may be configured to communicate with only the robots 10 located at one base.

Now, with reference to FIG. 2, a usage scenario of the remote control system 1a according to this embodiment will be described in outline. FIG. 2 is a diagram for explaining an example of a schematic remote control system according to an embodiment. FIG. 2 shows an example of a case where remote communication is performed between a telepresence robot (robot 10A) installed in base A and an administrator terminal 50 used by an administrator located in a remote location.

Base A shown in FIG. 2 is, for example, a factory or warehouse where predetermined work is performed by multiple workers (workers A, B, C, and D). As shown in FIG. 2, each of workers A, B, C, and D performs work on a workbench. Base A also has multiple robots 10A (robots 10A-1 and 10A-2) that travel autonomously within base A installed. A manager in a remote location uses a manager terminal 50 to remotely communicate with the robot 10A installed at base A to perform maintenance management of base A, etc.

In addition, multiple notification buttons 20A (20A-1 to 20A-5) are installed within base A. Of these, notification buttons 20A-1 to 20A-3 are installed on the workbench, and notification buttons 20A-4 and 20A-5 are mounted on robots 10A-1 and 10A-2, respectively. Notification button 20A is an operation means for a worker at base A to call a manager at a remote location. Note that the configuration of the operation means is not limited to notification button 20, and it may be any device that has a function for calling a manager. For example, if an abnormality occurs during work and a worker wants to communicate with the manager, the worker can call the manager at a remote location by pressing notification button 20A installed nearby.

The selection of the notification button 20A triggers the manager in a remote location to receive a notification indicating a call from the user. The manager can then communicate with the worker by starting remote communication between the manager terminal 50 and the robot 10A installed in base A.

Furthermore, the robots 10A-1 and 10A-2, which communicate remotely with the manager terminal 50, move to a destination that is set according to the installation position of the notification button 20A pressed by the worker, and communicate remotely with the manager terminal 50. Therefore, the worker who pressed the notification button 20A can use the robot 10A to communicate remotely with the manager at the location where he or she wishes to communicate.

When considering the use of a telepresence robot at an on-site location such as base A shown in Figure 2, conventionally, remote communication could only be initiated unilaterally by a remote manager, and there was no means for on-site workers to call the manager when they wanted to communicate with him, resulting in problems such as complaints and grumbling from the site. Therefore, the remote control system 1a provides an operating means for calling the manager at the base where the robot 10 is installed, and can start remote communication between bases using a call from the site as a trigger. In this way, the remote control system 1a can improve the convenience of remote communication using a telepresence robot.

●Configuration of the Robot Next, a specific configuration of the robot 10 will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of a schematic configuration of the robot according to the embodiment. FIG. 3 shows the state in which the robot 10 is located at a charging station 150 for charging the battery of the robot 10. The charging station 150 is installed in each base, and is used to charge the battery of the robot 10 and measure the operating state of the robot 10. The robot 10 stores the position of the charging station 150 installed in the base in advance, and periodically moves to the charging station 150 by instructions from a user in the base, remote operation from an administrator, or by itself judging the state of the remaining battery power. Then, the robot 10 charges the battery at the charging station 150 during a period in which remote communication with the administrator terminal 50 is not performed.

The robot 10 shown in FIG. 3 includes a housing 11 equipped with a control device 30 that controls the processing or operation of the robot 10, a camera 12, a support member 13, a display 14, a movement mechanism 15 (15a, 15b) for moving the robot 10, a movable arm 16 for making the robot 10 perform a specified task (operation), a lamp 17 and a notification panel 18 that notify the communication status of the robot 10, and a notification button 20.

Of these, the notification button 20 is an operating means for a user located at a base to call a remote administrator. When a user at the base presses the notification button 20 installed at the base, the notification button 20 transmits notification information for calling the administrator to the information processing server 70. The position of the notification button 20 provided on the robot 10 is not limited to this, and it is preferable that the notification button 20 is installed at a position where the user can easily press it. The notification button 20 may be detachably installed on the robot 10. In this case, the worker can install the notification button 20 at any location by carrying the notification button 20. The notification buttons 20 (e.g., notification buttons 20A-1, 20A-2, 20A-3) that are not installed on the robot 10 as shown in FIG. 2 also have a similar configuration.

The housing 11 is located in the torso of the robot 10 and contains a power supply unit that supplies the necessary power to the entire robot 10, a control device 30 that controls the processing or operation of the robot 10, and other components.

The photographing device 12 photographs subjects such as people, objects, and scenery located at the base where the robot 10 is installed, and acquires photographed images. The photographing device 12 is a digital camera (general photographing device) capable of acquiring planar images (detailed images) such as a digital single-lens reflex camera or compact digital camera. Photographed image data relating to the photographed images acquired by the photographing device 12 is transmitted to the administrator terminal 50 via a communication session established by the communication management server 90, which will be described later. The photographed images acquired by the photographing device 12 may be video or still images, or may be both video and still images. The photographed images acquired by the photographing device 12 may also include audio data along with the image data.

The display 14 is a display means for displaying various screens. For example, the display 14 displays an image of the location of a remote administrator who is called when the notification button 20 is pressed.

The support member 13 is a member for installing (fixing) the image capture device 12 to the robot 10 (housing 11). The support member 13 may be a pole or the like fixed to the housing 11, or may be a base fixed to the housing 11. The support member 13 may also be a movable member that can adjust the image capture direction (orientation) and position (height) of the image capture device 12.

The moving mechanism 15 is a unit that moves the robot 10, and is composed of wheels, a running motor, a running encoder, a steering motor, a steering encoder, etc. A detailed description of the movement control of the robot 10 is omitted because it is an existing technology. However, the robot 10 receives, for example, a request to start an operation from the information processing server 70 or a driving instruction from the administrator (administrator terminal 50) who is the operator, and the moving mechanism 15 moves the robot 10 based on the received request to start an operation or a driving instruction. The moving mechanism 15 may be a bipedal foot type or a single wheel type. In addition, the shape of the robot 10 is not limited to the vehicle type shown in FIG. 3, and may be, for example, a bipedal human type, a form that imitates a living thing, or a form that imitates a specific character.

The movable arm 16 has an operating means that enables the robot 10 to perform additional operations other than the movement. As shown in FIG. 3, the movable arm 16 is provided with, for example, a hand at the tip of the movable arm 16 as an operating means for grasping an object such as a part. The robot 10 can perform a predetermined task (operation) by rotating or deforming the movable arm 16.

The lamp 17 and the notification panel 18 notify users at the base of the communication status of the robot 10 by changing the lamp lighting pattern and panel display depending on the status of remote communication with the administrator. The lamp 17 and the notification panel 18 are an example of a communication status notification means.

The imaging device 12 may be a wide-angle imaging device capable of acquiring a spherical (360°) panoramic image. The wide-angle imaging device is, for example, a spherical imaging device that captures an object to obtain two hemispherical images that are the basis of a spherical (panoramic) image. The wide-angle imaging device may also be, for example, a wide-angle camera or a stereo camera that can acquire a wide-angle image having an angle of view equal to or greater than a predetermined value. In other words, the wide-angle imaging device is imaging means that can acquire an image (spherical image, wide-angle image) captured using a lens with a focal length shorter than a predetermined value.

The robot 10 may also be configured to include multiple imaging devices 12. In this case, the robot 10 may be configured to include, as the imaging devices 12, both a wide-angle imaging device and a general imaging device that can capture a part of a subject captured by the wide-angle imaging device to obtain a detailed image (planar image).

In addition to the above configuration, the robot 10 may also have various sensors capable of detecting information about the surroundings of the robot 10. The various sensors are, for example, sensor devices such as a barometer, a thermometer, a photometer, a human presence sensor, or a light meter.

●Hardware configuration●
Next, the hardware configuration of each device or terminal constituting the remote control system 1a will be described with reference to Figures 4 to 6. Note that components may be added or deleted from the hardware configuration shown in Figures 4 to 6 as necessary.

●Hardware configuration of the robot Fig. 4 is a diagram showing an example of the hardware configuration of the robot according to the embodiment. The robot 10 includes a control device 30 that controls the processing or operation of the robot 10. As described above, the control device 30 is provided inside the housing 11 of the robot 10. Note that the control device 30 may be provided outside the housing 11 of the robot 10, or may be provided as a device separate from the robot 10.

The control device 30 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, a HDD (Hard Disk Drive) 304, a media interface (I/F) 305, an input/output I/F 306, an audio input/output I/F 307, a network I/F 308, a short-range communication circuit 309, an antenna 309a of the short-range communication circuit 309, and an external device connection I/F 311 and a bus line 310.

The CPU 301 controls the entire robot 10. The CPU 301 is a calculation device that realizes each function of the robot 10 by reading programs or data stored in the ROM 302 or HD (Hard Disk) 304a, etc., onto the RAM 303 and executing the processing.

The ROM 302 is a non-volatile memory that can retain programs or data even when the power is turned off. The RAM 303 is a volatile memory used as a work area for the CPU 301. The HDD 304 controls the reading and writing of various data from the HD 304a under the control of the CPU 301. The HD 304a stores various data such as programs. The media I/F 305 controls the reading and writing (storage) of data from a recording medium 305a such as a USB (Universal Serial Bus) memory, a memory card, an optical disk, or a flash memory.

The input/output I/F 306 is an interface for inputting and outputting characters, numbers, various instructions, etc., between various external devices. The input/output I/F 306 controls the display of various information such as a cursor, menu, window, characters, or images on a display 14 such as an LCD (Liquid Crystal Display). The display 14 may be a touch panel display equipped with an input means. In addition to the display 14, the input/output I/F 306 may be connected to an input means such as a mouse or keyboard. The sound input/output I/F 307 is a circuit that processes the input and output of sound signals between the microphone 307a and the speaker 307b under the control of the CPU 301. The microphone 307a is a type of built-in sound collection means that inputs sound signals under the control of the CPU 301. The speaker 307b is a type of playback means that outputs sound signals under the control of the CPU 301.

The network I/F 308 is a communication interface that communicates (connects) with other devices or apparatuses via the communication network 9. The network I/F 308 is, for example, a communication interface such as a wired or wireless LAN. The network I/F 308 may also include a communication interface such as 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), Wi-Fi, WiMAX (Worldwide Interoperability for Microwave Access), Zigbee (registered trademark), or millimeter wave wireless communication. The short-range communication circuit 309 is a communication circuit such as NFC (Near Field communication) or Bluetooth (registered trademark). The external device connection I/F 311 is an interface for connecting other devices to the control device 30.

Bus line 310 is an address bus, data bus, etc. for electrically connecting the above components, and transmits address signals, data signals, various control signals, etc. CPU 301, ROM 302, RAM 303, HDD 304, media I/F 305, input/output I/F 306, sound input/output I/F 307, network I/F 308, short-range communication circuit 309, and external device connection I/F 311 are connected to each other via bus line 310.

Furthermore, the control device 30 is connected to the drive motor 101, the actuator 102, the acceleration/direction sensor 103, the GPS (Global Positioning System) receiver 104, the photographing device 12, the power supply unit 105, the notification button 20, the lamp 17, and the notification panel 18 via the external device connection I/F 311.

The drive motor 101 rotates the movement mechanism 15 based on commands from the CPU 301 to move the robot 10 along the ground. The actuator 102 deforms the movable arm 16 based on commands from the CPU 301. The acceleration/direction sensor 103 is a sensor such as an electronic magnetic compass that detects geomagnetism, a gyrocompass, and an acceleration sensor. The GPS receiver 104 receives GPS signals from GPS satellites. The power supply unit 105 is a unit that supplies the necessary power to the entire robot 10.

5 is a diagram showing an example of the hardware configuration of an administrator terminal according to an embodiment. The administrator terminal 50 includes a CPU 501, a ROM 502, a RAM 503, an EEPROM (Electrically Erasable Programmable Read-Only Memory) 504, an image sensor I/F 505, a CMOS (Complementary Metal Oxide Semiconductor) sensor 505a, and a media I/F 506.

The CPU 501 controls the overall operation of the administrator terminal 50. The CPU 501 is a computing device that realizes each function of the administrator terminal 50 by reading programs or data stored in the ROM 502 or the like onto the RAM 503 and executing processing.

ROM 502 stores programs used to drive CPU 501, such as IPL (Initial Program Loader). RAM 503 is used as a work area for CPU 501. EEPROM 504 reads and writes various data, such as programs for the administrator terminal, under the control of CPU 501.

The CMOS sensor 505a captures an image of a subject (mainly a self-portrait) under the control of the CPU 501 to obtain image data. The image sensor I/F 505 is a circuit that controls the driving of the CMOS sensor 505a. The media I/F 506 controls the reading and writing (storing) of data to a recording medium 506a such as a flash memory.

The administrator terminal 50 also includes a network I/F 507, an audio input/output I/F 508, a microphone 508a, a speaker 508b, a display 511, a keyboard 512, a mouse 513, an external device connection I/F 514, a short-range communication circuit 515, and an antenna 515a for the short-range communication circuit 515.

The network I/F 507 is a communication interface that communicates (connects) with other devices or apparatuses via the communication network 9. The network I/F 507 is, for example, a communication interface such as a wired or wireless LAN. The network I/F 507 may also include a communication interface such as 3G, LTE, 4G, 5G, Wi-Fi, WiMAX, Zigbee, or millimeter wave wireless communication. The sound input/output I/F 508 is a circuit that processes the input and output of sound signals between the microphone 508a and the speaker 508b under the control of the CPU 501. The microphone 508a is a type of built-in sound collection means that inputs sound signals under the control of the CPU 501. The speaker 508b is a type of playback means that outputs sound signals under the control of the CPU 501.

The display 511 is a type of display means such as liquid crystal or organic electroluminescence (EL) that displays an image of a subject, various icons, etc. The display 511 may be a touch panel display equipped with an input means. The keyboard 512 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The mouse 513 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The external device connection I/F 514 is an interface for connecting various external devices. The short-range communication circuit 515 is a communication circuit such as NFC or Bluetooth.

The administrator terminal 50 also includes a bus line 509. The bus line 509 is an address bus, a data bus, etc., for electrically connecting each component such as the CPU 501 shown in FIG. 5.

●Hardware configuration of server Fig. 6 is a diagram showing an example of the hardware configuration of an information processing server according to an embodiment. The information processing server 70 is constructed using a general computer. The information processing server 70 includes a CPU 701, a ROM 702, a RAM 703, a HD 704, a HDD 705, a media I/F 707, a network I/F 708, a display 711, a keyboard 712, a mouse 713, a DVD-RW (Digital Versatile Disk Rewritable) drive 715, a timer 716, and a bus line 710.

The CPU 701 controls the overall operation of the information processing server 70. The ROM 702 stores programs used to drive the CPU 701. The RAM 703 is used as a work area for the CPU 701. The HDD 705 controls the reading or writing of various data from the HD 704 under the control of the CPU 701. The HD 704 stores various data such as programs. The media I/F 707 controls the reading or writing (storage) of data from a recording medium 706 such as a flash memory.

The network I/F 708 is an interface for data communication using the communication network 9. The display 711 displays various information such as a cursor, a menu, a window, characters, or an image. The keyboard 712 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The mouse 713 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. The DVD-RW drive 715 controls the reading of various data from the DVD-RW 714 as an example of a removable recording medium. The DVD-RW 714 may be a DVD-R or the like. The DVD-RW drive 715 may also be a Blu-ray drive that controls the reading or writing (storing) of various data from a BD-RE (Blu-ray (registered trademark) Disc Rewritable), or a CD-RW drive that controls the reading or writing of various data from a CD-RW (Compact Disc-ReWritable). The timer 716 is a measurement device having a time measurement function. The timer 716 may be a computer-based software timer.

Furthermore, the information processing server 70 includes a bus line 710. The bus line 710 is an address bus, a data bus, etc., for electrically connecting each component such as the CPU 701 shown in FIG. 6.

The communication management server 90 is constructed using a general computer, and as shown in FIG. 6, includes a CPU 901, ROM 902, RAM 903, HD 904, HDD 905, media I/F 907, network I/F 908, display 911, keyboard 912, mouse 913, DVD-RW drive 715, and bus line 910. These are configured similarly to the CPU 701, ROM 702, RAM 703, HDD 705, media I/F 707, network I/F 708, display 711, keyboard 712, mouse 713, DVD-RW drive 715, and bus line 710 in the information processing server 70, and therefore will not be described here.

The above programs may be distributed by recording them on a computer-readable recording medium in the form of an installable or executable file. Examples of recording media include CD-Rs (Compact Disc Recordables), DVDs, Blu-ray discs, and SD cards. The recording media may be provided domestically or internationally as a program product. For example, the information processing server 70 realizes the information processing method according to the present invention by executing the program according to the present invention.

●Function configuration●
Next, the functional configuration of the remote control system 1a according to the present embodiment will be described with reference to Fig. 7 to Fig. 13. Fig. 7 and Fig. 9 are diagrams showing an example of the functional configuration of the remote control system according to the embodiment. Fig. 7 and Fig. 9 show terminals, devices, and servers shown in Fig. 1 that are related to the processes or operations described below.

●Functional configuration of the control device First, the functional configuration of the control device 30 that controls the processing or operation of the robot 10 will be described with reference to Fig. 7. The control device 30 has a transmitting/receiving unit 31, a receiving unit 32, a display control unit 33, a determining unit 34, a communication control unit 35, a status information acquiring unit 36, a position information acquiring unit 37, a photographing instruction unit 38, a photographed image acquiring unit 41, a user identification unit 42, a movement destination setting unit 43, a movement control unit 44, an arm operation control unit 45, a communication status notification control unit 49, and a storage/reading unit 39. Each of these units is a function or a means that is realized by any of the components shown in Fig. 4 operating according to an instruction from the CPU 301 according to the program deployed on the RAM 303. The control device 30 also has a storage unit 3000 constructed by the ROM 302, HD 304a, or recording medium 305a shown in Fig. 4. The storage unit 3000 is an example of a storage means.

The transmission/reception unit 31 is realized by commands from the CPU 301 shown in FIG. 4 and the network I/F 308, and has a function of transmitting and receiving various data or information to and from other devices via the communication network 9. For example, the transmission/reception unit 31 receives, via the communication network 9, operation start request information from the information processing server 70, which requests the robot 10 to start an operation. The transmission/reception unit 31 also transmits the captured image data acquired by the captured image acquisition unit 41 to the administrator terminal 50. The transmission/reception unit 31 is an example of an acquisition means.

The reception unit 32 is realized by commands from the CPU 301 shown in FIG. 4 and the input/output I/F 306, and has a function of receiving operation input to an input means such as the display 14. The display control unit 33 is realized by commands from the CPU 301 shown in FIG. 4 and the input/output I/F 306, and has a function of displaying various screens on the display 14. The judgment unit 34 is realized by commands from the CPU 301 shown in FIG. 4, and has a function of making various judgments.

The communication control unit 35 is realized by an instruction from the CPU 301 shown in FIG. 4, and is a function that controls remote communication with the administrator terminal 50. The communication control unit 35 is an example of a communication control means. The status information acquisition unit 36 is realized by an instruction from the CPU 301 shown in FIG. 4 and the external device connection I/F 311, and is a function that acquires status information indicating the status of the robot 10. The status information acquisition unit 36 generates and acquires, for example, the connection status of the external device to the robot 10, the remaining battery level of the robot 10 acquired from the power supply unit 105, the driving status of the moving mechanism 15 acquired from the movement control unit 44, etc. as status information of the robot 10. The position information acquisition unit 37 is realized by an instruction from the CPU 301 shown in FIG. 4 and the external device connection I/F 311, and is a function that acquires detection results such as the direction of each direction (azimuth angle, magnetic north) detected by the acceleration/direction sensor 103 or the GPS receiving unit 104. The detection results for each direction, etc., are position information that indicates the position and orientation of the robot 10 at a given point in time.

The photographing instruction unit 38 is realized by commands from the CPU 301 shown in FIG. 4 and the external device connection I/F 311, and is a function that instructs the photographing device 12 to perform photographing processing. The photographing instruction unit 38 transmits, for example, instruction information to the photographing device 12 to instruct the photographing device 12 to perform photographing.

The captured image acquisition unit 41 is realized by commands from the CPU 301 shown in FIG. 4 and the external device connection I/F 311, and is a function for acquiring captured images acquired by the image capture device 12. The captured image acquisition unit 41 acquires, for example, captured image data from the image capture device 12, which is an image captured by the image capture device 12 by capturing an image of a subject.

The user identification unit 42 is realized by a command from the CPU 301 shown in FIG. 4, and is a function for identifying a user who has pressed the notification button 20 installed in the base. The user identification unit 42 identifies the user who has pressed the notification button 20, for example, by recognizing a facial image shown in a captured image acquired by the captured image acquisition unit 41.

The destination setting unit 43 is realized by a command from the CPU 301 shown in FIG. 4, and is a function for setting the destination of the robot 10. The destination setting unit 43 sets the destination of the robot 10 based on, for example, operation start request information transmitted from the information processing server 70. The destination setting unit 43 is an example of a setting means.

The movement control unit 44 is realized by commands from the CPU 301 and the external device connection I/F 311 shown in FIG. 4, and is a function that controls the movement of the robot 10 by driving the movement mechanism 15. The movement control unit 44 moves the robot 10 by controlling the driving of the movement mechanism 15 in response to operation start request information transmitted from the information processing server 70, for example. The movement control unit 44 is an example of a movement control means.

The arm operation control unit 45 is realized by commands from the CPU 301 shown in FIG. 4 and the external device connection I/F 311, and is a function that controls the operation of the movable arm 16. The arm operation control unit 45 changes the orientation or position of the movable arm 16 by deforming the movable arm 16, for example, based on an operation request transmitted from the administrator terminal 50. Changes the orientation or position of the movable arm 16.

The communication status notification control unit 49 is realized by commands from the CPU 301 shown in FIG. 4 and the external device connection I/F 311, and is a function that controls the notification of the communication status by the lamp 17 and the notification panel 18.

The storage/readout unit 39 is executed by commands from the CPU 301 shown in FIG. 4, and has the function of storing various data in the storage unit 3000 or reading various data from the storage unit 3000. The image data and sound data received when communicating with the administrator terminal 50 are overwritten and stored in the storage unit 3000 each time they are received. Of these, the image data before being overwritten displays an image on the display 14, and the sound data before being overwritten outputs sound from the speaker 307b. The storage unit 3000 also stores the captured image data acquired by the captured image acquisition unit 41. The captured image data stored in the storage unit 3000 may be configured to be deleted when a predetermined time has passed since it was acquired by the captured image acquisition unit 41, or the data transmitted to the administrator terminal 50 may be deleted.

User Information Management Table FIG. 8A is a conceptual diagram showing an example of a user information management table according to the embodiment. A user information management DB 3001 configured by a user information management table as shown in FIG. 8A is constructed in the storage unit 3000. In this user information management table, a user ID and a user name for identifying a user located at a base where the robot 10 is installed, and a feature amount are associated and managed. Among them, the feature amount is a feature amount for identifying a face of a person included in the photographed image data acquired by the photographed image acquisition unit 41. The user information management table is used when a process of matching a face of a person included in the photographed image data to be processed (face matching process) is executed as image processing in the user identification unit 42.

Notification Pattern Management Table FIG. 8B is a conceptual diagram showing an example of a notification pattern management table according to an embodiment. In the storage unit 3000, a notification pattern management DB 3009 configured by a notification pattern management table as shown in FIG. 8B is constructed. In this notification pattern management table, information indicating a communication state with the manager terminal 50 and a notification pattern corresponding to the communication state are associated and managed. Among them, the communication state includes states such as "communicating" indicating that communication with another user or manager is being performed, "waiting for communication" indicating a reserved state for communication with another user or manager, and "waiting" indicating an available state in which processing for starting communication with another user or manager has not been started. In addition, the notification pattern shows a notification method that allows the user to visually recognize the communication state of the robot 10. The notification pattern shows, for example, a different lighting method of the lamp 17 according to the communication state. The lamp 17 lights up in a manner that allows distinction for each notification pattern, for example, by the color to be turned on or the regularity of blinking.

●Functional configuration of the administrator terminal Next, the functional configuration of the administrator terminal 50 will be described with reference to FIG. 7. The administrator terminal 50 has a transmission/reception unit 51, a reception unit 52, a display control unit 53, a judgment unit 54, and a storage/readout unit 59. Each of these units is a function or a means that is realized when any of the components shown in FIG. 5 operates according to an instruction from the CPU 501 according to a program deployed on the RAM 503. The administrator terminal 50 also has a storage unit 5000 constructed by the ROM 502 or the recording medium 506a shown in FIG. 5. The storage unit 5000 is an example of a storage means. Furthermore, the administrator terminal 50 has a dedicated application program for remotely operating the robot 10 installed therein. The administrator terminal 50 realizes each function, for example, when the CPU 501 executes the installed application program.

The transmission/reception unit 51 is realized by commands from the CPU 501 shown in FIG. 5 and the network I/F 507, and has the function of transmitting and receiving various data or information with other devices via the communication network 9. The transmission/reception unit 51 receives, for example, captured image data transmitted from the robot 10 (control device 30) via the communication network 9. The transmission/reception unit 51 also receives, for example, status information indicating the status of the robot 10 from the robot 10 (control device 30) via the communication network 9. The transmission/reception unit 51 is an example of a communication request receiving means. The transmission/reception unit 51 is also an example of a communication means.

The reception unit 52 is realized by commands from the CPU 501 shown in FIG. 5 and input means such as the keyboard 512 or mouse 513, and has a function of receiving various selections or operation inputs to the administrator terminal 50. The display control unit 53 is realized by commands from the CPU 501 shown in FIG. 5, and has a function of displaying various screens on the display 511 of the administrator terminal 50. The judgment unit 54 is realized by commands from the CPU 501 shown in FIG. 5, and has a function of making various judgments.

The storage/reading unit 59 is executed by commands from the CPU 501 shown in FIG. 5, and has the function of storing various data in the storage unit 5000 or reading various data from the storage unit 5000. Image data and sound data received during communication with the robot 10 (control device 30) are overwritten and stored in the storage unit 5000 each time they are received. Of these, an image is displayed on the display 511 using the image data before being overwritten, and sound is output from the speaker 508b using the sound data before being overwritten.

Functional configuration of the information processing server Next, the functional configuration of the information processing server 70 will be described with reference to Fig. 9. The information processing server 70 has a transmission/reception unit 71, a judgment unit 72, a search unit 73, a generation unit 74, a measurement unit 75, and a storage/readout unit 79. Each of these units is a function or a means that is realized when any of the components shown in Fig. 6 operates according to an instruction from the CPU 701 in accordance with a program deployed on the RAM 703. The information processing server 70 also has a storage unit 7000 that is constructed by the ROM 702, HD 704, or recording medium 706 shown in Fig. 6. The storage unit 7000 is an example of a storage means.

The transmission/reception unit 71 is realized by commands from the CPU 701 shown in FIG. 6 and the network I/F 708, and is a function for transmitting and receiving various data or information to and from other devices via the communication network 9. For example, the transmission/reception unit 71 receives notification information from the notification button 20 via the communication network 9, which is transmitted when the selection of the notification button 20 is accepted. The transmission/reception unit 71 also transmits, for example, operation start request information via the communication network 9 to the robot 10, requesting the start of an operation for moving the robot 10 to a specific destination. Furthermore, the transmission/reception unit 71 transmits a communication request to the administrator terminal 50 for remote communication with the robot 10. The transmission/reception unit 71 is an example of a notification information receiving means. The transmission/reception unit 71 is also an example of an output means. Furthermore, the transmission/reception unit 71 is also an example of a communication request transmitting means.

The judgment unit 72 is realized by an instruction from the CPU 701 shown in FIG. 6, and is a function for making various judgments. The search unit 73 is realized by an instruction from the CPU 701 shown in FIG. 6, and is a function for searching for the robot 10 requesting an action based on notification information transmitted from the notification button 20. The search unit 73 is an example of a decision means. The generation unit 74 is realized by an instruction from the CPU 701 shown in FIG. 6, and is a function for generating various data or information. The generation unit 74 generates, for example, communication history information indicating the communication history between the robot 10 and the administrator terminal 50. The measurement unit 75 is realized by an instruction from the CPU 701 shown in FIG. 6 and a timer 716, and is a function for measuring the processing time until a communication session between the robot 10 and the administrator terminal 50 is established.

The memory/read unit 79 is executed by instructions from the CPU 701 shown in FIG. 6, and has the function of storing various data in the memory unit 7000 or reading various data from the memory unit 7000.

Base Information Management Table Fig. 10A is a conceptual diagram showing an example of a base information management table according to the embodiment. A base information management DB 7001 configured by a base information management table as shown in Fig. 10A is constructed in the storage unit 7000. In this base information management table, a base ID and a base name for identifying a base where a robot 10 to be controlled by the information processing server 70 is installed, and a button ID for identifying a notification button 20 installed at the base are associated and managed. When the notification button 20 is pressed by a user in the base, the information processing server 70 identifies the base where the robot 10 to be controlled is installed based on the button ID of the notification button 20 that transmitted the notification information.

Position Information Management Table FIG. 10B is a conceptual diagram showing an example of a position information management table according to the embodiment. A position information management DB 7002 configured by a position information management table as shown in FIG. 10B is constructed in the storage unit 7000. In this position information management table, a terminal ID and a terminal name for identifying the robot 10, and position information indicating the position of the robot 10 are associated and managed for each base ID. Among these, the position information includes information of latitude and longitude, and is transmitted from the robot 10 to the information processing server 70 as necessary. Note that the format of the position information is not limited to information of latitude and longitude, and may be any information that can determine the position of the robot 10 within the base.

Condition Information Management Table FIG. 10C is a conceptual diagram showing an example of a condition information management table according to an embodiment. A condition information management DB 7003 configured by a condition information management table as shown in FIG. 10C is constructed in the storage unit 7000. In this condition information management table, a button ID for identifying the notification button 20 and condition information indicating a condition for transmitting a control request to the robot 10 are associated and managed for each base ID. Among them, the condition information includes conditions related to items such as the performance or operating state of the robot 10. Items included in the condition information include, for example, the remaining battery level of the robot 10, the performance of the imaging device 12 (camera), or the performance of the movable arm 16. Note that the items included in the condition information are not limited to these, and may include, for example, items of the usage history such as the number of times and frequency of an operation start request made from the information processing server 70 to the robot 10. In addition, the items included in the condition information are the same as the items included in the state information stored in the state information management DB 7004 described later.

The battery remaining capacity item in the condition information indicates the threshold battery remaining capacity of the robot 10 to which the control request is being made. For example, the battery remaining capacity condition for a control request triggered by the selection of the notification button 20A identified by the button ID "btA01" is 90% or more (≧90%).

In addition, the camera item in the condition information indicates the performance threshold of the imaging device 12 equipped to the robot 10. In the example shown in FIG. 11(A), the imaging range based on the type of imaging device 12 is set as the performance threshold of the imaging device 12. In this case, the imaging devices 12 are arranged in order of increasing performance, from wide-angle camera, movable camera, to normal camera. A movable camera is a camera that has a rotation mechanism and can take images while changing the imaging direction (orientation), and a normal camera is a general digital camera that can obtain planar images (detailed images). For example, the camera condition in the case of a control request triggered by the selection of the notification button 20A identified by the button ID "btA02" is that the robot 10 has performance equal to or greater than that of a normal camera. Also, for example, the camera condition in the case of a control request triggered by the selection of the notification button 20A identified by the button ID "btA03" is that the robot 10 has performance equal to or greater than that of a movable camera. Note that the performance threshold of the imaging device 12 may be a condition that expresses the resolution or imaging range numerically.

Furthermore, the arm item in the condition information indicates the performance threshold of the movable arm 16 equipped on the robot 10. For example, the arm condition in the case of a control request triggered by the selection of the notification button 20A identified by the button ID "btA01" is that the robot 10 is equipped with a robot hand.

In this way, different condition information can be set in the condition information management table for each base and each notification button 20, so that the conditions for making a control request to the robot 10 can be set according to the type of base or the installation position of the notification button 20. Note that the information processing server 70 may be configured to be able to add or modify the contents of the condition information shown in the condition information management table as appropriate.

Status Information Management Table FIG. 11A is a conceptual diagram showing an example of a status information management table according to an embodiment. A status information management DB 7004 configured by a status information management table as shown in FIG. 11A is constructed in the storage unit 7000. In this status information management table, a terminal ID and a terminal name for identifying the robot 10, and status information indicating the status of the robot 10 are associated and managed for each base ID. Among them, the status information includes data on items related to the performance and current operating status of the robot 10. The items included in the status information are the same as the items included in the condition information shown in the condition information management table of FIG. 10C. The status information of the robot 10 is transmitted to the information processing server 70 from each robot 10 installed at each base as needed or periodically. The information processing server 70 determines whether to make a control request to the robot 10 based on the status information shown in the status information management table and the condition information shown in the condition information management table.

Administrator Information Management Table FIG. 11B is a conceptual diagram showing an example of an administrator information management table according to an embodiment. An administrator information management DB 7005 configured by an administrator information management table as shown in FIG. 11B is constructed in the storage unit 7000. In this administrator information management table, an administrator ID and an administrator name for identifying an administrator who manages a base located at a remote location, administrator destination information indicating the destination of the administrator, and a button ID for identifying the notification button 20 are associated and managed. Among them, the administrator destination information is, for example, an email address of the administrator or an application ID assigned to each administrator. For example, when an administrator uses a dedicated administrator terminal 50, the administrator destination information may be an IP address indicating the destination of the administrator terminal 50. In addition, in the administrator information management table, a notification button 20 to be managed is assigned to each administrator, and the information processing server 70 notifies the assigned administrator of a request for communication with the robot 10 in the base according to the notification button 20 selected by the user.

Notification History Management Table FIG. 11C is a conceptual diagram showing an example of a notification history management table according to an embodiment. A notification history management DB 7006 configured by a notification history management table as shown in FIG. 11C is constructed in the storage unit 7000. In this notification history management table, a base ID, a button ID for identifying the notification button 20 selected by the user, an administrator ID for identifying the administrator who performed remote communication with the robot 10 using the administrator terminal 50, a user ID for identifying the user who selected the notification button 20, a communication time indicating the time when remote communication between the administrator terminal 50 and the robot 10 was performed in response to the selection of the notification button 20, and a link to a storage destination of notification history information indicating a history of remote communication between the administrator terminal 50 and the robot 10 are associated and managed. When the remote communication between the administrator terminal 50 and the robot 10 is terminated, the information processing server 70 generates notification history information and stores the generated notification history information in a storage destination indicated in the notification history management table. Note that the base ID, button ID, administrator ID, user ID, and communication time are examples of notification history data, and a configuration in which some items are not included as notification history data may be used.

Functional configuration of the communication management server Next, the functional configuration of the communication management server 90 will be described with reference to Fig. 9. The communication management server 90 has a transmission/reception unit 91, an authentication unit 92, a judgment unit 93, a creation unit 94, and a storage/readout unit 99. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Fig. 6 operates in response to an instruction from the CPU 901 in accordance with a program deployed on the RAM 903. The communication management server 90 also has a storage unit 9000 constructed from the ROM 902, HD 904, or recording medium 906 shown in Fig. 6.

The transmission/reception unit 91 is realized by commands from the CPU 901 shown in FIG. 6 and the network I/F 908, and has the function of transmitting and receiving various data or information to and from other devices via the communication network 9.

The authentication unit 92 is realized by a command from the CPU 901 shown in FIG. 6, and has a function of authenticating the source of the login request based on the login request information received by the transmission/reception unit 91. The authentication unit 92 searches the authentication management DB 9001 in the memory unit 9000, for example, using the terminal ID and password included in the login request received by the transmission/reception unit 91 as search keys. The authentication unit 92 then performs terminal authentication by determining whether the same set of terminal ID and password is managed in the authentication management DB 9001.

The determination unit 93 is realized by a command from the CPU 901 shown in FIG. 6, and is a function that determines whether the terminal ID of the administrator terminal 50 is managed in a session management table described later. The creation unit 94 is realized by a command from the CPU 901 shown in FIG. 6, and is a function that creates a session ID to be used for communication.

The storage/reading unit 99 is executed by commands from the CPU 901 shown in FIG. 6, and has the function of storing various data in the storage unit 9000 or reading various data from the storage unit 9000. The storage unit 9000 also stores destination list frame data (not including destination list content information such as the icons, "rA01", and "robot 10A-1" shown in FIG. 18) in the destination list screen 900 (see FIG. 18) described below.

Authentication Management Table Fig. 12A is a conceptual diagram showing an example of an authentication management table according to an embodiment. An authentication management DB 9001 configured by an authentication management table as shown in Fig. 12A is constructed in the storage unit 9000. In this authentication management table, each password is associated with each terminal ID of all administrator terminals 50 managed by the communication management server 90. For example, the authentication management table shown in Fig. 12A indicates that the terminal ID of administrator terminal 50A is "o01" and the password is "aaaa".

Terminal Management Table FIG. 12B is a conceptual diagram showing an example of a terminal management table according to the embodiment. A terminal management DB 9002 configured by a terminal management table as shown in FIG. 12B is constructed in the storage unit 9000. In this terminal management table, for each terminal ID of each terminal (robot 10 and manager terminal 50), the terminal name of each terminal, the IP address of the terminal, operation status information indicating the current operation status of each terminal, and the base name indicating the base where the robot 10 is located when the terminal is a robot 10 are associated and managed. For example, in the terminal management table shown in FIG. 12B, it is shown that the manager terminal 50 with the terminal ID "o01" has the terminal name "manager terminal 50A", the IP address of this manager terminal 50 is "1.2.1.3", and the operation status is "online (available for calls)". In addition, it is shown that the robot 10 with the terminal ID "rA01" has a terminal name "robot 10A-1", an IP address of this robot 10 is "1.3.2.3", an operating status is "online (capable of calling)", and a base name is "base A".

Destination List Management Table FIG. 13A is a conceptual diagram showing an example of a destination list management table according to the embodiment. A destination list management DB 9003 configured by a destination list management table as shown in FIG. 13A is constructed in the storage unit 9000. In this destination list management table, the terminal IDs of the destination candidate robots 10 registered as candidates of the destination robot 10 are associated with each terminal ID of the administrator terminal 50 as a starting terminal requesting the start of communication with the robot 10 and managed. For example, in the destination list management table shown in FIG. 13A, it is shown that the destination candidates that can request the start of communication from the starting terminal (administrator terminal 50A) with the terminal ID "o01a" are the robot 10A-1 with the terminal ID "rA01", the robot 10A-2 with the terminal ID "rA02", the robot 10C-1 with the terminal ID "rC01", and the like. The terminal IDs of the robots 10 that are destination candidates are updated by being added or deleted in response to a request for addition or deletion from an arbitrary start terminal (manager terminal 50) to the communication management server 90.

Session Management Table FIG. 13B is a conceptual diagram showing an example of a session management table according to an embodiment. A session management DB 9004 configured by a session management table as shown in FIG. 13B is constructed in the storage unit 9000. In this session management table, for each session ID for identifying a session used when communicating between the robot 10 and the administrator terminal 50, the terminal IDs of the robot 10 and the administrator terminal 50 using the session specified by this session ID are associated and managed. For example, in the session management table shown in FIG. 13B, it is shown that the terminals using the session executed using the session ID "sel" are the administrator terminal 50A with the terminal ID "o01", the robot 10A-2 with the terminal ID "rA02", and the robot 10C-1 with the terminal ID "rC01".

Processing or operation of the embodiment
Next, the processing or operation of the remote control system 1a according to the embodiment will be described with reference to Fig. 14 to Fig. 28. In the following description, the processing executed by the control device 30 included in the robot 10 will be described as the processing executed by the robot 10.

●Call process from a base system First, a process of calling a remote manager from a user (worker) located in a base will be described with reference to Figs. 14 to 25. Fig. 14 is a sequence diagram showing an example of a call process from a base system in a remote control system according to an embodiment. Note that Fig. 14 describes a process when any of the notification buttons 20A installed in base A is pressed by a user, but a call process to a manager by a user at another base (for example, base B or base C) has a similar configuration. Also, Fig. 14 describes an example of a case where remote communication is performed between the robot 10A-1 and the manager terminal 50A as shown in Fig. 2, but a similar process is performed when remote communication is performed between another robot 10 and the manager terminal 50.

The robot 10A-1 installed at base A is in a standby state until it receives a control request from the information processing server 70 (step S11). Specifically, during the period until it receives the control request from the information processing server 70, the robot 10A-1 moves to a charging station 150 as shown in FIG. 3, for example, and charges its battery via the power supply unit 105. During the period until it receives the control request from the information processing server 70, the robot 10A-1 may travel autonomously within base A or may stop at a predetermined position that has been stored in advance. In other words, the robot 10A-1 waits in a state in which it can start the requested operation when it receives a control request.

Next, when a user (worker) in the base A wants to call a manager in a remote location due to, for example, the occurrence of some abnormality during work, he/she presses the notification button 20A installed in the base A. In this case, when the user presses the notification button 20A, the reception unit 22 of the notification button 20A receives the selection of the button (step S12). When the selection of the button is received in step S12, the location information acquisition unit 23 acquires the current location information of the notification button 20A (step S13). Note that the acquisition of the location information by the location information acquisition unit 23 is not limited to the configuration in which it is performed at the timing when the selection of the button is received by the reception unit 22, and may be performed periodically at a predetermined timing. Then, the transmission/reception unit 21 transmits notification information for calling a manager in a remote location to the information processing server 70 (step S14). Here, the notification information includes the button ID of the notification button 20A and the location information acquired in step S13. As a result, the transmission/reception unit 71 of the information processing server 70 receives the notification information transmitted from the notification button 20A.

Next, the information processing server 70 executes a search process for the robot 10 located at base A to be controlled based on the notification information received in step S14 (step S15). Here, a detailed configuration of the search process for the robot 10 will be described with reference to FIG. 15. FIG. 15 is a flowchart showing an example of a robot search process in the information processing server according to the embodiment.

First, the storage and reading unit 79 searches the location information management DB 7001 (see FIG. 10(A)) using the location information included in the notification information received by the transmission and reception unit 71 as a search key to read out the corresponding location ID (step S151). In this case, the location ID read out by the storage and reading unit 79 is "locatinA", which is the location ID of location A. The storage and reading unit 79 also searches the condition information management DB 7003 (see FIG. 10(C)) using the location ID read out in step S151 as a search key to read out the corresponding condition information (step S152).

Next, the search unit 73 searches the location information management DB 7002 (see FIG. 10B) using the location information included in the notification information received by the transmission/reception unit 71 as a search key to identify the robot 10 located closest to the selected notification button 20A (step S153). Specifically, the search unit 73 extracts the terminal ID associated with the location information indicating the location closest to the location information included in the notification information from among the robots 10 managed in the location information management table corresponding to the base ID read out in step S151 (in this case, the base ID "locatinA"). Then, the search unit 73 identifies the robot 10 identified by the extracted terminal ID as the robot 10 located closest to the selected notification button 20A. In the following description, the search unit 73 will be described as identifying the robot 10A-1 identified by the terminal ID "rA01" as the robot 10 located closest to the selected notification button 20A. In the description, the search unit 73 identifies the robot 10 located closest to the notification button 20A, but the robot 10 may be identified using a photographed image sent from the robot 10 or information on a base that has been stored in advance, taking into consideration not only the distance from the notification button 20A but also the ease of movement, such as obstacles within the base and the movement route (for example, a straight route is preferable).

Next, the storage/readout unit 79 retrieves the status information of the robot 10A-1 by searching the status information management DB 7004 (see FIG. 11A) using the terminal ID of the robot 10A-1 identified in step S153 as a search key (step S154). Then, the judgment unit 72 judges whether the status information of the robot 10A-1 retrieved in step S154 satisfies the conditions indicated in the condition information retrieved in step S152 (step S155). Here, as shown in FIG. 11A, the status information includes information indicating the performance and current operating status of the robot 10. For example, the robot 10A-1 has a remaining battery level of 80%, the type of the mounted image capture device 12 is a movable camera, and the type of the mounted arm is a robot hand. Here, the status information is periodically transmitted from each robot 10 to the information processing server 70, and the status information management DB 7004 manages the status information of each robot 10. The status information may be configured to be transmitted from the robot 10 to the information processing server 70 in response to an acquisition request from the information processing server 70.

Here, if the status information of robot 10A-1 satisfies the condition indicated in the condition information read in step S152 (YES in step S155), the judgment unit 72 shifts the process to step S156. On the other hand, if the status information of robot 10A-1 does not satisfy the condition indicated in the condition information read in step S152 (NO in step S155), the judgment unit 72 repeats the process from step S153. In this case, in step S153, the search unit 73 searches the position information management DB 7002 (see FIG. 10(B)) from among the robots 10A other than robot 10A-1 to identify the robot 10 located closest to the selected notification button 20A.

Next, the determination unit 72 determines whether the robot 10A-1 identified in step S153 is within a predetermined range from the notification button 20A that transmitted the notification information received by the transmission/reception unit 71 (step S156). Specifically, the determination unit 72 determines whether the position information included in the notification information received by the transmission/reception unit 71 and the position information of the robot 10A-1 identified in step S153 are within a predetermined range. Here, the predetermined range is a range in which the position information included in the notification information and the position information of the robot 10A-1 are considered to be approximately the same. In other words, the predetermined range that serves as the determination criterion here is for determining whether the notification button 20A is mounted on the robot 10A-1, and is determined according to the type and accuracy of the data indicating the position information.

When the determination unit 72 determines that the robot 10A-1 is present within a predetermined range from the notification button 20A (YES in step S156), the process proceeds to step S157. Then, the generation unit 74 generates a button mounting flag indicating that the notification button 20A that transmitted the notification information is mounted on the robot 10A-1 identified in step S153 (step S157). On the other hand, when the determination unit 72 determines that the robot 10A-1 is not present within a predetermined range from the notification button 20A (NO in step S156), the process ends. Note that the generation unit 74 may generate information indicating whether or not the notification button 20A is mounted as a button mounting flag. In this case, when the determination unit 72 determines that the robot 10A-1 is not present within a predetermined range from the notification button 20A, the generation unit 74 generates a button mounting flag indicating that the notification button 20A is not mounted.

As a result, the information processing server 70 can identify, from among the multiple robots 10A installed at site A, the robot 10A-1 to be used for remote communication with the administrator terminal 50 used by the administrator, based on the position of the notification button 20A selected by the user at site A to call the administrator.

Returning to FIG. 14, the description will continue with the call process from the user at base A to the administrator. The transmission/reception unit 71 of the information processing server 70 transmits operation start request information indicating a control request requesting the start of operation of the robot 10 to be controlled to the robot 10A-1 identified by the search unit 73 (step S16). Here, the operation start request information includes the button mounting flag generated by the generation unit 74 and the position information of the notification button 20A received in step S14. Note that, if the information processing server 70 has not performed the process of step S157, the operation start request information may be configured not to include the button mounting flag, or may include information indicating that the notification button 20A is not mounted on the robot 10A-1 as the button mounting flag. As a result, the transmission/reception unit 31 of the robot 10A-1 receives the operation start request information transmitted from the information processing server 70.

Next, when the operation start request information is received in step S16, the communication control unit 35 of the robot 10A-1 places the robot 10A-1 in a communication standby state (step S17). Here, the communication standby state means, for example, that the robot 10A-1 is in a reserved state for remote communication with the administrator corresponding to the operation start request information received in step S16, without accepting control requests from other users or the administrator. Then, the robot 10A-1 executes the operation start process of the robot 10A-1 based on the operation start request information received in step S16 (step S18).

Now, the contents of the operation start processing of the robot 10A-1 will be described with reference to FIG. 16. FIG. 16 is a flowchart showing an example of the operation start processing in the robot according to the embodiment. First, the determination unit 34 determines whether or not a button mounting flag has been received by the transmission/reception unit 31 (step S181). When the determination unit 34 determines that a button mounting flag has been received, i.e., that the operation start request information includes a button mounting flag (YES in step S181), it transitions the process to step S182. In this case, the notification button 20A selected by the user in step S12 is the notification button 20A mounted on the robot 10A-1 (for example, the notification button 20-4 shown in FIG. 2).

Next, the captured image acquisition unit 41 acquires captured images of the subject around the robot 10A-1 (step S182). Specifically, the image capture instruction unit 38 transmits a capture instruction to the image capture device 12. Then, based on the image capture instruction from the image capture instruction unit 38, the captured image acquisition unit 41 acquires captured images of the subject using the image capture device 12.

Next, the user identification unit 42 identifies the user included in the captured image acquired by the captured image acquisition unit 41 (step S183). Specifically, the user identification unit 42 performs face matching processing (face authentication processing) on the image of the person's face included in the captured image using the feature amounts of each user stored in the user information management DB 3001 (see FIG. 8). The user identification unit 42 then identifies the user having the feature amount matched by the face matching processing as the user included in the captured image. Note that the face matching processing by the user identification unit 42 uses a general-purpose technology, so a detailed description is omitted.

Next, the destination setting unit 43 sets the position of the user identified by the user identification unit 42 as the destination of the robot 10A-1 (step S184). Then, the movement control unit 44 uses the movement mechanism 15 to start moving to the specific destination set by the destination setting unit 43 (step S185). In this case, the robot 10A-1 moves by following the user identified by the user identification unit 42. In addition, the user identified by the user identification unit 42 is the user who pressed the notification button 20A (notification button 20A-4 shown in FIG. 2) mounted on the robot 10A-1.

The robot 10A-1 starts moving while photographing the user with the photographing device 12, and by recognizing the photographed user in the same manner as the face matching process described above, identifies the user's location as it moves. Therefore, even if the user moves to a location where they wish to communicate remotely with the administrator, the robot 10A-1 can move by following the user, with the location of the user who pressed the notification button 20A as its destination. In addition, the user who pressed the notification button 20A can guide the robot 10A-1 to the location where they wish to communicate remotely with the administrator.

On the other hand, in step S181, if the determination unit 34 determines that the button mounting flag has not been received, i.e., that the operation start request information does not include a button mounting flag (NO in step S181), the process proceeds to step S186. In this case, the notification button 20A selected by the user in step S12 is not mounted on the robot 10A-1, but is a notification button 20A installed at a position away from the robot 10A-1 (for example, notification buttons 20A-1, 20A-2, and 20A-3 shown in FIG. 2).

The photographing instruction unit 38 sets the photographing direction (orientation) of the photographing device 12 to the direction indicated by the position information of the notification button 20A included in the operation start request information received by the transmission/reception unit 31 (step S186). For example, if the photographing device 12 is a movable camera, the photographing instruction unit 38 rotates the movable camera so that the direction indicated by the position information of the notification button 20A becomes the photographing direction of the movable camera. Also, if the support member 13 as shown in FIG. 3 is movable, the photographing instruction unit 38 rotates (turns) the support member 13 so that the direction indicated by the position information of the notification button 20A becomes the photographing direction of the photographing device 12. The following process is the same as that from step S182 onwards.

In this case, the robot 10A-1 also moves by following the user identified by the user identification unit 42. The user identified by the user identification unit 42 is the user who pressed the notification button 20A (notification button 20A-4 shown in FIG. 2) installed in a position away from the robot 10A-1. The robot 10A-1 identifies the user who pressed the notification button 20A by the user identification unit 42 by photographing the position (direction) of the selected notification button 20A. Therefore, the robot 10A-1 can move to the position of the user who is near the notification button 20A installed in a position away from the robot 10A-1. In addition, the user who is located away from the robot 10A-1 can guide the robot 10A-1 to the location where the user is located.

This allows the robot 10A-1 to start moving, with the location of the user who pressed the notification button 20A as its destination. Furthermore, the robot 10A-1 can identify the location of the user who pressed the notification button 20A, which is set as the destination of the robot 10A-1, regardless of the installation location of the selected notification button 20A.

The user's position set by the destination setting unit 43 is not the user's actual position, but a position that is a predetermined distance away from obstacles such as people and objects, including the user, so that the robot 10A-1 does not collide with obstacles such as people and objects at base A when moving. Furthermore, if an obstacle or the like is included in the captured image acquired in step S182, the robot 10A-1 may transmit a notification to the information processing server 70 indicating that it is unable to start the operation (movement). In this case, the information processing server 70 executes the robot search process again as shown in FIG. 15, among the robots 10 other than the robot 10A-1.

Returning to FIG. 14, the description will continue with the call process from the user at base A to the administrator. The storage/reading unit 79 of the information processing server 70 retrieves the corresponding administrator information by searching the administrator information management DB 7005 (see FIG. 11(B)) using the button ID included in the notification information received in step S14 as a search key (step S19). Here, the administrator information includes the administrator ID, the administrator name, and the administrator destination information. Next, the transmission/reception unit 71 of the information processing server 70 transmits communication request information indicating a communication request to the robot 10A-1 determined in step S15 to the administrator destination information read in step S19 (step S20). Here, the communication request information includes the terminal ID of the robot 10A-1 determined in step S15 and the button ID of the notification button 20A received in step S14. As a result, the transmission/reception unit 51 of the administrator terminal 50A used by the administrator of the destination indicated in the administrator destination information read in step S19 receives the communication request information transmitted from the information processing server 70. The transmitting/receiving unit 71 of the information processing server 70 also transmits the administrator information read in step S19 to the robot 10A-1 (step S21). This administrator information includes at least the administrator's name. As a result, the transmitting/receiving unit 31 of the robot 10A-1 receives the administrator information transmitted from the information processing server 70.

Note that steps S16 to S18 and steps S19 to S21 may be performed in reverse order, or may be performed in parallel.

Then, the robot 10A and the administrator terminal 50A execute the process of establishing a communication session, triggered by the administrator who has acquired the communication request information sent in step S20 (step S22).

Now, the process of establishing a communication session between the robot 10A-1 and the administrator terminal 50A will be described with reference to Figs. 17 to 19. Fig. 17 is a sequence diagram showing an example of the process in the preparation stage for starting data transmission and reception between a robot and an administrator terminal according to an embodiment. Here, the process of transmitting and receiving each piece of management information in the preparation stage before starting data transmission and reception between the administrator terminal 50A as the initiating terminal and the robot 10A-1 as the destination terminal will be described.

First, the transmission/reception unit 51 of the administrator terminal 50A transmits login request information to the communication management server 90 via the communication network 9 (step S101). Specifically, when the user of the administrator terminal 50A turns on the power switch of the administrator terminal 50A, the power is turned on. Then, in response to the power being turned on, the transmission/reception unit 51 of the administrator terminal 50A transmits login request information from the transmission/reception unit 51 to the communication management server 90 via the communication network 9. As a result, the transmission/reception unit 91 of the communication management server 90 receives the login request information transmitted from the administrator terminal 50A.

This login request information includes a terminal ID and a password for identifying the starting terminal as the administrator terminal 50A. The terminal ID and password are data read from the memory unit 5000 by the memory/read unit 59 and sent to the transmission/reception unit 51. Note that the terminal ID and password are not limited to this, and a terminal ID and password entered by the user using an input means such as the keyboard 511 may be sent. Also, a terminal ID and password read from a recording medium such as a SIM (Subscriber Identity Module Card) card or an SD card connected to the administrator terminal 50A may be sent.

When login request information is sent from the administrator terminal 50A to the communication management server 90, the communication management server 90, which is the receiving side, can obtain the IP address of the administrator terminal 50A, which is the sending side. Note that the login request does not necessarily have to be initiated by turning on the power switch, but may be sent in response to a user input to an input means such as the display 511.

Next, the authentication unit 92 of the communication management server 90 searches the authentication management table (see FIG. 12(A)) in the memory unit 9000 using the terminal ID and password included in the login request information received by the transmission/reception unit 91 as search keys, and performs authentication by determining whether the same terminal ID and the same password are managed in the authentication management DB 9001 (step S102). Below, we will explain the case where the authentication unit 92 determines that the administrator terminal 50A is a terminal with valid usage authority.

Next, if the authentication unit 92 determines that the login request is from a starting terminal with valid usage authority because the same terminal ID and the same password are managed by the authentication unit 92, the storage/reading unit 99 reads out the destination list frame data from the storage unit 9000 (step S103).

The transmission/reception unit 91 transmits authentication result information indicating the authentication result obtained by the authentication unit 92 to the administrator terminal 50A that made the login request via the communication network 9 (step S104). This authentication result information includes the destination list frame data read in step S103. As a result, the transmission/reception unit 51 of the administrator terminal 50A receives the authentication result information. Then, the storage/reading unit 59 of the administrator terminal 50A stores the destination list frame data received in step S104 in the storage unit 5000 (step S105).

Next, when the transmitting/receiving unit 51 receives authentication result information indicating that the authentication result indicates that the terminal has valid usage authority, it transmits destination list content request information requesting the contents of the destination list to the communication management server 90 via the communication network 9 (step S106). This destination list content request information includes the terminal ID of the administrator terminal 50A. As a result, the transmitting/receiving unit 91 of the communication management server 90 receives the destination list content request information.

Next, the storage/reading unit 99 of the communication management server 90 searches the destination list management DB 9003 (FIG. 13(A)) using the terminal ID "o01" of the manager terminal 50 received in step S106 as a search key to read out the terminal IDs of all corresponding destination candidates (step S107). Furthermore, the storage/reading unit 99 searches the terminal management DB 9002 (see FIG. 12(B)) using each terminal ID read in step S107 as a search key to read out the terminal name, operating status information, and base name of the corresponding destination candidate (step S108).

Next, the transmission/reception unit 91 transmits the destination list content information to the administrator terminal 50A via the communication network 9 (step S109). This destination list content information includes the terminal IDs of the destination candidates, the terminal names of the destination candidates, the operating status information, and the base names read in steps S107 and S108. As a result, the transmission/reception unit 51 of the administrator terminal 50A receives the destination list content information.

Next, the display control unit 53 of the administrator terminal 50A displays on the display 511 a destination list screen 900 created using the destination list frame data stored in the storage unit 5000 in step S105 and the destination list content information received in step S109 (step S110). FIG. 18 is a diagram showing an example of a destination list screen displayed on the administrator terminal according to the embodiment. The destination list screen 900 shown in FIG. 18 displays, for each destination candidate, an icon indicating the operating status of the destination candidate's terminal (robot 10), the terminal ID of the destination candidate's terminal, the destination name of the destination candidate, and the base name where the destination candidate's terminal is located. Note that the "terminal name" received in step S109 is displayed as the "destination name" on the destination list screen 900 shown in FIG. 18.

Next, the process from the selection of a destination candidate in the administrator terminal 50 to the start of sending and receiving image data will be described with reference to FIG. 19. FIG. 19 is a sequence diagram showing an example of the process from the selection of a destination candidate to the start of sending and receiving image data according to the embodiment.

First, the reception unit 52 of the administrator terminal 50A receives from the user the selection of a destination candidate (here, robot 10A-1) on the destination list screen 900 shown in FIG. 18 (step S111). Then, the transmission/reception unit 51 transmits start request information indicating a desire to start transmission/reception of image data, etc., to the communication management server 90 (step S112). This start request information includes the terminal ID of the administrator terminal 50A and the terminal ID of the destination candidate terminal. As a result, the transmission/reception unit 91 of the communication management server 90 receives the start request information.

Next, the judgment unit 93 of the communication management server 90 judges whether the terminal ID of the administrator terminal 50A received in step S112 is managed in the session management table (see FIG. 13(B)). Here, the explanation will continue below for the case where the terminal ID of the candidate destination terminal (robot 10A-1) is not managed.

If the terminal ID of the destination candidate terminal is not managed, the creation unit 94 creates a new session ID (step S114). Then, the storage/reading unit 99 adds and stores a new record in the session management table (see FIG. 13(B)) that associates the session ID created in step S114 with the terminal ID of the administrator terminal 50A received in step S112 and the terminal ID of the destination candidate terminal (step S115). Here, as shown in FIG. 13(B), the session ID "se3" and the terminal IDs "o01" and "rA01" are associated and managed by adding a new record.

Next, the transmission/reception unit 91 transmits session start request information to the administrator terminal 50A, requesting the start of a session (step S116). This session start request information includes the session ID created in step S114. As a result, the transmission/reception unit 51 of the administrator terminal 50A receives the session start request information.

The storage/reading unit 99 of the communication management server 90 searches the terminal management DB 9002 (see FIG. 12B) using the terminal ID of the candidate destination terminal (robot 10A-1) received in step S112 as a search key to read out the corresponding IP address (step S117). The transmission/reception unit 91 then transmits session start request information requesting the start of a session to the candidate destination terminal (robot 10A-1) indicated by the IP address read in step S117 (step S118). This session start request information includes the session ID created in step S114. As a result, the transmission/reception unit 31 of the destination terminal (robot 10A-1) receives the session start instruction.

As a result, the initiating terminal (administrator terminal 50A) and the destination terminal (robot 10A-1) each establish a communication session with the communication management server 90 (steps S119-1 and S119-2).

Then, as shown in FIG. 14, the transmission/reception unit 91 of the communication management server 90 transmits a session establishment notification to the information processing server 70 indicating that a communication session between the robot 10A-1 and the administrator terminal 50A has been established (step S23). As a result, the transmission/reception unit 71 of the information processing server 70 receives the session establishment notification transmitted from the communication management server 90. The robot 10A-1 and the administrator terminal 50A perform remote communication via the communication management server 90 using the communication session established by the processing shown in FIG. 17 to FIG. 19 (steps S24-1, S24-2). As a result, the remote control system 1a can start remote communication between the robot 10A-1 installed at the base A and the administrator terminal 50 located at a remote location. Then, the user who pressed the notification button 20A at the base A can use the robot 10A-1 to communicate with the administrator at a remote location who was called by pressing the notification button 20A as a trigger.

Here, the robot 10A-1 may perform the processes of steps S17 and S18 and steps S22 and S24-1 in parallel. After the robot 10A-1 transitions to a communication standby state in step S17, if the robot 10A-1 receives a session start request sent from the communication management server 90 (step S118 in FIG. 19), it is in a state where it can start remote communication at any time. That is, the robot 10A-1 can start remote communication with the administrator terminal 50A, for example, while moving to a destination set by the destination setting unit 43. In this case, the user who pressed the notification button 20A can communicate with the administrator while moving to a location that they want the remote administrator to see.

Another example of the robot's operation start processing Here, another example of the operation start processing of the robot 10A-1 shown in FIG. 16 will be described. FIG. 20 is a flowchart showing another example of the operation start processing in the robot according to the embodiment. The processing shown in FIG. 20 differs from the processing shown in FIG. 16 in the processing after the determination unit 34 determines that the button mounting flag has not been received (NO in step S181). That is, in FIG. 20, the processing when the notification button 20A pressed by the user is not mounted on the robot 10A-1 and is installed at a position away from the robot 10A-1 differs from the processing shown in FIG. 16. Only the points different from the processing shown in FIG. 16 will be described below. In step S181, when the determination unit 34 determines that the button mounting flag has not been received by the transmitting/receiving unit 31, that is, the operation start request information does not include the button mounting flag (NO in step S181), the determination unit 34 shifts the processing to step S186a.

The destination setting unit 43 of the robot 10A-1 sets the location indicated in the location information included in the operation start request information received by the transmission/reception unit 31 as the destination of the robot 10A-1 (step S186a). Then, the movement control unit 44 of the robot 10A-1 starts moving to the specific destination set by the destination setting unit 43 using the movement mechanism 15 (step S185). In this case, the robot 10A-1 moves to the location of the selected notification button 20A. Therefore, if the selected notification button 20A is installed in a location away from the robot 10A-1, the robot 10A-1 can move to the location of the selected notification button 20A. Also, a user who is in a location away from the robot 10A-1 can guide the robot 10A-1 to the location of the notification button 20A that he or she pressed.

The position set by the destination setting unit 43 is not the exact position of the notification button 20A, but a position that is a predetermined distance away from obstacles such as people or objects at base A so that the robot 10A-1 does not collide with such obstacles when moving.

As a result, the robot 10A-1 can set the destination of the robot 10A-1 not only to the location of the user who pressed the notification button 20A, but also to the location of the selected notification button 20A, depending on the installation location of the selected notification button 20A. Note that when the notification button 20A shown in FIG. 16 or FIG. 20 is not mounted on the robot 10A-1, the method of setting the destination may be configured such that the method to be executed is preset, or, for example, the process shown in FIG. 20 is executed if the user cannot be identified by the process shown in FIG. 16.

Notification of communication status Next, a process for notifying a user in a base of the communication status of the robot 10 will be described with reference to Figs. 21 to 23. Fig. 21 is a flow chart showing an example of a process for notifying a user in a base of a communication status in a robot according to an embodiment. The process of Fig. 21 is started from a state in which the robot 10 is on standby as shown in Fig. 22(A). As shown in Fig. 22(A), the robot 10 in the standby state has the lamp 17 off and "on standby" displayed on the notification panel 18. Such a notification by the lamp 17 and the notification panel 18 corresponds to "notification pattern C" stored in the notification pattern management DB 3009 (see Fig. 8(B)).

If the robot 10A-1 has been transitioned to a communication standby state in step S17 (YES in step S201), the process proceeds to step S202. On the other hand, the robot 10A-1 repeats the process of step S201 until it is transitioned to a communication standby state in step S17 (NO in step S201).

Next, the storage/readout unit 39 retrieves the notification pattern (notification pattern B in this case) corresponding to the communication state "waiting for communication" by searching the notification pattern management DB 3009 (see FIG. 8B) (step S202). Then, the communication state notification control unit 49 switches the communication state notification according to the notification pattern retrieved in step S202 (step S203). Specifically, as shown in FIG. 22B, the communication state notification control unit 49 turns on the lamp 17 based on a notification pattern (notification pattern B) different from the waiting notification pattern (notification pattern C). In addition, the communication state notification control unit 49 causes the notification panel 18 to display information of "waiting for communication". The communication state notification control unit 49 controls the color or blinking regularity of the lamp 17 according to the notification pattern, as well as the information displayed on the notification panel 18, so that the user can visually grasp the communication state.

Next, if the robot 10A-1 receives the administrator information in step S21 (YES in step S204), the robot 10A-1 transitions to step S205. On the other hand, the robot 10A-1 repeats the process of step S204 until the administrator information is received in step S21 (NO in step S204). Then, the communication status notification control unit 49 causes the notification panel 18 to display the administrator name included in the administrator information received in step S21 (step S205), as shown in FIG. 22(B).

Next, when a communication session with the administrator terminal 50A is established in step S23 (YES in step S206), the robot 10A-1 shifts the process to step S207. On the other hand, the robot 10A-1 repeats the process of step S206 until a communication session with the administrator terminal 50A is established in step S23 (NO in step S206). Then, the communication status notification control unit 49 switches the notification of the communication status according to the notification pattern (notification pattern A in this case) corresponding to the communication status "communicating" read by searching the notification pattern management DB 3009 (see FIG. 8 (B)) (step S207). Specifically, as shown in FIG. 23, the communication status notification control unit 49 switches the illumination of the lamp 17 based on the notification pattern corresponding to the communication status "communicating". In addition, the communication status notification control unit 49 switches the display of the notification panel 18 from "waiting for communication" to "communicating".

In this way, the robot 10A-1 can change the color or blinking regularity (timing) of the lamp 17, or the information displayed on the notification panel 18, depending on the communication status, allowing users at the base to understand the communication status of the robot 10A-1 or information about the communication partner. For example, the users at the base can understand from the notified information whether the robot 10A-1 is on standby or waiting for communication, thereby reducing the discomfort or inconvenience caused by the sudden start of remote communication that the users themselves do not intend.

The notification by the lamp 17 and the notification panel 18 as the communication information control means may be configured to be notified by at least one of them. The information displayed on the notification panel 18 may be divided and displayed on multiple notification panels, or some or all of it may be displayed on the display 14. Furthermore, the communication status notification means is not limited to the lamp 17 and the notification panel 18, and may be configured to notify the communication status of the robot 10 by a sound emitted from a speaker, etc.

Cancellation of a communication request Next, a process of canceling a communication request for starting remote communication between the robot 10 and the manager terminal 50 will be described. First, a process of canceling a communication request when remote communication does not start even after a predetermined time has elapsed in a communication waiting state will be described with reference to Fig. 24. Fig. 24 is a flowchart showing an example of a process of canceling a communication request in the information processing server according to the embodiment.

When the measurement unit 75 of the information processing server 70 transmits the communication request information to the administrator terminal 50A in step S20, the measurement unit 75 starts measuring the processing time using the timer 716 (step S401). In this case, the timing to start measuring the processing time may be when the operation start request information is transmitted in step S16, or when the session establishment process in step S22 is started.

Next, if a predetermined time has elapsed since the start of measurement in step S401 (YES in step S402), the measurement unit 75 transitions the process to step S403. Then, the transmission/reception unit 71 transmits cancellation information indicating that the communication request is to be cancelled to the robot 10A-1 and the administrator terminal 50A (step S403).

On the other hand, if the predetermined time has not elapsed since the start of measurement in step S401 (NO in step S402), the measurement unit 75 transitions the process to step S404. If the information processing server 70 receives a session establishment notification in step S23 (YES in step S404), the communication session is established within the predetermined time, so the process ends and remote communication between the robot 10A-1 and the administrator terminal 50A is initiated. On the other hand, if the information processing server 70 does not receive a session establishment notification in step S23 (NO in step S404), it repeats the process from step S402.

Furthermore, the process of canceling a communication request in response to a request from a user within a base will be described with reference to FIG. 25. FIG. 25 is a sequence diagram showing an example of the process of canceling a communication request in a remote control system according to an embodiment. Note that in the process of FIG. 25, the robot 10A-1 will be described as being in a communication standby state to which it was transitioned in step S17.

First, when a user in base A presses the notification button 20A installed in base A, the reception unit 22 of the notification button 20A accepts the selection of the button (step S451). The notification button 20A in this case may be the same as the button selected in step S12, or may be a dedicated button for canceling the communication request. When the selection of the button is accepted in step S451, the location information acquisition unit 23 acquires the current location information of the notification button 20A (step S452). Note that the acquisition of the location information by the location information acquisition unit 23 is not limited to a configuration in which it is performed at the timing when the selection of the button is accepted by the reception unit 22, and may be performed periodically at a predetermined timing. Then, the transmission/reception unit 21 transmits cancellation request information indicating a request to cancel the remote communication to the information processing server 70 (step S453). Here, the cancellation request information includes the button ID of the notification button 20A and the location information acquired in step S452. As a result, the transmission/reception unit 71 of the information processing server 70 receives the cancellation request information sent from the notification button 20A.

Next, the storage/reading unit 79 of the information processing server 70 searches the administrator information management DB 7005 (see FIG. 11B) using the button ID included in the cancellation request information received in step S453 as a search key to read out the corresponding administrator information (step S454). Here, the administrator information includes the administrator ID, administrator name, and administrator address information. Next, the transmission/reception unit 71 of the information processing server 70 transmits cancellation information indicating that the communication request for the robot 10A-1 determined in step S15 is to be cancelled to the administrator address information read out in step S454 (step S455). This cancellation information includes the terminal ID of the robot 10A-1 determined in step S15, and the button ID of the notification button 20A received in step S454. As a result, the transceiver unit 51 of the administrator terminal 50A used by the administrator whose destination is indicated in the administrator destination information read in step S454 receives the cancellation information sent from the information processing server 70 and stops the process of starting remote communication with the robot 10A-1.

In this way, the remote control system 1a can cancel a process to start remote communication that has already started if remote communication between the robot 10A-1 and the manager terminal 50A does not start after a predetermined time has elapsed or in response to a request from a user at the base. This allows a user at the base to cancel the process in a simple manner when there is no response from the manager despite having called the manager or when the call is no longer necessary after having called the manager, thereby improving convenience.

Processing for Ending Remote Communication Next, processing for ending remote communication between the administrator terminal 50 and the robot 10 will be described with reference to Fig. 26 and Fig. 27. Fig. 26 is a sequence diagram showing an example of processing for ending remote communication between a robot according to an embodiment and an administrator terminal. Note that the processing shown in Fig. 26 is processing in the case where the administrator terminal 50A and the robot 10A-1 have been performing remote communication using the communication session established by the processing shown in Figs. 17 to 19.

First, the transmitter/receiver 31 of the robot 10A-1 transmits communication end information to the information processing server 70 indicating that remote communication with the administrator terminal 50A has ended (step S31). Here, the communication end information transmitted from the robot 10A-1 includes the captured image data relating to the captured image acquired by the captured image acquisition unit 41, and a user ID for identifying the user who selected the notification button 20 in step S12.

The transmitting/receiving unit 51 of the administrator terminal 50A also transmits communication end information to the information processing server 70 indicating that remote communication with the robot 10A-1 has ended (step S32). Here, the communication end information transmitted from the administrator terminal 50A includes an administrator ID for identifying the administrator who performed remote communication using the administrator terminal 50A. As a result, the transmitting/receiving unit 71 of the information processing server 70 receives the communication end information transmitted from the administrator terminal 50A. Note that the order of the processing of steps S31 and S32 may be reversed, or may be performed in parallel.

Then, the generating unit 74 of the information processing server 70 generates notification history information indicating the notification history from the user at base A to the administrator at the remote location using the communication end information received in steps S31 and S32 (step S33). The storing and reading unit 79 then stores the notification history information generated in step S33 in the notification history management DB 7006 (see FIG. 11(C)) (step S34). Specifically, the storing and reading unit 79 stores the notification history information generated by the generating unit 74 in the storage destination link indicated in the notification history management table.

Here, an example of the notification history information stored and managed in the notification history management DB 7006 will be described. FIG. 27 is a diagram showing an example of the notification history information according to the embodiment. FIG. 27 shows an example of a notification history display screen 600 in which the notification history information stored in the notification history management DB 7006 is displayed on the display 14 of the robot 10 or the display 511 of the administrator terminal 50. The notification history display screen 600 shown in FIG. 27 includes a notification history data display area 611 for displaying the notification history data, a base image 613 of the inside of base A photographed by the imaging device 12 of the robot 10A-1, and a text display area 615 showing information related to the remote communication performed between the robot 10A-1 and the administrator terminal 50A.

Of these, the notification history data display area 611 displays information on the worker name (user name), base name, button name (name of the selected notification button 20A), manager name, and communication time identified by the notification history data stored in the notification history management DB 7006. The text display area 615 displays, in text format, the content of the conversation during remote communication and notes entered using the input means provided on the manager terminal 50 or the robot 10A-1. The notification history display screen 600 may also include an image of the manager captured by the CMOS sensor 505a of the manager terminal 50.

By managing the notification history information in the information processing server 70, the remote control system 1a can store information about who called whom and what the remote communications were about. Therefore, the user or manager of the base can use the information shown in the notification history information as a means of improving events that have occurred or are likely to occur within the base. Note that, although an example has been described in which the notification history information is stored and managed in the information processing server 70, the notification history information may also be stored in the robot 10 or the manager terminal 50.

Processing at the End of Remote Communication Next, a movement process of the robot 10 at the end of remote communication will be described with reference to Fig. 28. Fig. 28 is a flowchart showing an example of a process at the end of remote communication in the robot according to the embodiment.

First, when the robot 10A-1 has ended remote communication with the administrator terminal 50A (YES in step S301), the process proceeds to step S302. On the other hand, when the robot 10A-1 is in remote communication with the administrator terminal 50A (NO in step S301), the robot 10A-1 repeats the process of step S301 and continues remote communication with the administrator terminal 50A.

Next, the destination setting unit 43 sets a preset reference position as the destination of the robot 10A-1 (step S302). Here, the reference position is the installation position of the power receiving station as shown in FIG. 3, or the position of the robot 10A-1 before the start of movement in step S185, etc. Then, the movement control unit 44 uses the movement mechanism 15 to start movement to the destination set by the destination setting unit 43 (step S303).

In this way, when remote communication ends, the robot 10 starts moving to a preset reference position and automatically moves away from the user, thereby reducing the user's discomfort caused by the robot 10 being nearby even after remote communication ends.

As described above, the remote control system 1a according to the embodiment can, when a user at a base presses the notification button 20, trigger the user at the base to call a manager at a remote location and perform remote communication. In addition, the remote control system 1a can move the robot 10A-1 installed at the base to a specific destination that is set based on the position information of the selected notification button 20A, allowing the user at the base to communicate with the manager at the location where they wish to perform remote communication.

Modifications of the embodiment
Next, a remote control system according to a modified embodiment will be described with reference to Figs. 29 and 30. The same configurations and functions as those of the above embodiment will be denoted by the same reference numerals, and the description thereof will be omitted. The remote control system 1b according to the modified embodiment is a system including a robot 10 (hereinafter referred to as a robot having a server function) having the function of the information processing server 70 as shown in the above embodiment, among a plurality of robots installed in a base. In the following description, the robot 10 having the server function will be described as the robot 10a (control device 30a), and the other robot 10 not having the server function will be described as the robot 10b (control device 30b). The robot 10b (control device 30b) has the same configuration as the robot 10 (control device 30) described in the above embodiment. The control device 30a is an example of an information processing device.

Functional configuration Fig. 29 is a diagram showing an example of the functional configuration of a remote control system according to a modified embodiment. Note that the functions of the devices or terminals other than the robot 10a (control device 30a) having a server function are similar to those shown in Fig. 7, and therefore description thereof will be omitted.

The control device 30a that controls the processing or operation of the robot 10a has a transmission/reception unit 31a, a reception unit 32a, a display control unit 33a, a judgment unit 34a, a communication control unit 35a, a status information acquisition unit 36a, a position information acquisition unit 37a, a photography instruction unit 38a, a photographed image acquisition unit 41a, a user identification unit 42a, a movement destination setting unit 43a, a movement control unit 44a, an arm operation control unit 45a, a communication status notification control unit 49a, and a memory/readout unit 39. These have the same configuration as the transmission/reception unit 31, the reception unit 32, the display control unit 33, the judgment unit 34, the communication control unit 35, the status information acquisition unit 36, the position information acquisition unit 37, the photography instruction unit 38, the photographed image acquisition unit 41, the user identification unit 42, the movement destination setting unit 43, the movement control unit 44, the arm operation control unit 45, the communication status notification control unit 49, and the memory/readout unit 39 in the robot 10 (robot 10b), respectively. Furthermore, in addition to the above configuration, the control device 30a has a search unit 46a, a generation unit 47a, and a measurement unit 48a. The search unit 46a, the generation unit 47a, and the measurement unit 48a have the same configuration as the search unit 73, the generation unit 74, and the measurement unit 75 in the information processing server 70, respectively. Here, the transmission/reception unit 31a is an example of a notification information receiving means. Also, the transmission/reception unit 31a is an example of an output means. Furthermore, the transmission/reception unit 31a is an example of a communication request sending means. Also, the search unit 46a is an example of a determination means.

In addition to the configuration of the memory unit 3000 of the robot 10 (robot 10b), the memory unit 3000a also includes a base information management DB 3002, a position information management DB 3003, a condition information management DB 3004, a state information management DB 3005, an administrator information management DB 3006, and a notification history management DB 3007. These have the same configuration as the base information management DB 7001, the position information management DB 3003, the condition information management DB 7003, the state information management DB 7004, the administrator information management DB 7005, and the notification history management DB 7006 in the information processing server 70, respectively. The memory unit 3000a is an example of a storage means.

● Processing or Operation of Modified Example of the Embodiment Next, the operation or process of the remote control system 1b according to the modified example of the embodiment will be described with reference to FIG. 30. FIG. 30 is a sequence diagram showing an example of a call process from a base system in the remote control system according to the modified example of the embodiment. In FIG. 30, the process executed by the control device 30aA-5 included in the robot 10aA-5 will be described as the process executed by the robot 10aA-5. Also, the process executed by the control device 30bA-1 included in the robot 10bA-1 will be described as the process executed by the robot 10bA-1.

The robots 10bA-1 and 10aA-5 installed at base A are in a standby state until they receive a control request from the information processing server 70 (steps S51-1 and S51-2). This process is similar to the process of step S11 shown in FIG. 14.

Next, when a user (worker) located at base A wants to call a manager at a remote location, for example, due to the occurrence of some kind of abnormality during work, he or she presses the notification button 20A installed at base A. In this case, when the user presses the notification button 20A, the reception unit 22 of the notification button 20A accepts the selection of the button (step S52). When the selection of the button is accepted in step S52, the position information acquisition unit 23 acquires the current position information of the notification button 20A (step S53). Then, the transmission/reception unit 21 transmits notification information for calling a manager at a remote location to the robot 10aA-5 having a server function (step S54). Here, the notification information includes the button ID of the notification button 20A and the position information acquired in step S53. As a result, the transmission/reception unit 31a of the robot 10aA-5 receives the notification information transmitted from the notification button 20A.

Next, the robot 10aA-5 executes a search process for the robot 10 located at base A to be controlled based on the notification information received in step S54 (step S55). The search process for the robot 10 in the robot 10aA-5 is similar to the process in the information processing server 70 as shown in FIG. 15, so a description thereof will be omitted. In this case, the robot 10aA-5 will be described as having identified the robot 10bA-1 as the control target.

Next, the transmitter/receiver 31a of the robot 10aA-5 transmits operation start request information indicating a control request requesting that the robot 10 to be controlled start operating to the robot 10bA-1 identified by the search unit 46 (step S56). Here, the operation start request information includes the button mounting flag generated by the generator 47 and the position information of the notification button 20A received in step S54. Note that if the processing of step S157 has not been performed by the robot 10aA-5, the operation start request information does not include the button mounting flag. As a result, the transmitter/receiver 31 of the robot 10bA-1 receives the operation start request information transmitted from the robot 10aA-5.

Next, when the operation start request information is received in step S56, the communication control unit 35 of the robot 10bA-1 puts the robot 10bA-1 into a communication standby state (step S57). Here, the communication standby state means, for example, that the robot 10bA-1 does not accept control requests from other users or administrators and is in a reserved state for remote communication with the administrator corresponding to the operation start request information received in step S56. Then, the robot 10bA-1 executes an operation start process for the robot 10bA-1 based on the operation start request information received in step S56 (step S58). The operation start process in the robot 10bA-1 is similar to the process shown in FIG. 16 or FIG. 20, so a description thereof will be omitted. The robot 10bA-1 starts moving to the destination specified by the destination setting unit 43a by executing the process shown in FIG. 16 or FIG. 20.

Furthermore, the storage/reading unit 39a of the robot 10aA-5 retrieves the corresponding administrator information by searching the administrator information management DB 3006 using the button ID included in the notification information received in step S54 as a search key (step S59). Here, the administrator information includes the administrator ID, the administrator name, and the administrator address information. Next, the transmission/reception unit 31a of the robot 10aA-5 transmits communication request information indicating a communication request to the robot 10bA-1 determined in step S55 to the administrator address information retrieved in step S59 (step S60). Here, the communication request information includes the terminal ID of the robot 10bA-1 determined in step S55, and the button ID of the notification button 20A received in step S54. As a result, the transmission/reception unit 51 of the administrator terminal 50A used by the administrator of the address indicated in the administrator address information retrieved in step S59 receives the communication request information transmitted from the robot 10aA-5. The transmitter/receiver 31a of the robot 10aA-5 also transmits the administrator information read in step S59 to the robot 10bA-1 (step S61). This administrator information includes at least the administrator's name. As a result, the transmitter/receiver 31 of the robot 10bA-1 receives the administrator information transmitted from the robot 10aA-5. Note that the processing of steps S56 to S58 and the processing of steps S59 to S61 may be performed in reverse order, or may be performed in parallel.

Then, the robot 10bA-1 and the administrator terminal 50A execute the communication session establishment process shown in FIG. 17 to FIG. 19, triggered by the administrator who has acquired the communication request information transmitted in step S60 (step S62). The transmission/reception unit 91 of the communication management server 90 transmits a session establishment notification to the robot 10aA-5 indicating that a communication session between the robot 10bA-1 and the administrator terminal 50A has been established (step S63). As a result, the transmission/reception unit 31a of the robot 10aA-5 receives the session establishment notification transmitted from the communication management server 90. Then, the robot 10bA-1 and the administrator terminal 50A perform remote communication via the communication management server 90 using the communication session established by the process of step S62 (steps S64-1, S64-2). As a result, the remote control system 1b can start remote communication between the robot 10bA-1 installed at the base A and the administrator terminal 50A located at a remote location, similar to the case of the remote control system 1a described in the above embodiment.

In this way, even in the case where the remote control system 1b according to the modified embodiment is configured to include a robot 10a with a server function in the base system 2, the user at the base can call an administrator at a remote location and perform remote communication when the user at the base presses the notification button 20. Note that other processes of the information processing server 70 described in the above embodiment are executed by the robot 10a performing similar processes. In other words, the remote control system 1b according to the modified embodiment can also perform the processes shown in Figures 21 to 28.

●Summary●
As described above, the information processing device according to one embodiment of the present invention is an information processing device (for example, information processing server 70 or control device 30a) that controls the operation of a telepresence robot (for example, robot 10A) installed in a specific base (for example, base A). The information processing device receives notification information including position information of the notification button 20 from a notification button 20 (one example of an operation means) that accepts operations from a user installed in the base, outputs a request to start operation to a specific destination to a specific telepresence robot (for example, robot 10A-1) determined based on the position information included in the received notification information and the position information of the telepresence robot, and transmits a communication request to the specific telepresence robot to an administrator terminal 50 (for example, administrator terminal 50A) that performs remote communication with the telepresence robot. Then, the information processing device starts the movement of the specific telepresence robot to the specific destination and starts remote communication between the administrator terminal 50 and the specific telepresence robot. As a result, the information processing device can start remote communication between the specific telepresence robot determined according to the position of the notification button 20A and the administrator terminal 50. In addition, the information processing device can move a specific telepresence robot to a location where a user wants to perform remote communication by starting the movement of the specific telepresence robot to a specific destination. Therefore, the information processing device can improve the convenience of remote communication using the telepresence robot.

Furthermore, an information processing device according to one embodiment of the present invention (e.g., information processing server 70 or control device 30a) determines a specific telepresence robot (e.g., robot 10A-1) that is closest to the notification button 20A (one example of an operating means) based on the position information included in the received notification information and the position information of multiple telepresence robots (e.g., robot 10A), and if the remaining battery charge of the determined specific telepresence robot is equal to or greater than a predetermined value, outputs a request to the specific telepresence robot to start operating to a specific destination. This allows the information processing device to request the appropriate telepresence robot to start operating among the telepresence robots installed within the base, depending on the position of the selected notification button 20A and the state of the telepresence robot.

Furthermore, the telepresence robot according to one embodiment of the present invention is a telepresence robot (e.g., robot 10, 10b) that starts operating in response to a request from an information processing device (e.g., information processing server 70 or control device 30a). The telepresence robot acquires an operation start request to a specific destination from the information processing device, starts moving to the specific destination indicated in the acquired operation start request, and when the movement starts, starts remote communication with the manager terminal 50 in response to a request from the manager terminal 50. This allows the telepresence robot to start moving and start remote communication with the manager terminal 50 based on an operation start request sent as a trigger by a call from a user at the base. Therefore, the telepresence robot can improve the convenience of remote communication with a manager at a remote location.

Furthermore, when a telepresence robot according to one embodiment of the present invention (e.g., robot 10, 10b) receives a request to start an operation to a specific destination, it photographs the user who operates the notification button 20 (one example of an operating means), and if the notification button 20 is mounted on the telepresence robot, it sets the position of the photographed user as the specific destination. The telepresence robot then starts moving to the specific destination that has been set. As a result, even if the user moves to a location where the user wants to communicate remotely with the administrator, the telepresence robot can move by following the user, with the position of the user who pressed the notification button 20 as the destination. Furthermore, the user who pressed the notification button 20 can guide the telepresence robot to the location where the user wants to communicate remotely with the administrator.

Furthermore, when a telepresence robot according to one embodiment of the present invention (e.g., robot 10, 10b) receives a request to start moving to a specific destination and the telepresence robot is not equipped with a notification button 20 (one example of an operating means), the telepresence robot sets the position of the notification button 20 as the specific destination. The telepresence robot then starts moving to the specific destination that has been set. This allows the telepresence robot to move to the position of the notification button 20 when the notification button 20 is installed at a distant location. Also, a user who is at a distant location from the telepresence robot can guide the telepresence robot to the position of the notification button 20 that they pressed.

In addition, in a telepresence robot (e.g., robot 10, 10b) according to one embodiment of the present invention, the acquired operation start request includes a button mounting flag (an example of mounting availability information) indicating whether or not a notification button 20 (an example of an operating means) is mounted on the telepresence robot. If the acquired button mounting flag indicates that the notification button 20 is mounted on the telepresence robot, the telepresence robot sets the position of the photographed user as a specific destination, and if the acquired button mounting flag indicates that the notification button 20 is not mounted on the telepresence robot, the telepresence robot sets the position of the notification button 20 as a specific destination. This allows the telepresence robot to set a destination according to the button mounting flag output from an information processing device (e.g., information processing server 70 or control device 30a), and therefore to start moving to a different destination according to the installation position of the selected notification button 20.

Furthermore, a remote control system according to one embodiment of the present invention is a remote control system 1a, 1b including a base control system 3 including an information processing device (e.g., an information processing server 70 or a control device 30a) and a telepresence robot (e.g., robot 10, 10b) that starts operating in response to a request from the information processing device, and an administrator terminal 50. The administrator terminal 50 receives a communication request from the information processing device, and starts remote communication with a specific telepresence robot (e.g., robot 10A-1) in response to the received communication request. In this way, the remote control system 1a, 1b can start remote communication between the telepresence robot and the administrator terminal 50 located at a remote location, triggered by the selection of a notification button 20A installed in the base.

The remote control system 1a, 1b according to one embodiment of the present invention also includes a storage means (e.g., storage unit 7000) that stores communication history information indicating the communication history between a specific telepresence robot (e.g., robot 10A-1) and the administrator terminal 50 (e.g., administrator terminal 50A), including information on the user who operated the notification button 20 (one example of an operation means) and the administrator who uses the administrator terminal 50. This allows the remote control system 1a, 1b to store information on who called whom and what the content of the remote communication that was executed was. Therefore, the user or administrator of the base can use the content shown in the notification history information as material for improving events that have occurred or may occur within the base thereafter.

●Additional Information●
So far, we have described an information processing device, telepresence robot, base control system, remote control system, information processing method, and program according to one embodiment of the present invention. However, the present invention is not limited to the above-described embodiment, and other embodiments can be added, modified, or deleted, etc., within the scope that can be conceived by a person skilled in the art. In any aspect, as long as the function and effect of the present invention are achieved, it is included in the scope of the present invention.

The functions of the above-described embodiments can be realized by one or more processing circuits. In this specification, the term "processing circuit" includes a processor that is programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), SOCs (Systems on a chip), GPUs (Graphics Processing Units), and conventional circuit modules that are designed to execute each function described above.

Furthermore, the various tables in the embodiments described above may be generated by the learning effect of machine learning, and by classifying the data of each associated item by machine learning, it is not necessary to use tables. Here, machine learning is a technology that allows a computer to acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates algorithms necessary for judgments such as data identification from learning data that is previously imported, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

1a, 1b Remote control system 3 Base control system 10, 10a, 10b Robot (an example of a telepresence robot)
12. Photographing device (an example of a photographing means)
17 Lamp (an example of a communication status notification means)
18 Notification panel (an example of a communication status notification means)
20 Notification button (an example of an operation means)
30, 30b Control device 30a Control device (an example of an information processing device)
31, 31b Transmitter/receiver (an example of an acquisition means)
31a Transmitting/receiving unit (an example of a notification information receiving means, an example of an output means, an example of a communication request transmitting means)
35, 35b Communication control unit (an example of a communication control means)
43, 43b Destination setting unit (an example of a setting means)
44, 44b Movement control unit (an example of a movement control means)
46a Search unit (an example of a determining means)
50 Administrator terminal 51 Transmitting/receiving unit (an example of a communication request receiving means, an example of a communication means)
70 Information processing server (an example of an information processing device)
71 Transmitting/receiving unit (an example of a notification information receiving means, an example of an output means, an example of a communication request transmitting means)
73 Search unit (an example of a determination means)
3000, 3000a, 3000b Storage unit (an example of a storage means)
5000 Memory unit (an example of a memory means)
7000 Storage unit (an example of a storage means)
90 Communication management server

JP 2018-121322 A

Claims (22)

An information processing device that controls the operation of a telepresence robot installed in a predetermined base,
a notification information receiving means for receiving notification information including location information of an operation means installed in the base and configured to receive operations from a user;
an output means for outputting a request to start an operation to a specific destination to a specific telepresence robot determined based on position information included in the received notification information and position information of the telepresence robot;
a communication request sending means for sending a communication request for the specific telepresence robot to an administrator terminal that remotely communicates with the telepresence robot;
An information processing device that starts movement of the specific telepresence robot to the specific destination and starts remote communication between the administrator terminal and the specific telepresence robot. The information processing device according to claim 1, further comprising:
a determining unit that determines the specific telepresence robot that is closest to the operating unit based on position information included in the received notification information and position information of the plurality of telepresence robots;
The output means is an information processing device that outputs the operation start request to the specific telepresence robot when the determined specific telepresence robot satisfies a predetermined condition regarding the state of the telepresence robot. the predetermined condition is a condition related to a remaining battery charge of the telepresence robot,
The information processing apparatus according to claim 2 , wherein the output means outputs the operation start request to the specific telepresence robot when a remaining battery charge of the specific telepresence robot is equal to or greater than a predetermined value. the specific destination is a position of a user who has performed an operation on the operation means,
The information processing device according to claim 1 , further comprising: a control unit for controlling a telepresence robot to start moving to a position of the user. the specific destination is a position indicated in the received position information of the operating means,
The information processing apparatus according to claim 1 , further comprising: a control unit for controlling a telepresence robot to move to a position of the operating means. The information processing device according to any one of claims 1 to 5, wherein the communication request sending means sends cancellation information for canceling the communication request to the administrator terminal if the remote communication is not started until a predetermined time has elapsed since the communication request was sent. the notification information receiving means receives a request to cancel the remote communication from the operation means;
The information processing apparatus according to claim 1 , wherein the communication request transmitting means transmits, to the manager terminal, cancellation information for canceling the communication request. A telepresence robot that starts an operation in response to a request from the information processing device according to any one of claims 1 to 7,
an acquisition means for acquiring the operation start request from the information processing device;
A movement control means for starting a movement to a specific movement destination indicated in the acquired operation start request;
a communication control means for initiating remote communication with the manager terminal in response to a request from the manager terminal when the movement is started;
A telepresence robot comprising: The telepresence robot according to claim 8, further comprising:
an image capturing means for capturing an image of a user who has performed an operation on the operation means when the operation start request is acquired;
a setting means for setting the photographed position of the user as the specific destination;
The movement control means is a telepresence robot that starts moving to the set specific destination when the position of the user photographed is set as the specific destination by the setting means . The telepresence robot according to claim 9, wherein the setting means sets the photographed position of the user as the specific destination when the operation means is mounted on the telepresence robot. The telepresence robot according to claim 8, further comprising:
a setting means for setting a position of the operating means as the specific movement destination when the operation start request is acquired;
The movement control means is a telepresence robot that starts moving to the set specific movement destination when the position of the operation means is set as the specific movement destination by the setting means . The telepresence robot according to claim 11, wherein the setting means sets the position of the operating means as the specific movement destination when the operating means is not mounted on the telepresence robot. The telepresence robot according to claim 10 or 12,
The operation start request includes mounting availability information indicating whether the operation means is mounted on the telepresence robot, and further includes
The setting means is
If the acquired mounting availability information indicates that the operation means is mounted on the telepresence robot, the position of the photographed user is set as a specific destination;
A telepresence robot that sets the position of the operating means as the specific destination when the acquired mounting availability information indicates that the operating means is not mounted on the telepresence robot. The telepresence robot according to any one of claims 8 to 13, wherein the movement control means starts movement to a preset reference position when the remote communication ends. The telepresence robot according to any one of claims 8 to 14, further comprising:
A telepresence robot comprising a communication status notifying means for notifying a communication status of the remote communication. The telepresence robot according to claim 15, wherein the communication status notification means lights a lamp so that the communication status can be distinguished. The telepresence robot according to claim 15 or 16, wherein the communication status notification means displays information about the administrator who uses the administrator terminal. A base control system comprising the information processing device according to any one of claims 1 to 7 and the telepresence robot that starts operating in response to a request from the information processing device. A remote control system comprising the base control system according to claim 18 and the manager terminal,
The administrator terminal,
a communication request receiving means for receiving the communication request from the information processing device;
A communication means for initiating remote communication with the particular telepresence robot in response to a received communication request;
A remote control system comprising: 20. The remote control system of claim 19,
A remote control system comprising a storage means for storing communication history information indicating a communication history between the specific telepresence robot and the administrator terminal, including information on the user and the administrator who uses the administrator terminal. An information processing method executed by an information processing device that controls an operation of a telepresence robot installed in a predetermined base, comprising:
a notification information receiving step of receiving, from an operation means installed in the base and configured to receive an operation from a user, notification information including location information of the operation means;
an output step of outputting a request to start an operation to a specific destination to a specific telepresence robot determined based on position information included in the received notification information and position information of the telepresence robot;
A communication request transmission step of transmitting a communication request for the specific telepresence robot to an administrator terminal that performs remote communication with the telepresence robot;
An information processing method for initiating movement of the specific telepresence robot to the specific destination and initiating remote communication between the administrator terminal and the specific telepresence robot. A program for causing a computer to execute the method according to claim 21.