JP7591891B2 - Communication Systems and Robots - Google Patents

Description

The present invention relates to a communication system and a robot.

In recent years, videoconferencing systems using the Internet have become widespread, and telepresence robots are being used that not only allow users to talk face-to-face, but also allow users in remote locations to control the camera's direction and position.

For example, the following Patent Document 1 describes a camera-equipped mobile object that includes a camera that provides a captured image to a remote operator, and a mask processing unit that executes mask processing to hide at least a portion of the captured image from the remote operator and switches the mask processing depending on the application.

JP 2019-062308 A

Conventional telepresence robots are started to be used when a remote operator establishes a communication connection with the telepresence robot using a remote operation terminal, and operation from the remote operator is the starting point. However, there are cases where a user at the location where the telepresence robot is placed (hereinafter also referred to as a local user) also wants to contact the remote operator via the telepresence robot. Despite this, the local user has no choice but to wait for the remote operator to contact them. Thus, currently, the initiation of communication between the remote operator and the local user is unilaterally initiated by the remote operator.

Therefore, the present invention provides technology that enables more convenient communication for both the robot operator and the local user.

A communication system according to one aspect of the present invention includes an unfixed robot and a computer capable of communicating with the robot, the robot having a camera, a drive unit capable of remote control by the computer, a first input unit for inputting instructions to the robot, and a call unit for sending a call notification to the computer in response to instructions input to the first input unit when a communication connection between the robot and the computer is not established, and the computer has a second input unit for inputting commands to the robot, and a display unit for displaying an image captured by the camera and displaying, when the call notification is received from the robot, that the notification has been received.

According to this aspect, by sending a call notification from the robot to the computer in response to an input to the first input unit by a local user, it is possible to prompt the remote operator of the robot to connect to the robot from the computer. Therefore, it is possible to realize a communication system that is easy to use for both the remote operator and the local user.

In the above aspect, when the display unit displays that the call notification has been received, the computer establishes a communication connection with the robot in response to an input to the second input unit. According to this aspect, the computer can connect to the robot that has sent the call notification.

In the above aspect, the robot may further have a storage unit that stores information representing at least one computer capable of communicating with the robot, and the call unit may transmit the call notification to a computer corresponding to information selected from the information representing the at least one computer in response to input to the first input unit. According to this aspect, the call notification can be transmitted to a computer selected from computers whose information is previously stored in the robot.

In the above aspect, the robot may connect to the Internet in response to an input to the first input unit, search for information on computers that can be connected to the robot, and the call unit may transmit the call notification to a computer that corresponds to information selected from the searched information in response to the input to the first input unit. According to this aspect, a call notification can be transmitted to the searched computer via the Internet.

In the above aspect, the first input unit is a voice input unit that accepts voice input, the robot further has a voice recognition unit that recognizes the voice input to the first input unit and converts it into text, and the call unit may send the call notification to a preset computer among the at least one computer when a voice including specific text is input to the first input unit. According to this aspect, a call notification can be sent to a preset computer in response to the user's voice.

A robot according to another aspect of the present invention is a mobile or wearable robot that has a driving unit that can be remotely controlled by a computer, an input unit that inputs instructions to the robot, and a calling unit that sends a call notification to the computer in response to an input to the input unit when a communication connection with the computer is not established.

According to this aspect, by sending a call notification from the robot to the computer in response to an input to the input unit by the local user, it is possible to prompt the remote operator of the robot to connect to the robot from the computer. Therefore, it is possible to realize a communication system that is easy to use for both the remote operator and the local user.

The present invention makes it possible to realize a communication system and robot that are easy to use for both the remote operator of the telepresence robot and the local user.

1 is a diagram showing a configuration of a communication system according to an embodiment of the present invention. FIG. 1 is a diagram showing the physical configuration of a computer according to an embodiment of the present invention. FIG. 1 is a diagram showing a physical configuration of a robot according to an embodiment of the present invention. 4 is a flowchart of a process executed by a computer and a robot according to the present embodiment. FIG. 11 is a diagram showing an example of a screen displayed on the robot according to the embodiment. FIG. 11 is a diagram showing an example of a screen displayed on a computer according to the embodiment. FIG. 13 is a diagram showing a modified example of a screen displayed on the robot according to the embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the attached drawings. In each drawing, the same reference numerals are used to denote the same or similar configurations.

(Embodiment)
1 is a diagram showing a configuration of a communication system 100 according to an embodiment of the present invention. The communication system 100 includes an unfixed robot 20 and a computer 10 capable of communicating with the robot 20. Here, the robot 20 and the computer 10 being capable of communicating with each other includes being capable of wireless communication or wired communication, and including being capable of communication via the Internet or a LAN (Local Area Network) or being capable of short-distance wireless communication. Note that, although one robot 20 and one computer 10 are exemplified in this embodiment, the number of these is arbitrary. In other words, one robot 20 may be capable of communicating with multiple computers 10, and one computer 10 may be capable of communicating with multiple robots 20.

The remote operator 1 operates the computer 10 to remotely control the robot 20 that is connected to the computer 10 for communication. This allows the remote operator 1 to communicate with another user (hereinafter also referred to as a local user) 2 at the location where the robot 20 is located via the robot 20, and to make the robot 20 perform desired actions.

The robot 20 has a camera and a drive unit that can be remotely controlled by a computer that is communicatively connected to the robot 20. The robot 20 is, for example, a general-purpose or dedicated telepresence robot or avatar robot, and may have a moving unit such as wheels.

The robot 20 operates based on commands from the computer 10 with which it is connected via communication. The connection modes of the computer 10 to the robot 20 may include a private mode that requires user authentication, and a public mode that does not require user authentication. User authentication in the private mode may be performed using a known method, and information for user authentication may be registered in advance. In addition, a predetermined range of access rights is granted to the computer 10 connected to the robot 20 in the public mode.

The robot 20 being not fixed includes a case where the robot 20 is a mobile type having a moving part such as wheels, and a case where the robot 20 is a wearable type having a driving part such as a manipulator that can be worn by a person. A mobile robot is shown in, for example, Patent Document 1. The moving parts of a mobile robot include those that run on one wheel, two wheels or multiple wheels, those that run on caterpillar tracks, those that run on rails, those that move by jumping, those that walk on two legs, four legs or multiple legs, those that navigate on or underwater with a screw, and those that fly with a propeller or the like. A wearable robot is disclosed, for example, in MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa and Masahiko Inami, "Fusion: full body surrogacy for collaborative communication," Proceeding SIGGRAPH '18 ACM SIGGRAPH 2018 Emerging Technologies Article No. 7. Furthermore, the robot 20 includes vehicles and heavy machinery that can run automatically or semi-automatically, drones and airplanes. The robot 20 also includes a robot equipped with a camera that is installed in a sports stadium or the like and can move on rails. The robot 20 also includes a satellite-type robot that is launched into outer space and can control its attitude and the direction in which the camera shoots.

The computer 10 has an input unit for inputting commands to the robot 20, and a display unit for displaying images captured by the camera of the robot 20. The computer 10 is, for example, a smartphone, but may also be another computer (information processing device) such as a personal computer.

Figure 2 is a diagram showing the physical configuration of the computer 10 according to this embodiment. The computer 10 has a CPU (Central Processing Unit) 10a corresponding to the calculation unit, a RAM (Random Access Memory) 10b corresponding to the storage unit, a ROM (Read only Memory) 10c corresponding to the storage unit, a communication unit 10d, an input unit 10e, and a display unit 10f. These components are connected to each other via a bus so that data can be transmitted and received. Note that in this example, the computer 10 is configured by a single computer, but the computer 10 may be realized by combining multiple computers. Also, the configuration shown in Figure 2 is an example, and the computer 10 may have other configurations than these, or may not have some of these configurations.

The CPU 10a is a control unit that controls the execution of programs stored in the RAM 10b or ROM 10c and calculates and processes data. The CPU 10a is a calculation unit that executes a program (communication program) that controls communication via the robot. The CPU 10a receives various data from the input unit 10e and communication unit 10d, and displays the results of calculations on the data on the display unit 10f or stores them in the RAM 10b.

RAM 10b is a storage unit in which data can be rewritten, and may be composed of, for example, a semiconductor memory element. RAM 10b may store data such as programs executed by CPU 10a and user information. Note that these are merely examples, and RAM 10b may store data other than these, or some of these data may not be stored.

ROM 10c is a memory section from which data can be read, and may be configured, for example, with a semiconductor memory element. ROM 10c may store, for example, a communication program or data that is not rewritten.

The communication unit 10d is an interface that connects the computer 10 to other devices. The communication unit 10d may be connected to a communication network such as the Internet.

The input unit 10e accepts instructions and information input from a user, and may include, for example, a keyboard and a touch panel. The input unit 10e may also include a microphone for voice input.

The display unit 10f visually displays the results of calculations performed by the CPU 10a, and may be configured, for example, with an LCD (Liquid Crystal Display). The display unit 10f may display images captured by the camera 20h of the robot 20. The display unit 10f may also display emails, short messages (SMS), text messages, etc. received by the computer 10.

In addition, the computer 10 may have an audio output unit that generates a notification sound when the computer 10 receives an email or message, or that reads out the received email or message.

The communication program may be provided by being stored in a computer-readable storage medium such as RAM 10b or ROM 10c, or may be provided via a communication network connected by communication unit 10d. In computer 10, CPU 10a executes the communication program to realize various operations for controlling robot 20. Note that these physical configurations are merely examples and do not necessarily have to be independent configurations. For example, computer 10 may be provided with an LSI (Large-Scale Integration) in which CPU 10a is integrated with RAM 10b and ROM 10c.

Figure 3 is a diagram showing the physical configuration of the robot 20 according to this embodiment. The robot 20 has a CPU 20a equivalent to a calculation unit, a RAM 20b equivalent to a storage unit, a ROM 20c equivalent to a storage unit, a communication unit 20d, an input unit 20e, a display unit 20f, a drive unit 20g, and a camera 20h. Each of these components is connected via a bus so that data can be transmitted and received between them. Note that the configuration shown in Figure 3 is an example, and the robot 20 may have other components or may not have some of these components.

The CPU 20a is a control unit that controls the execution of programs stored in the RAM 20b or the ROM 20c and calculates and processes data. The CPU 20a is a calculation unit that executes a program (communication program) that controls communication via the robot. The CPU 20a receives various data from the input unit 20e and the communication unit 20d, and displays the results of calculations on the data on the display unit 20f or stores them in the RAM 20b. The CPU 20a also controls the drive unit 20g, which controls the operation of the robot 20.

The CPU 20a also functions as a calling unit that sends a call notification to the computer 10 in response to an input to the input unit 20e when a communication connection has not been established between the robot 20 and the computer 10.

RAM 20b is a storage unit in which data can be rewritten, and may be configured, for example, with a semiconductor memory element. RAM 20b may store programs executed by CPU 20a. Note that these are merely examples, and RAM 20b may store data other than these, or some of these may not be stored.

ROM 20c is a memory section from which data can be read, and may be configured, for example, with a semiconductor memory element. ROM 20c may store, for example, a communication program or data that is not rewritten.

The communication unit 20d is an interface that connects the robot 20 to other devices. The communication unit 20d may be connected to a communication network such as the Internet.

The input unit 20e accepts instructions and information input from the user, and may include, for example, a touch panel. The input unit 20e may also include a microphone for voice input.

The display unit 20f visually displays the results of calculations performed by the CPU 20a, and may be configured, for example, with an LCD. If the computer 10 is equipped with a camera, the display unit 20f may display an image captured by the camera of the computer 10.

The driving unit 20g includes a remotely controlled actuator, and includes a moving part such as wheels, a manipulator, etc. When the robot 20 is a mobile robot, the driving unit 20g includes at least a moving part such as wheels, but may also include a manipulator. When the robot 20 is a wearable robot, the driving unit 20g includes at least a manipulator.

The camera 20h includes an imaging element that captures still images or videos, and transmits the captured still images or videos to the computer 10 via the communication unit 20d.

The communication program may be provided by being stored in a computer-readable storage medium such as RAM 20b or ROM 20c, or may be provided via a communication network connected by communication unit 20d. In robot 20, CPU 20a executes the communication program to realize various operations for controlling robot 20. Note that these physical configurations are merely examples and do not necessarily have to be independent configurations. For example, robot 20 may be equipped with an LSI in which CPU 20a is integrated with RAM 20b and ROM 20c.

Figure 4 is a flowchart of the processes executed by the computer 10 and the robot 20 according to this embodiment. Figure 5 is a diagram showing an example of a screen displayed on the robot 20 according to this embodiment. Figure 6 is a diagram showing an example of a screen displayed on the computer 10 according to this embodiment.

While a communication connection with the computer 10 has not been established, the robot 20 waits for a connection from the computer 10 (S20).

When the robot 20 detects an input operation to the input unit 20e (S21: Yes), the robot 20 displays a preregistered operator name on the display unit 20f (S22). Here, the preregistered operator name is information for which authentication information is prestored in the robot 20, and is information representing a computer (e.g., a user or owner of a computer) that can communicate with the robot 20 (e.g., communication for calls such as voice calls and video calls). The robot 20 may hold such information in a format such as an address book.

For example, the robot 20 displays a menu screen as shown in FIG. 5(a) in response to a tap operation on the input unit 20e. Then, when the robot 20 detects a tap operation on the call button 201 on the menu screen, it displays a selection screen as shown in FIG. 5(b). On the selection screen, icons 202 to 205 displaying the names of operators registered in advance are displayed.

Next, when the robot 20 detects that an arbitrary operator has been selected (S23: Yes), it sends a call notification to the computer 10 of the selected operator (S24). For example, when the robot 20 detects a tap operation on the icon 202 labeled "Mr. A" among the icons 202 to 205 shown in FIG. 5(b), it sends a call notification by email or message to Mr. A's computer.

When the computer 10 receives a call notification from the robot 20 (S11: Yes), it displays the received call notification on the display unit 10f (S12). The notification screen shown in FIG. 6 displays a text message 101 indicating that the robot 20 is calling. Alternatively, the computer 10 may generate a ringtone indicating that the call notification has been received, or may read out the text message 101 aloud. The computer 10 may also display a connect button 102 including a link to connect to the calling robot 20, or a cancel button 103 to reject the connection, on the notification screen.

When the computer 10 detects an instruction to connect to the robot 20 in the input unit 10e (S13: Yes), the computer 10 executes a process to connect to the robot 20 (S14). For example, when the computer 10 detects a tap operation on the connection button 102 shown in FIG. 6, the computer 10 executes a connection process (e.g., a communication connection for a call with the robot 20). The connection process may include a login authentication process, etc. Note that even if the computer 10 does not receive a call notification (S11: No), it can connect to the robot 20 at any time by detecting an instruction to connect in the input unit 10e.

When the robot 20 receives a connection request from the computer 10 (S25: Yes), it executes authentication processing etc. as necessary and connects to the computer 10 (S26). While connected to the computer 10, the robot 20 operates based on commands from the computer 10. The display unit 20f of the robot 20 may display an image captured by the camera of the computer 10 while the robot 20 is connected to the computer 10 by telephone.

When the computer 10 detects an instruction to disconnect from the robot 20 in the input unit 10e (S15: Yes), it disconnects from the robot 20 (S16). The process then ends. When the connection from the computer 10 is disconnected (S27: Yes), the robot 20 ends the process.

As described above, according to this embodiment, the local user 2 at the location where the robot 20 is located can not only wait for the remote operator 1 to connect the computer 10 to the robot 20, but can also operate the robot 20 to send a call notification to the computer 10, thereby urging the remote operator 1 to connect to the robot 20. Therefore, it is possible to realize a communication system that is easy to use for both the remote operator 1 and the local user 2.

(Variation 1)
7 is a diagram showing an example of a screen displayed on the robot 20 according to a modified example of the above embodiment. In the above embodiment, when the robot 20 detects an input operation to the input unit 20e (S21: Yes), the preregistered operator name is displayed on the display unit 20f. However, the robot 20 may be configured to search for information on the computer 10 that can be connected without authentication via the Internet.

For example, the robot 20 displays a menu screen as illustrated in FIG. 7(a) in response to a tap operation on the input unit 20e. Then, when the robot 20 detects a tap operation on a selection button 211 on the menu screen, for example, displaying "MARKET PLACE", the robot 20 connects to the Internet, performs a search, and displays a service selection screen as illustrated in FIG. 7(b). On the service selection screen, icons 212-215 indicating services that can be provided via the robot 20 are displayed. Examples of services that can be provided via the robot 20 include medical consultation services and English conversation lessons. Alternatively, the service may be a service that involves a specific task, such as cleaning.

When the robot 20 detects a tap operation on the icon 213 displayed, for example, as "Service Y" on the service selection screen, the robot 20 displays a provider selection screen as shown in FIG. 7(c). On the provider selection screen, icons 216-219 indicating providers of service Y are displayed. Then, when the robot 20 detects a tap operation on the icon 217 displayed, for example, as "Company F" on the provider selection screen, the robot 20 displays a confirmation screen as shown in FIG. 7(d). The confirmation screen shows detailed information about Company F and the services that Company F provides.

Furthermore, when the robot 20 detects a tap operation on the request button 220 on the confirmation screen, it sends a call notification to the computer 10 of company F. When the computer 10 of company F receives the call notification from the robot 20, it executes a process to connect to the robot 20. Note that if the computer 10 that receives the call notification does not have the authority to connect to the robot 20, a predetermined range of authority may be automatically granted to the computer 10.

(Variation 2)
In the above embodiment, a call notification is sent to a computer selected from among computers with which the robot 20 can communicate in response to an operation on the screen displayed on the display unit 20f of the robot 20, but the call notification may be sent in response to a voice input to the robot 20. In this case, the CPU 20a also functions as a voice recognition unit that recognizes voice input from a microphone for voice input and converts it into text.

For example, when the robot 20 recognizes a specific voice such as "help me," it sends a call notification to a computer 10 that has been preset for emergency use. Examples of emergency computers include a computer used by a family member of the user 2 in the local area, or a computer belonging to the user's primary care doctor.

(Variation 3)
In the above embodiment, when the display unit 10f of the computer 10 displays the fact that the call notification has been received, the computer 10 executes a process of connecting communication for a call with the robot 20 in response to an input to the input unit 10e. As a modified example, the computer 10 may transmit a response notification to the call notification to the robot 20 in response to an input to the input unit 10e after the display unit 10f of the computer 10 displays the fact that the call notification has been received. The robot 20 may present a message in response to the response notification received from the computer 10 to the user (in front of the robot 20). The message in response to the response notification includes, for example, a voice message output from the speaker of the robot 20, or a text or image message displayed on the display unit of the robot 20. The message includes, for example, "Please wait a little longer,""I will connect now," and the like. After transmitting the response notification, the computer 10 may execute a process of connecting communication for a call with the robot 20 in response to an input to the input unit 10e.

The above-described embodiments and modifications are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The elements of the embodiments and modifications, as well as their arrangements, materials, conditions, shapes, sizes, etc., are not limited to those exemplified, and may be modified as appropriate. In addition, configurations shown in different embodiments and modifications may be partially substituted or combined.

1...Remote operator, 2...Local user, 10...Computer, 20...Robot, 10a, 20a...CPU, 10b, 20b...RAM, 10c, 20c...ROM, 10d, 20d...Communication unit, 10e, 20e...Input unit, 10f, 20f...Display unit, 20g...Drive unit, 20b...RAM, 20h...Camera, 100...Communication system, 101...Text message, 102...Connect button, 103...Cancel button, 201...Call button, 202-205, 212-219...Icon, 211...Select button, 220...Request button

Claims (6)

A communication system comprising a non-fixed robot and a computer capable of communicating with the robot,
The robot comprises:
Camera and
A drive unit that can be remotely controlled by the computer;
A first input unit that inputs an instruction to the robot;
a calling unit that transmits a call notification to the computer in response to an instruction input to the first input unit when a communication connection with the computer is not established,
The computer includes:
a second input unit for inputting a command for the robot;
a display unit that displays an image captured by the camera and, when the call notification is received from the robot, displays a message indicating that the notification has been received.
In the communication system,
The robot further includes a storage unit that stores information representing at least one computer that can communicate with the robot;
the calling unit transmits the call notification to a computer corresponding to information selected in response to the input to the first input unit from among the information representing the at least one computer.
Communication system . The communication system according to claim 1, wherein when the display unit displays that the call notification has been received, the computer establishes a communication connection for a call with the robot in response to an input to the second input unit. The robot connects to the Internet in response to an input to the first input unit, and searches for information about a computer that can be connected to the robot;
the calling unit transmits the call notification to a computer corresponding to information selected from the searched information in response to the input to the first input unit.
3. A communication system according to claim 1 or 2. the first input unit is a voice input unit that accepts voice input,
The robot further includes a voice recognition unit that recognizes a voice input to the first input unit and converts the voice into text,
the calling unit transmits the call notification to a preset computer among the at least one computer when a voice including specific text is input to the first input unit.
3. A communication system according to claim 1 or 2. when the display unit displays the receipt of the call notification, the computer transmits a response notification to the robot in response to the input to the second input unit, and then establishes a communication connection for a call with the robot;
The communication system according to claim 1 , wherein the robot presents a message to the user in response to the response notification received from the computer. A mobile or wearable robot that can be worn by a person ,
A drive unit that can be remotely controlled by a computer;
an input unit for inputting instructions to the robot;
a calling unit that transmits a call notification to the computer in response to an input to the input unit when a communication connection with the computer is not established;
A storage unit that stores information representing at least one computer that can communicate with the robot;
having
the calling unit transmits the call notification to a computer corresponding to information selected in response to the input to the input unit from among the information representing the at least one computer.
robot.

Images