WO2023146514A1

WO2023146514A1 - Notification presentation and execution on a remote device

Info

Publication number: WO2023146514A1
Application number: PCT/US2022/013763
Authority: WO
Inventors: Yiwei ZHAO; Xiaofeng Li
Original assignee: Innopeak Technology Inc
Current assignee: Innopeak Technology Inc
Priority date: 2022-01-25
Filing date: 2022-01-25
Publication date: 2023-08-03
Anticipated expiration: 2024-07-25

Abstract

In various embodiments, a method for distributing a notification across multiple devices is provided. The multiple devices are located within a distributed computing environment. In those embodiments, the method comprises receiving, at a first device in distributed computing environment, a notification task indicating a notification for presentation to a user associated with the first device. Then the notification is presented at the first device according to the notification task. A user response is received at the first device after the notification is presented. Then, at the first device, a determination is made that first device not capable of handling the user response. After that determination, the first device forwards the user response to a second device in the distributed computing environment for processing the user response according to the notification task.

Description

NOTIFICATION PRESENTATION AND EXECUTION ON A REMOTE DEVICE

BACKGROUND

[0001] Notification is generally known. A notification service provides means to send a notice to many persons at once. For example, if a flood were likely, residents of a community could be warned it's time to evacuate. If a school were suddenly closed for the day, students or parents could be told not come to school, or told to report to an alternate location. American Airlines can notify passengers by cellphone two hours before a flight's departure with information on the flight's status and gate number.

[0002] On a smart device, a notification is typically received from a service or is generated by the smart device. Typically, the notification is to notify a user of certain information. The notification is usually presented by the smart device, for example on a display of the smart device. Some smart device may present the notification audibly (for example, such as a smart speaker) rather than graphically or visually.

[0003] Some smart devices may be associated with some other smart devices. For example, slave/master is a technique used for a wearable device associated with a smart phone. Because the wearable device is limited in its communication and/or processing power, it typically is associated with the smart phone. Thus, the wearable device has to be registered with the smart phone in order for it to be able to receive a notification from the smart phone. In some techniques, the smart phone can be configured to forward the notification to the wearable device for presentation after the wearable device is registered with the smart phone.

SUMMARY

[0004] In various embodiments, a method for distributing a notification across multiple devices is provided. In this method, the multiple devices are located within a distributed computing environment such as user’s home or in a school. In those embodiments, the method comprises receiving, at a first device in distributed computing environment, a notification task indicating a notification for presentation to a user associated with the first device. Then the notification is presented at the first device according to the notification task. A user response is received at the first device after the notification is presented. Then, at the first device, a determination is made that first device not capable of handling the user response. After that determination, the first device forwards the user response to a second device in the distributed computing environment for processing the user response according to the notification task. [0005] In this way, user responses to a notification can be processed in a remote device as opposed to the notification presentation device. For instance, a notification in accordance with the present disclosure is first presented on the first device, e.g., a smart TV. A user response to the notification is then received by the smart TV. The smart TV determines that is not capable of processing this response and then forwards the user response to the second device, e.g., a smart phone, for processing the user response. In various embodiments, the user response may cause an action or state change to an application or app after the notification is presented, e.g., on the smart TV. For example, as illustration thus non-limiting, the notification may notify a user of a chat message to the user from a sender. The user can send a response message back using an input means associated with the smart TV. The response message can be forwarded to smart phone where a chat app is installed for processing, e.g., addressing it to the sender.

[0006] Other benefits and embodiments are contemplated.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] A further understanding of the nature and advantages of various embodiments may be realized by reference to the following figures. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

[0008] FIG. 1 illustrates one example multi-device environment in accordance with one embodiment.

[0009] FIG. 2 illustrates an example of implementing the distributed notification in accordance with the present disclosure.

[0010] FIG. 3 illustrates an example of the distributed notification module shown in FIG. 2 in accordance with the present disclosure.

[0011] FIG. 4 illustrates an example of a notification task for implementing the distributed notification in accordance with the disclosure.

[0012] FIG. 5 illustrates one example of multiple types of contents corresponding to different presentation methods are present in a notification in accordance with the present disclosure. [0013] FIG. 6 illustrates two examples of response handler 310 in accordance with the present disclosure.

[0014] FIG. 7 illustrates an example flow for processing a response received by a device after a notification has been presented on the device in the DAE.

[0015] FIG. 8 illustrates an example of a computer system that can be configured to implement various features and/or functions described herein.

DETAILED DESCRIPTION

[0016] Having described several example configurations, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over or otherwise modify the application of the invention. Also, a number of steps may be undertaken before, during, or after the above elements are considered.

NOTIFICATION

[0017] In accordance with the present disclosure, a notification may be referred to a piece of information to be presented to a user to notify the user of certain information such as a happening in a service and/or in an application. Therefore, for a purpose of the present disclosure, a notification in accordance with the present disclosure is distinguished over a content in an application presented to the user. For example, certain applications or devices may be configured to present a content to the user when the applications are running in the background. In that situation, the presentation of this content is not regarded as a notification in accordance with the present disclosure even though the content is presented to the user unsolicited. As used herein, a notification includes information indicating a happening, which may be related to a (computer) application or (smart device) app, to a service employed by the user, or any information of interest to the user. In that sense, a notification in accordance with the present disclosure is presented to the user to notify the user of this information so to let the user decide whether he/she should take an action in response.

DISTRIBUTED DEVICE ENVIRONMENT

[0018] Various embodiments relate to a distributed device environment wherein one or more users are associated with multiple devices. For illustrating the distributed notification in accordance with the present disclosure, FIG. 1 illustrates an example of a multiple device environment 100 in accordance with the present disclosure. As can be seen from FIG. 1, in the multi-device environment 100, devices 102a, 102b, 102c and any other devices (if any) are associated with users 104a-n. It should be understood, the number of devices and users in accordance with the present disclosure can be more or less than those shown in FIG. 1. For example, it is contemplated that the distributed notification in accordance with the present disclosure can be applied to a multiple device environment where there is one user and two devices. Other examples are contemplated.

[0019] In various embodiments, individual devices 102a-c can comprise a smart phone, a tablet, a laptop computer, a netbook, a desktop computer, a computer system (such as a server with terminal), a wearable device, a smart speaker, a raspberry Pi computer, an internet of things (loT) device, a smart camera, a smart home appliance, and/or any other devices. In various embodiments, the individual devices 102a-c share a distributed computing environment such that cross-device computing among these devices are implemented. For example, in one embodiment, it is contemplated that device 102a is a smartphone, device 102b is a smart TV, device 102c is a smart TV. In that embodiment, device 102b and/or device 102c are configured to execute one or more services provided by the device 102a, and vice versa. In this sense, in that embodiment, the devices 102a-c operate in a distributed operating environment. In one instance, the distributed operating environment 100 is distributed android environment (DAE), where devices 102a-c operate as android systems. However, the present disclosure may also include devices operating in a mixed operating environment. For instance, it is contemplated that device 102a may be iPhone running iOS and devices 102b and 102c may be a smart TV and a PC running android and Windows systems, respectively. Other types of operating systems that can be run by devices 102a- c are contemplated.

[0020] As can be seen, devices 102a-c, in this example, are connected to a cloud 106, for instance, provided by a vendor such as Google. However, this is not intended to be limiting. In some other examples, one or more of devices 102a-c may or may not be connected to the cloud 106. Also shown are third party systems HOa-n, for example, such as a messaging system, a video service system, a streaming service system, a gaming system, and/or any other third-party systems. As shown, these systems can be connected to the devices 102a-c via the cloud 106. However, this is also not intended to be limiting. It is contemplated that these systems may be connected to devices 102a-c directly or indirectly over one or more communications networks not involving cloud 106.

[0021] As also can be seen, the devices 102a-c, in this example, are connected to one another over one or more communication channels represented by the dotted lines shown in FIG. 1. For instance, the device 102a may be connected to the device 102b over a local wi-fi network and connected to the device 102c over a Bluetooth connection; and the devices 102b and 102c may be connected over the local wi-fi network. Other examples of communication channels among devices 102a-c are contemplated. In some embodiments, the devices 102a-c may also be connected to one another through the cloud 106.

[0022] As mentioned above, in the distributed device environment 100 shown in FIG. 1, the devices 102a-c can run different operating systems and software. Traditionally, when there is a notification for presentation to the user(s) 104, the notification is typically presented on the device where the notification is received. For instance, as an illustration, a notification may be transmitted by the cloud 106 for presentation to the user(s) 104 on the device 102a, which may be a smartphone. In that instance, the notification is then only presented on device 102a after device 102a receives it regardless whether the user(s) 104 will be indeed notified. In a situation where the user(s) 104 is watching a TV program on the device 102b (for example, a smart TV) in a living room and the device 102a is being charged in a study room away from the living room, the presentation of the notification on the device 102a would not notify the user(s) 104. In that scenario, it is desired to present the notification on the device 102b, rather than the device 102a.

[0023] In US Patent Application entitled “Distributed Notification”, various techniques for presenting a notification on a device in the multi-device environment 100 are described and illustrated. In those techniques, the device presenting the notification may not be the device that receives the notification. For instance, a notification may be first received by device 102a from cloud 106 and then presented on device 102b (rather than device 102a). This may be referred to remote presentation of a notification on a device that is not a notification receiving device. One insight provided by the present disclosure is that remote presentation of the notification in the multi-device environment 100 can lead to challenges in a few situations.

[0024] A first challenge is that the presentation device may not have a capability for processing the notification after it is presented on the device. As mentioned in section “Notification” above, while some notification need not allow a user to take actions, some notifications may allow a user to take an action in response to the notification. As illustration, for instance, a weather alert notification may simply notify the user(s) 104 of a weather condition. Thus, such a notification may be presented to the user on the device and disappear. For example, the weather alert notification may be reported to the user audibly on device 102b if device 102b is a smart speaker. However, a notification such as chat message notification may provide more than just a chat message received. For example, the chat message notification may provide the user(s) 104 a few “canned” responses for the user to respond to the message received in the notification, such as “Got it, thanks!”, “I will be on my way”, “sorry I am in something right now, and will get it back to you as soon as I am done”, etc. In that case, device 102b may not be capable of enabling the user(s) 104 to take one of these actions in response to the chat message notification.

[0025] Another challenge is that a notification presenting device in the multi-device environment 100 may not be capable of handling user inputs even if actions are provided to the user(s) 104 in the notification presentation. For instance, if the device 102b is a smart TV, it can be configured to present the chat message notification with the canned responses in a notification window shown on the display. However, device 102b is not able to process a user input in response to the notification. For instance, device 102b in that case will not be able to take a user selection of canned “Yes!” as a response and send it back to a chat service provided by a third- party system such as system 110a shown in FIG. 1. For device 102b to do that, it would need to install a chat application employing the chat service to know how and where to address this response. This may not be possible due to various constraints, such as device 102b does not have an operating system and/or processing power to execute the chat application.

[0026] Still another challenge is that some devices in the multi-device environment 100 may be capable of only processing some responses provided by user(s) 104 in response to the notification, but not all of them. For example, a notification presented on device 102b (smart TV) may have an action called “dismiss” to allow the user to dismiss the presentation of the notification on device 102b. This simple action may be processed and completed by device 102b - by stop presenting notification. However, as mentioned above, actions related to a function or feature in an application/service/app for the notification may not be readily processed by device 102b.

[0027] Thus, in accordance with the present disclosure, techniques are provided to present and/or process user inputs for a notification on a device in multi-device environment 100. In some embodiments, a notification is presented on a presenting device in multi-device environment 100, and the presenting device is configured to receive user inputs for the notification, determine whether it can process the user inputs and pass to another device multi-device environment 100 for processing the user inputs if the presenting device itself is not capable of doing so. In some embodiments, the presenting device is configured to process some types of user inputs for the notification based on its processing capability and pass other types of user inputs to another device for further processing.

EXAMPLE IMPLEMENTATION OF DISTRIBUTED NOTIFICATION

[0028] IG. 2 illustrates an example of implementing the distributed notification in accordance with the present disclosure. It is described with reference to FIG. 1. Attention is now directed to FIG. 2. As shown in this example, device 102a shown in FIG. 1 is configured to receive data from cloud 106 and comprises components facilitating the distribution notification in accordance with the present disclosure. Also shown in this example are devices 102b and 102c. Although, no details are shown for these devices in FIG. 2, it should be understood that these devices may have more or less than the same components shown to be included in device 102a.

[0029] In this example, device 102a includes a message service component 202, applications (or apps) 204a-c, a system user interface presentation component 206, a notification manager component 208, a system framework 210, and/or any other components. While these components will be described below, they should not be understood as limiting the present disclosure. That is, it is understood, in some other embodiments, device 102a may not have one or more of these components, or may have some or all of these components implemented as a same component.

[0030] In this example, as shown, the device 102a is configured to receive data from cloud 106. The data from cloud 106 may indicate one or more actions, results, implementations, and/or any other forms of information for device 102a. For example, the data may include information indicating a chat message received for the user(s) 104 associated with the device 102a, information indicating an email message received for the user(s) 104, information indicating a video to be presented to the user(s) 104, and/or any other information. After the data is received by device 102a, the message service component 202 is configured to generate notification information. The notification information includes information indicating a notification for presentation to the user(s) 104 by virtue of the data has been received. For instance, by way of example, hence not intended to be limiting, the data may include information indicating a chat message for the user(s) 104, and the message service component 202 is configured to generate a notification for presentation to the user(s) to notify the user(s) 104 that the chat message has been received.

[0031] The applications 204a-c are individual programs implemented on the device 102a. For example, application 204a may be a chat program, application 204b may be a video streaming service, and application 204c may be an email client. Although three applications 204a-c are shown in this example, it is understood that in some other examples the number of applications (or apps) implemented on the device 102a may be more or less than the three. As shown, an individual application, such as application 204a, is configured to generate the notification information on its own, independent from the message service component 202 mentioned above. For example, application 204a may be configured to generate its own action within application 204a. However, it is also understood that the application 204a may be configured to receive data from cloud 106, for example, to implement a service provided by a third party system, such as system 110a shown in FIG. 1. It is understood the notification information generated by application 204a may be separate and independent from the notification information generated by the message service component 202. In some instances, the application 204a may be configured to work with message service component 202 to generate only one piece of notification information. However, it is not necessarily the only case. In some instances, it is contemplated that the application 204a generates notification information different from the notification information generated by the message service component 202.

[0032] The system user interface presentation component 206 is configured to present a notification on the device 102a to the user(s) 104. As shown in FIG. 2, in this example, the system user interface presentation component 206 is operatively connected to notification manager component 208. The system user interface presentation component 206 is configured to receive the notification presentation information generated by the notification manager component 208 and present it on the device 102a. This may involve generating graphical representation of the notification to the user(s) 104. Thus, for achieving this, the system user interface presentation component 206 is configured with knowledge for implementing the presentation of the notification, such as graphical code libraries to implement the presentation of the notification. It should be noted, in this example, the system user interface presentation component 206 is not operatively connected to the individual applications 204a-c or message service component 202. As will be described, this design to facilitate “hiding” distributed notification mechanism from the system user interface presentation component 206. From system user interface presentation component 206’ s point of view, whether the notification comes from the system user interface presentation component 206 residing in device 102a, or device 102b/c is not of a concern to the system user interface presentation component 206. As shown, the notification task may also be generated by device 102b/c (for example by a notification manager component 208 on device 102b/c).

[0033] The notification manager component 208 is configured to receive the notification information and generate notification task for notification presentation on one of devices 102a-c. As shown in this example, the notification manager component 208 includes a distributed notification module 2082, which is configured to generate the notification task based on the notification information. FIG. 3 illustrates an example of the distributed notification module 2082 shown in FIG. 2 in accordance with the present disclosure. Attention is now is directed to FIG. 3, which will be described with references to FIGS. 1-2.

[0034] As shown in FIG. 3, the example of distributed notification module 2082 is implemented in a processor 302 configured to execute program components. In this example, the distributed notification module 2082 includes a notification routing module 304, a content transformation module 306, a task handler 308, an action handler 310, a cross-device communication module 312, and/or any other modules.

[0035] The notification routing module 304 is configured to determine a device in the multidevice environment, such as the one shown in FIG. 1, where the device 102a resides for presenting a notification in accordance with the notification information. As mentioned above, an insight provided by the present disclosure is that the notification in accordance with the notification information is not necessarily to be presented on a device where the notification information is received. In the example shown in FIG. 1, for instance, the notification in accordance with the notification information is determined by the notification routing module 304 to be presented on device 102c, rather than device on 102a. As will be described below, such a determination by the notification routing module 304, involves obtaining one or more information pieces and based on the obtained information piece(s), the notification routing module 304 determines it is the device 102c to present notification in accordance with the notification information.

[0036] The notification routing module 304 is configured to determine a device among the devices in the multi-device environment, such as devices 102a-c. In accordance with the present disclosure is that an individual notification is to be presented to the user(s) 104 on a device that has the best opportunity to notify the user(s) in the multi-device environment. As mentioned, one or more devices in the multi-device environment 100 may receive the notification from cloud 106 or a 3^rd party system 110, but it may be determined that notification is not presented to the user on the notification receiving device(s), but rather on one or more another device in the multi-device environment 100 because that device or those devices may have a better opportunity to present the notification to the user(s) 104 to have the user(s) 104 take an action in response to the notification.

[0037] As illustration, a notification of chat message may be first received by the device 102a from cloud 106. The device 102a, via its notification routing module 304, may determine that the device to present the notification to the user(s) 104 is device 102b, rather than device 102a. For instance, the device 102a determines that the device 102b is the device to present the notification to the user(s) 104 instead of the device 102a. As will be described below, the notification routing module 304 of the device 102a is configured to route the chat notification to device 102b after this determination is made.

[0038] In various implementation, the notification routing module 304 is configured to generate a notification task after a device is determined for presenting a notification. The notification task is to be processed by that device to present the notification. As mentioned, an insight in accordance with the present disclosure is distributed notification presentation in the multi-device environment. For individual devices in this environment, a design consideration for the distributed notification is that the individual devices should not have the knowledge on which device the notification is generated when the notification is presented by the individual devices. As illustration, in this case, the notification is received by device 102a and is to be presented on device 102b. The device 102b does not concern the notification is first received by device 102a. For presentation purpose, the device 102b may treat the notification as if it were first received by device 102b. This may be referred to notification presentation transparency in the distributed notification mechanism in accordance with the present disclosure. In this way, the individual devices in the multi-device environment 100 can come and go dynamically such that no additional configuration is to be taken for them to participate in the distributed notification in the multi-device environment. In this sense, the distributed notification in accordance with the present disclosure achieves a dynamicity not achieved by conventional technologies. Traditionally, a device (such as a smart watch) is configured to be associated with a smart phone in order for that device to receive notification through the smart phone. In accordance with the present disclosure, the smart watch does not have to be configured as being associated with the smart phone in order for it to receive the notification. As described and illustrated, in accordance with the present disclosure, the smart phone determines which device to present the notification, and that device does not have to necessarily be associated with the smart phone.

[0039] A notification task is one way of implementing such presentation transparency. FIG. 4 illustrates an example of a notification task for implementing the distributed notification in accordance with the disclosure. As shown, in this example, a given notification task generated by notification routing module 304 includes a destination device ID, a source device ID, a notification ID, a notification content and presentation method, and/or any other information pieces. The destination device ID indicates a device where the notification is to be presented in the multidevice environment. For example, the destination device ID indicates an ID for device 102b. The source device ID indicates a device where the notification is received and/or processed in the multi-device environment. For example, the source device ID indicates an ID for device 102a. The notification ID indicates an identification for the notification, which may be assigned by a service or application the notification pertains to or may be assigned within the multi-device environment. As mentioned, the notification ID is used in the multi-device environment to identify a particular notification to ensure the notification is not repeatedly processed or presented within the multidevice environment. The notification content and presentation method indicates one or more contents of the notification and corresponding method(s) for presenting the content(s) of the notification. In accordance with the present disclosure, a given notification does not necessarily have one content and one presentation method for presenting the content. In accordance with the present disclosure, the given notification can have more than one type of contents and corresponding presentation method.

[0040] FIG. 5 illustrates one example of multiple types of contents corresponding to different presentation methods are present in a notification in accordance with the present disclosure. As shown, in this example, the notification 500 has five type contents including a small icon, information indicating an app name for the notification, a title for the notification, a first action (Action 1 - for example, reply “yes”) that can be acted on, and a second action (Action 2 - example, reply “no”) that can be acted on. As can be seen, the small icon may be presented visually (show), the app name may be presented audibly (play), the title may be presented audibly, action 1 may be presented visually, and action 2 may be presented visually. As mentioned, such information may be captured in notification content and presentation method field in the notification task 400 generated by notification routing module 304. As will be described below, this notification task is to be processed by the device (for example, device 102b in this case) in the multi-device environment to present the notification.

[0041] The content transformation module 306 is configured to transform a content of the notification to be presented on a device in the multi-device environment. In implementation, after a notification task, such as the notification task 400 shown in FIG. 4, is generated by the notification routing module 304, the content transformation module 306 can be configured to determine whether the notification presenting device (e.g., device 102b) can present the notification in accordance with the notification task. One way of handling this is that the content transformation module 306 can be configured to obtain a presentation capability of device 102b because it is indicated by the notification task as the destination for the notification task. The presentation capability of devices in the multi-device environment 100 may be centrally managed and updated as the devices come and go in the multi-device environment 100. One skilled in the art would understand there are a number of ways for managing presentation capability information for devices in the multi-device environment 100.

[0042] Based on the presentation capability of the presentation device, content transformation module 306 can be configured to determine whether all of the contents (data) in the notification tasks can be presented by the presentation device. For example, if the device 102b is a smart speaker and is determined as the presentation device by notification routing module 304 on device 102a, the content transformation module 306 on device 102a determines some contents of the notification is not capable of being presented by device 102b. For instance, if the notification is a chat message notification generated by a chat service provided by a third party provider, and includes a preview of a video message sent to the user(s) 104, the device 102b is not able to present the preview video graphically because it is merely a smart speaker.

[0043] As mentioned, for addressing this challenge in the multi-device environment 100, the content transformation module 306 on device 102a can be configured to determine whether the device 102b can present the notification and in the case it is determined that device 102b cannot present the notification, it can be configured to transform the content of the notification (for example, in the notification task) to a format presentable on device 102b. In this case, the content transformation module 306 can be configured to transform the preview video in the chat message notification to an audio format - in a form of announcing to the user(s) 104 that a video message has been received in the chat service. In implementation, the content transformation module 306 can be configured to modify the notification task by replacing the original content of the notification (e.g., preview of the video message) with the transformed content (e.g., the aforementioned audio format).

[0044] The task handler 308 is configured to process a notification task. This may involve facilitating a presentation device to present the notification . As mentioned, the notification routing module 304 may determine another device in the DAE should be the presentation device. However, as also mentioned, the presentation device may not have a capability to process a user input after the notification is presented on the presentation device.

[0045] By way of illustration, not intended to be limiting, the notification routing module 304 may determine that the notification should presented on device 102b, which is a smart speaker, rather than on the device 102a, which is a smart phone. In that example, the presentation device 102b has a limited capability of processing user input. For example, if the notification is a chat message, the notification may allow the user(s) 104 to respond to the message by using a “canned” response such as “got it” selected from a list of stored canned responses by the user. This involves processing the user input to determine that “got it” is selected by the user as the response and sending this response to the third-party system 110a to be transmitted to the other party in the chat with the user. In this example, the smart speaker does not have the capability for such processing. However, the smart speaker 102b may have the capability to prompt the user to provide a response using speech. In this example, the notification task generated by notification routing module 304 can include an instruction indicating to the presentation device 102b that it is to receive a speech input from the user and forwards the speech input to device 102a for processing. Thus, after the notification is presented by the presentation device 102b, a prompt is played on the presentation device 102b to solicit a response from the user, who may then provide a speech indicating “got it”. After receiving this response from the user, the presentation device 102b then forwards the response to the device 102a for processing, which may involve, as mentioned, transforming the speech response to a text and transmitting the text to the third-party system 110a for communication to the other party in the chat with the user.

[0046] Response handler 310 is configured to process a user input provided by a user and determine where the user input may be processed and how to be processed. One insight provided by the present disclosure is that user input in response to a notification may be classified into at least two categories: 1) response input and 2) action input. A response input may be referred to inputs that provide a response to the notification, which may cause an action to be initiated in an app on behalf of the user. For example, the user’s response “got it” to a chat message is a response input to a notification about the chat message to cause the chat app to initiate an action of sending “got it” the text to the other party in the chat. An action input is distinguished from the response input in that it corresponds to an action selected by the user to be executed in response to the notification. For example, the user may select to dismiss the chat message in response to the notification or may select to simply mark the message read or unread as an action response. In that sense, a response input triggers an action to be executed using the user input for the action, and an action response triggers an action without user input.

[0047] One insight provided by the present disclosure is that a presentation device in the DAE may be capable of handling an input response and/or action response after the presentation of the notification. For example, if the presentation device is a smart speaker, it is capable of handling an action response to dismiss a chat message, but not capable of handling an input response to send “got it” to the other party in the chat. Such an input response involves transmitting “got it” to that party via the third-party server 110, which may in turn involves resolving various communication addresses in this transmission processes. In that example, the smart speaker relies on the notification forwarding device - for example a smart phone to handling such an input response.

[0048] FIG. 6 illustrates two examples of response handler 310 in accordance with the present disclosure. As can be seen, the response handler 310 installed on the device 102a comprises an action proxy 3102, a user input proxy 3104, a remote app handler 3106, and/or any other components. The response handler 310 installed on the device 102b comprises a remote app handler 3106, an action proxy 3102, a user input proxy 3104 and/or any other components. In this example, the device 102b is a smart speaker and the device 102a is a smart phone. As can be seen, the device 102a generates a notification task because it determines that a notification should be presented on device 102b rather than device 102a in the DAE. After presenting the notification in accordance with the notification task, the device 102b receives user input data or action in response to the notification. In this example, the device 102b does not have capabilities to handle either the action or the user input data response. Thus, the remote app handler 3106 in the response handler 310 on the device 102b is configured to forward the user action response and/or input data response to the device 102b for execution/processing.

[0049] As can be seen, the response handler 310 on the device 102a receives the user action or input data response from the device 102b. The remote app handler 3106 in the response handler 310 on the device 102a is configured to determine whether the response from the user is an action response or an input response. If the response is an action response, the response is forwarded to the action proxy 3102 for further processing, which may involve sending the action response to the app for execution. If the response is a user input data response, the response is forward to the user input proxy 3104 for further processing, which may involve processing the user input data (for example, parsing the data) and determining one or more actions to be executed in the app based on the user input data.

[0050] In some implementation, the response handler 310 is configured to determine whether a response provided by the user is an action response or an input response; and determine whether the response is to be handled on the device locally or to be processed remotely in accordance with the task instruction received by a task handler. For example, the response handler 310 on the device 102a shown in FIG. 6 may be configured to receive a user response after a notification is presented on the device 102a. In that case, the action proxy 3102 is configured to process the action response and the user input proxy 3104 is configured to process the user input response. In that case, because the device 102a can handle the action response and/or the user input response, the remote app handler 3106 is not involved to forward the user response to another device in the DAE for processing. Thus, for a given device that is capable of handling action response and/or user input data response, the response handler 310 on the given device is configured to cause the given device to handle the user response directly instead of forward the user response to another device in the DAE for processing.

[0051] FIG. 7 illustrates an example flow for processing a response received by a device after a notification has been presented on the device in the DAE. The operations of method 700 presented below are intended to be illustrative. In some embodiments, method 700 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order in which the operations of method 700 are illustrated in FIG. 7 and described below is not intended to be limiting.

[0052] In some embodiments, method 700 may be implemented in any one of devices 102a, 102b and 102c shown in FIG. 1, which may each include one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information). The one or more processing devices may include one or more devices executing some or all of the operations of method 700 in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured through hardware, firmware, and/or software to be designed for execution of one or more of the operations of method 700.

[0053] At 702, a response is received. The response may be provided by a user after a notification is presented on a device in the DAE. As described above, the response may include an action response and/or user input data response provided by the user. In some implementations, operations at 702 may be performed by a response handler the same as or substantially similar to the response handler 310 described and illustrated herein.

[0054] At 704, a determination is made to determine whether the response received at 702 includes an action response or a user input data response. In some implementations, operations at 702 may be performed by a response handler the same as or substantially similar to the response handler 310 described and illustrated herein. As can be seen, if the response is determined to be an action response, the flow proceeds to determination 706, and if the response is determined to be a user input data response, the flow proceeds to determination 708. In some implementations, operations at 704 may be performed by a response handler the same as or substantially similar to the response handler 310 described and illustrated herein.

[0055] At 706, a determination is made as to whether the device that receives the response at 702 is capable of handling the action response locally on the device. As mentioned, some device(s) in the DAE may not be capable of handling an action response, for example a smart speaker. As can be seen, if the determination at 706 is yes, the flow proceeds to 714, where the action response is processed locally on the device. In some implementations, operations at 706 may be performed by an action proxy the same as or substantially similar to the action proxy 3102 described and illustrated herein. [0056] At 708, a determination is made as to whether the device that receives the response at 702 is capable of handling the user input data response locally on the device. As mentioned, some device(s) in the DAE may not be capable of handling a user input data response, for example a smart speaker. As can be seen, if the determination at 708 is yes, the flow proceeds to 716 where the user input data response is processed locally on the device. In some implementations, operations at 708 may be performed by a user input proxy the same as or substantially similar to the user input proxy 3104 described and illustrated herein

[0057] As can be seen, if the determination at 706 or 708 is no, the flow proceeds to 710. At 710, a notification task is obtained. As described, the notification instruction is transmitted from a device in the DAE where the notification is forwarded from. The notification instruction includes information indicating where a particular type of response (e.g., action and/or user input data response) is to be processed. In some implementations, operations at 706 may be performed by a response handler the same as or substantially similar to the response handler 310 described and illustrated herein.

[0058] At 712, the response is transmitted to another device in the DAE for processing according to the notification instruction obtained at 710. In some implementations, operations at 712 may be performed by a response handler the same as or substantially similar to the response handler 310 described and illustrated herein.

[0059] Any of the computer systems mentioned herein may utilize any suitable number of subsystems. Examples of such subsystems are shown in FIG. 8 in computer system 10, which can be configured to implement various features and/or functions described herein. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be the components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components.

[0060] The subsystems shown in FIG. 8 are interconnected via a system bus 75. Additional subsystems such as a printer 74, keyboard 78, storage device(s) 79, monitor 76, which is coupled to display adapter 82, and others are shown. Peripherals and input/output (I/O) devices, which couple to I/O controller 71, can be connected to the computer system by any number of means known in the art such as input/output (I/O) port 77 (e.g., USB, FireWire®). For example, I/O port 77 or external interface 81 (e.g. Ethernet, Wi-Fi, etc.) can be used to connect computer system 10 to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus 75 allows the central processor 73 to communicate with each subsystem and to control the execution of instructions from system memory 72 or the storage device(s) 79 (e.g., a fixed disk, such as a hard drive or optical disk), as well as the exchange of information between subsystems. The system memory 72 and/or the storage device(s) 79 may embody a computer readable medium. Any of the data mentioned herein can be output from one component to another component and can be output to the user.

[0061] A computer system can include a plurality of the same components or subsystems, e.g., connected together by external interface 81 or by an internal interface. In some embodiments, computer systems, subsystem, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components.

[0062] It should be understood that any of the embodiments of the present invention can be implemented in the form of control logic using hardware (e.g. an application specific integrated circuit or field programmable gate array) and/or using computer software with a generally programmable processor in a modular or integrated manner. As used herein, a processor includes a single-core processor, multi-core processor on a same integrated chip, or multiple processing units on a single circuit board or networked. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and/or methods to implement embodiments of the present invention using hardware and a combination of hardware and software.

[0063] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer readable medium for storage and/or transmission, suitable media include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk), flash memory, and the like. The computer readable medium may be any combination of such storage or transmission devices.

[0064] Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and/or wireless networks conforming to a variety of protocols, including the Internet. As such, a computer readable medium according to an embodiment of the present invention may be created using a data signal encoded with such programs. Computer readable media encoded with the program code may be packaged with a compatible device or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g. a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.

[0065] Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps. Thus, embodiments can be directed to computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing respective steps or a respective group of steps. Although presented as numbered steps, steps of methods herein can be performed at a same time or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, any of the steps of any of the methods can be performed with modules, circuits, or other means for performing these steps.

[0066] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of embodiments of the invention. However, other embodiments of the invention may be directed to specific embodiments relating to each individual aspect, or specific combinations of these individual aspects.

[0067] The above description of exemplary embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the teaching above. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated.

[0068] A recitation of "a", "an" or "the" is intended to mean "one or more" unless specifically indicated to the contrary. The use of “or” is intended to mean an “inclusive or,” and not an “exclusive or” unless specifically indicated to the contrary.

[0069] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted to be prior art.

[0070] Having described several example configurations, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over or otherwise modify the application of the invention. Also, a number of steps may be undertaken before, during, or after the above elements are considered.

Claims

WHAT IS CLAIMED IS:

1 A method for distributing a notification across multiple devices, the multiple devices including a first device and a second device, and the method being implemented by one or more electronic processors configured to execute machine-readable instructions, wherein when the machine-readable instructions are executed, the one or more processors are caused to perform: receiving, at the first device, a notification task indicating a notification for presentation to a user associated with the first device; presenting, at the first device, the notification according to the notification task; receiving, at the first device, a user response to the notification; determining, at the first device, that the first device is not capable of handling the user response; and forwarding, at the first device, the user response to the second device processing according to the notification task.

2. The method of claim 1, wherein the user response is an action response or a user input response; and, wherein the method further comprises: determining that the user response is the user input response and that the first device is not capable of processing the user input response.

3. The method of claim 1, wherein the user response is an action response or a user input response; and, wherein the method further comprises: determining that the user response is the action response and that the first device is not capable of processing the action response.

4. The method of claim 1, further comprising receiving, at the first device, a second user response to the notification, determining that the first device is capable of the second user response, and processing the second user response at the first device.

5 . The device of claim 1, wherein the determination, at the first device, that the first device is not capable of handling the user response comprising determining that the first device is able to process at least one user input provided by the user in response to the notification being presented on the first device.

6. The method of claim 1, wherein the notification task specifies that the user response is to be processed at the second device.

7. The method of claim 1, wherein the user response causes an action to be performed to the presentation of the notification without user input data.

8. The method of claim 1, wherein the user response causes an action to be performed in an app with user input data included in the user response.

9. A system for distributing a notification across multiple devices, the multiple devices including a first device and a second device, and the system comprising one or more electronic processors configured to execute machine-readable instructions, wherein when the machine-readable instructions are executed, the one or more processors are caused to perform: receiving, at the first device, notification task indicating a notification for presentation to a user associated with the first device; presenting, at the first device, the notification according to the notification task; receiving, at the first device, a user response to the notification; determining, at the first device, that the first device is not capable of handling the user response; and forwarding, at the first device, the user response to the second device processing according to the notification task.

10. The system of claim 9, wherein the user response is an action response or a user input response; and, wherein the one or more processors are further caused to perform: determining that the user response is the user input response and that the first device is not capable of processing the user input response.

11. The system of claim 9, wherein the user response is an action response or a user input response; and, wherein the method further comprises: determining that the user response is the action response and that the first device is not capable of processing the action response.

12. The system of claim 9, wherein the one or more processors are further caused to perform receiving, at the first device, a second user response to the notification, determining that the first device is capable of the second user response, and processing the second user response at the first device.

13. The system of claim 9, wherein the determination, at the first device, that the first device is not capable of handling the user response comprising determining that the first device is able to process at least one user input provided by the user in response to the notification being presented on the first device.

14. The system of claim 9, wherein the notification task specifies that the user response is to be processed at the second device.

15. The system of claim 9, wherein the user response causes an action to be performed to the presentation of the notification without user input data.

16. The system of claim 9, wherein the user response causes an action to be performed in an app with user input data included in the user response.