CN111135557B - Interaction method and device for multiple screens - Google Patents

Abstract

The embodiment of the invention discloses an interaction method and device for multiple screens. The method comprises the following steps: responding to receiving the selection operation of the user on the multi-screen interaction options, and projecting a current display interface to a target screen; presenting an option to select an input device; receiving a selection operation of an option of selecting an input device; and responding to the selection operation instruction, adopting a virtual input device, generating an operation interface comprising at least one virtual input option based on the current display interface projected to the target screen, and displaying the operation interface on the local screen and not displaying the current display interface projected to the target screen. The method can realize information sharing, reduce the heating and energy consumption of the electronic equipment where the local screen is located, and improve the accuracy of information input of the local operation interface.

Description

Interaction method and device for multiple screens

Technical field

The present disclosure relates to the field of computer technology, specifically to the field of terminal sharing technology, and in particular to an interactive method and device for multiple screens.

Background technique

In the existing technology, when users play games, play audio and video, and perform other interactive operations with the terminal, they can transmit the screen of the smart terminal device to the large-screen device through WIFI-P2P technology, and the large-screen device can The pictures or audio and video are decoded and presented.

However, in the existing technology, the human-computer interaction screen on the mobile phone is transmitted to the large-screen device through multi-screen sharing technology while human-computer interaction is performed. The mobile phone terminal device and the large-screen device play the human-computer interaction screen at the same time, and the virtual screen of the mobile phone The input options cannot produce a physical touch feeling. If the human-computer interaction personnel focus on the large-screen device, there will be more misoperations when using the virtual input options of the mobile phone. The mobile phone terminal is prone to heat and high power consumption.

Contents of the invention

Embodiments of the present disclosure provide interaction methods and devices for multiple screens.

In a first aspect, embodiments of the present disclosure provide an interaction method for multi-screens, including: in response to receiving a user's selection operation on a multi-screen interaction option, projecting the current display interface to the target screen; and presenting options for selecting an input device. ; Receive a selection operation for selecting an option of the input device; use the virtual input device in response to the selection operation instruction, generate an operation interface including at least one virtual input option based on the current display interface projected to the target screen, and display the operation interface on the local screen , do not display the current display interface projected to the target screen.

In some embodiments, in response to receiving the user's selection operation on the multi-screen interaction option, projecting the current display interface to the target screen includes: in response to receiving the local user's selection operation on the multi-screen interaction option, including the local user's interaction operation. Project the current display interface of the interactive operation with the remote user to the target screen.

In some embodiments, generating an operation interface including at least one virtual input option based on the current display interface projected to the target screen includes: using AI big data to predict the user portrait of the user in the current display interface projected to the target screen; based on the prediction The user portrait is used to generate an operation interface including at least one virtual input option.

In some embodiments, generating an operation interface including at least one virtual input option based on the current display interface projected to the target screen includes: generating an operation interface including a preset background and at least one virtual input option based on the current display interface projected to the target screen. The operation interface, wherein the preset background includes a default background or a background set by the user based on background setting options.

In some embodiments, virtual input options include: virtual interactive handles.

In some embodiments, the interaction method further includes: in response to the user's selection operation on the virtual input option, presenting vibrations with a vibration frequency corresponding to the selected virtual input option.

In some embodiments, the interaction method further includes: in response to the user's selected operation on the generated virtual input option, presenting a movable identification of the virtual input option; in response to the user's movement operation on the virtual input option having the movable identification, Move the virtual input option indicated by the move operation to the target location indicated by the move operation; deselect the virtual input option in response to a user input exit operation for the selected operation.

In some embodiments, the interaction method further includes: in response to the selection operation instruction, using the physical input device, based on the preset corresponding relationship between the local physical input device and the current display interface, initiating the use of the local physical input device, and turning the local screen black.

In some embodiments, initiating the use of the local entity input device includes popping up an entity interaction handle.

In some embodiments, the target screen is larger in size than the local screen.

In some embodiments, in response to receiving the user's selection operation on the multi-screen interaction option, projecting the current display interface to the target screen includes: in response to receiving the user's selection operation on the multi-screen interaction option, projecting the current display interface to the target screen via an adapter of the external device in the TV. The current display interface is projected to the TV screen.

In a second aspect, embodiments of the present disclosure provide an interactive device for multiple screens, including: a display interface projection unit configured to project the current display interface to a target in response to receiving a user's selection operation on a multi-screen interactive option. a screen; a selection option presentation unit configured to present options for selecting an input device; a selection operation receiving unit configured to receive a selection operation for an option for selecting an input device; and an operation interface display unit configured to use a selection method in response to a selection operation instruction. The virtual input device generates an operation interface including at least one virtual input option based on the current display interface projected to the target screen, and displays the operation interface on the local screen without displaying the current display interface projected to the target screen.

In some embodiments, the display interface projection unit is further configured to: in response to receiving the local user's selection operation on the multi-screen interaction option, project the current display interface including the local user's interactive operation and the remote user's interactive operation to the target screen. .

In some embodiments, the operation interface display unit is further configured to: use AI big data to predict the user's user portrait of the current display interface projected to the target screen; and generate an operation including at least one virtual input option based on the predicted user portrait. interface.

In some embodiments, the operation interface display unit is further configured to: generate an operation interface including a preset background and at least one virtual input option based on the current display interface projected to the target screen, wherein the preset background includes a default background or a user The background is set based on the background settings option.

In some embodiments, the virtual input options in the operation interface display unit include: virtual interactive handles.

In some embodiments, the interactive device further includes: an option vibration presentation unit configured to, in response to the user's selection operation on the virtual input option, present a vibration with a vibration frequency corresponding to the selected virtual input option.

In some embodiments, the interactive device further includes: a mobile logo presentation unit configured to present a movable logo of the virtual input option in response to the user's selected operation on the generated virtual input option; the input option mobile unit configured to In response to the user's movement operation on the virtual input option having the movable identification, the virtual input option indicated by the movement operation is moved to the target position indicated by the movement operation; the option selection release unit is configured to respond to the user input selection Exit operation for a given operation and deselect the virtual input option.

In some embodiments, the interactive device further includes: an input device activating unit configured to use the physical input device in response to the selection operation instruction, and start using the local entity input based on the preset corresponding relationship between the local entity input device and the current display interface. Install and blacken the local screen.

In some embodiments, the input device initiating unit is further configured to: pop up the physical interaction handle.

In some embodiments, the size of the target screen in the display interface projection unit is larger than the local screen.

In some embodiments, the display interface projection unit is further configured to: in response to receiving the user's selection operation on the multi-screen interaction option, project the current display interface to the TV screen via the adapter of the external device in the TV.

In a third aspect, embodiments of the present disclosure provide an electronic device/terminal/server, including: one or more processors; a storage device for storing one or more programs; when one or more programs are processed by one or more Execution by multiple processors causes one or more processors to implement any of the above interactive methods for multi-screens.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the interactive method for multiple screens as described above is implemented.

The interactive method and device for multi-screen provided by the embodiments of the present disclosure first respond to receiving the user's selection operation of the multi-screen interactive option, project the current display interface to the target screen; present the option of selecting the input device; receive the selection input Selection operation of options of the device; using a virtual input device in response to the selection operation instruction, generating an operation interface including at least one virtual input option based on the current display interface projected to the target screen, and displaying the operation interface on the local screen without displaying the projection to The current display interface of the target screen. In this process, the local current display interface can be projected to the target screen to realize information sharing, and the local screen can be used to display the operation interface including at least one virtual input option without displaying the projected current display interface, which reduces the local screen cost. The heat and energy consumption of the electronic equipment where it is located improves the accuracy of input information on the local operation interface.

In some embodiments, the electronic device can emit vibrations of different frequencies to remind the user according to the selected virtual button, thereby enhancing the accuracy of the user's operation.

In some embodiments, AI big data is used to predict the user's portrait on the current display interface, and the operation interface is generated based on the user's portrait, which improves the pertinence of the operation interface and makes it more in line with the user's operating habits.

In some embodiments, a local entity interactive handle is used to operate the current display interface projected to the target screen, which improves the accuracy and efficiency of input information.

Description of the drawings

Other features, objects and advantages of the present disclosure will become more apparent upon reading the detailed description of the non-limiting embodiments with reference to the following drawings:

Figure 1 is an exemplary system architecture diagram in which the present disclosure may be applied;

Figure 2a is a schematic flowchart of an embodiment of an interaction method for multiple screens according to an embodiment of the present disclosure;

Figure 2b exemplarily shows a schematic diagram of receiving a single user's selection operation on a multi-screen interaction option and projecting the current display interface to the TV screen;

Figure 2c exemplarily shows a schematic diagram of receiving a single user's selection operation on a multi-screen interaction option and projecting the current display interface interacting with multiple users to a TV screen;

Figure 2d exemplarily shows a schematic diagram of the operation interface generated when the application to which the current display interface belongs is the first application;

Figure 2e exemplarily shows a schematic diagram of an operation interface with a background generated when the application to which the current display interface belongs is a second application;

Figure 2f exemplarily shows a schematic diagram of the operation interface generated when the application to which the current display interface belongs is a third application;

Figure 2g exemplarily shows a schematic diagram showing different vibration frequencies for the operation interface in Figure 2d;

Figure 2h exemplarily shows a schematic diagram showing different vibration frequencies for the operation interface in Figure 2f;

Figure 3 is an exemplary application scenario of an interaction method for multiple screens according to an embodiment of the present disclosure;

Figure 4a is a schematic flowchart of yet another embodiment of an interaction method for multiple screens according to an embodiment of the present disclosure;

Figure 4b exemplarily shows a schematic diagram in which the actual input options in the actual interactive handle are in raised mode;

Figure 4c exemplarily shows a schematic diagram of the actual input option in the actual interactive handle being in concave mode;

Figure 4d exemplarily shows a schematic diagram in which the actual input option in the actual interactive handle is a planar sensing mode;

Figure 5 is an exemplary structural diagram of an embodiment of an interactive device for multi-screens of the present disclosure;

FIG. 6 is a schematic structural diagram of a computer system suitable for implementing a server according to an embodiment of the present disclosure.

Detailed ways

The present disclosure will be further described in detail below in conjunction with the accompanying drawings and examples. It can be understood that the specific embodiments described here are only used to explain the relevant invention, but not to limit the invention. It should also be noted that, for convenience of description, only the parts related to the invention are shown in the drawings.

It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other. The present disclosure will be described in detail below in conjunction with embodiments with reference to the accompanying drawings.

FIG. 1 shows an exemplary system architecture 100 to which embodiments of the interactive method for multiple screens or the interactive device for multiple screens of the present disclosure may be applied.

As shown in Figure 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium used to provide communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

Users can use terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, and 103, such as game applications, browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

The terminal devices 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices that support browser applications, including but not limited to tablet computers, laptop computers, desktop computers, and so on. When the terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. It may be implemented, for example, as multiple software or software modules for providing distributed services, or as a single software or software module. There are no specific limitations here.

The server 105 may be a server that provides various services, such as a background server that provides support for applications performed on the terminal devices 101, 102, and 103. The background server can analyze and process the received request and other data, and feed the processing results back to the terminal device.

It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it may be implemented as, for example, multiple software or software modules used to provide distributed services, or it may be implemented as a single software or software module. There are no specific limitations here.

In practice, the interactive method for multi-screen provided by the embodiment of the present disclosure can be executed by the terminal devices 101, 102, and 103, and the interactive device for multi-screen can also be provided in the terminal device 101, 102, and 103.

It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Depending on implementation needs, there can be any number of end devices, networks, and servers.

Continuing to refer to FIG. 2a , FIG. 2a illustrates a process 200 of one embodiment of an interaction method for multiple screens according to the present disclosure. The interactive method for multiple screens includes the following steps:

In step 201, in response to receiving the user's selection operation on the multi-screen interaction option, the current display interface is projected to the target screen.

In this embodiment, the execution subject of the multi-screen interaction method (for example, the terminal device shown in FIG. 1) can receive the user's selection operation of the multi-screen interaction option through the input device.

Among them, the multi-screen interaction option refers to the option of projecting the current display interface of the software running on the local screen to other screens.

The selection operation can be a preset operation that can be recognized as a selection action.

For example, the terminal device can collect the user's click operation, and if the operation position of the click operation corresponds to the multi-screen interaction option, then the click operation is recognized as a selection operation of the multi-screen interaction option.

For another example, the terminal device can collect the user's operation trajectory and identify whether the collected operation trajectory matches the preset operation trajectory. If they match, the collected operation trajectory will be used as the selected operation of the multi-screen interaction option. The preset operation track is a preset selected operation corresponding to the multi-screen interaction option.

The target screen is the screen indicated by the multi-screen interaction option and has a projection relationship with the local screen. The size of the target screen can be greater than, equal to, or smaller than the size of the local screen. This application does not limit this.

When the execution subject projects the current display interface to the target screen, the user terminal that interacts with the current display interface may be a single user terminal or multiple user terminals.

In some optional implementations of this embodiment, in response to receiving the user's selection operation of the multi-screen interaction option, projecting the current display interface to the target screen includes: in response to receiving the local user's selection operation of the multi-screen interaction option, projecting the current display interface to the target screen. The current display interface including the local user's interactive operations and the remote user's interactive operations is projected to the target screen.

In this implementation, if there are multiple user terminals that interact with the current display interface, then the above execution subject can be used as a terminal device that transmits the current display interface to the target screen, and as a terminal that interacts with the current display interface. One of the devices, and other terminal devices can be used as other terminal devices to interact with the current display interface, thereby improving the adaptability to simultaneous interaction by multiple users.

Specifically, when the user selects the multi-screen interaction option, the above-mentioned execution subject can determine whether the current operation interface supports simultaneous interaction with multiple terminals. If supported, a many-to-one control mode can be provided to allow the terminal device currently selecting the multi-screen interaction option. As screen output and console output, other terminal devices serve as console output.

In some optional implementations, in response to receiving the user's selection operation of the multi-screen interaction option, projecting the current display interface to the target screen may include: in response to receiving the user's selection operation of the multi-screen interaction option, via an external connection in the TV The device's adapter projects the current display to the TV screen.

In this implementation, by using the TV screen as the target screen, the display clarity of the current display interface can be improved, thereby improving interaction efficiency.

As shown in FIG. 2 b , FIG. 2 b exemplarily illustrates a situation in which a single user's selection operation on a multi-screen interactive option is received and the current display interface is projected onto the TV screen. The operator 210 projects the current display interface to the TV screen and displays the operation interface including virtual input options on the terminal device. The onlookers 211 and 212 watch the operator 210's interactive operations on the TV screen.

As shown in FIG. 2c , FIG. 2c exemplarily illustrates the situation of receiving a single user's selection operation on a multi-screen interaction option and projecting the current display interface interacting with multiple users to the TV screen. Among them, the operator 220 projects the current display interface to the TV screen and displays the operation interface including virtual input options on the terminal device. The interactive operations of other operators 221 and 222 with the current interactive interface are also on the current interactive interface on the TV screen. show.

Returning to Figure 2a, in step 202, an option is presented to select an input device.

In this embodiment, when the execution subject projects the current display interface to the target screen, the option of selecting the input device may be presented on the local screen and/or the target screen. For example, an option to select a virtual input device and an option to select a physical input device are presented.

In step 203, a selection operation for selecting an option of the input device is received.

In this embodiment, when the above execution body presents the options of the input device, the user's selection operation on the options of the input device can be received.

In step 204, in response to the selection operation instruction, a virtual input device is used to generate an operation interface including at least one virtual input option based on the current display interface projected to the target screen, and the operation interface is displayed on the local screen and is not displayed on the target screen. the current display interface.

In this embodiment, if the selection operation instruction uses a virtual input device, then the execution subject can generate an operation interface consistent with the current display interface projected to the target screen. The operating interface may include one, two or more virtual input options. The virtual input option may be in the form of a virtual interactive button or some special virtual input option, such as a virtual interactive handle.

When generating the operation interface, the above execution subject can generate different operation interfaces according to different current display interfaces. Or, different operation interfaces are generated according to different applications to which the current display interface belongs or different interaction stages of the same application. Specifically, the above-mentioned way for the execution subject to generate the operation interface can be determined by the way of generating the interface in the existing technology or in the technology developed in the future, and this application is not limited to this. For example, the operation interface can be generated based on the association between the preset operation interface and the current display interface.

After generating the operation interface, the above execution subject displays the operation interface on the local screen and does not display the current display interface projected to the target screen on the local screen. At this time, the local screen serves as a virtual input device, providing the user with an operation interface for inputting information. For example, when the current interactive interface is a game interface, the above execution subject can generate different button layouts according to different games.

Exemplarily, Figure 2d, Figure 2e and Figure 2f respectively show the situation in which different operation interfaces are generated in the virtual interaction device for different applications to which the current display interface belongs. Among them, FIG. 2d shows the situation of the operation interface generated when the application to which the current display interface belongs is the first application (for example, the game GOK of the Kingdom). Figure 2e shows a situation in which an operation interface with a background is generated when the application to which the current display interface belongs is a second application (for example, Naruto). FIG. 2f shows the operation interface generated when the application to which the current display interface belongs is a third application (for example, QQ Speed QQspeed).

In some optional implementations of this embodiment, the above-mentioned generation of an operation interface including at least one virtual input option based on the current display interface projected to the target screen may include: using AI big data to predict the user's current display interface projected to the target screen. Display a user portrait of the interface; and generate an operation interface including at least one virtual input option based on the predicted user portrait.

In this implementation, the above-mentioned execution subject can use artificial intelligence (AI) big data locally or in the cloud to predict the user's user portrait of the current display interface projected to the target screen when the user uses the virtual input device. The user portrait here is an effective tool for describing target users and connecting user demands and design direction. For example, the user portrait may include the current user's age, gender, interests, hobbies, professionalism, personality traits, etc., and may also include the layout of virtual input options in the virtual input device corresponding to the current interactive interface for the user category to which the current user belongs. The layout of the current user's virtual input options in the virtual input devices corresponding to different interactive interfaces.

After predicting the user portrait, the execution subject can generate an operation interface that conforms to the user portrait, conforms to the current display interface, and includes at least one virtual input option based on the predicted user portrait.

In this implementation, the operation interface including at least one virtual input option is generated based on the predicted user portrait, which can improve the pertinence of the generated operation interface and make it more in line with the needs of the user.

In some optional implementations of this embodiment, the above-mentioned generation of an operation interface including at least one virtual input option based on the current display interface projected to the target screen may include: based on the current display interface projected to the target screen, generating including a preset Background, an operation interface of at least one virtual input option, wherein the preset background includes a default background or a background set by the user based on the background setting options.

In this implementation, by introducing a default background or a background set by the user based on the background setting options into the operation interface, the pertinence of the operation interface can be improved, making the virtual input options clearer, clearer, or more in line with user needs.

In some optional implementations of this embodiment, the interaction method further includes: in response to the user's selection operation on the virtual input option, presenting vibrations with a vibration frequency corresponding to the selected virtual input option.

In this implementation, when the user selects one or more options of the virtual input options, the above execution subject can present different vibration frequencies for different options to prompt the user that the option corresponding to the vibration frequency is selected, thereby Improved visibility of virtual input options.

Please refer to Figures 2g and 2h. Figure 2g exemplarily shows a situation in which different vibration frequencies 231, 232 are presented for the operation interface in Figure 2d. Figure 2h exemplarily shows a situation in which different vibration frequencies 233, 234 are presented for the operation interface in Figure 2f.

In some optional implementations of this embodiment, the interaction method further includes: in response to the user's selected operation on the generated virtual input option, presenting a movable identification of the virtual input option; in response to the user's selection of the virtual input option with a movable identification. The movement operation of the input option moves the virtual input option indicated by the movement operation to the target position indicated by the movement operation.

In this implementation, the selection operation is a predefined operation for selecting a virtual input option; the move operation is a predefined operation for moving a virtual input option with a movable identifier.

When the user selects a virtual input option, the local screen of the above-mentioned execution subject may present movable identifiers of one or all virtual input options. At this time, the virtual input options with movable identifiers are in a movable state. When the user performs a moving operation on a virtual input option with a movable flag, the virtual input option indicated by the moving operation can be moved to a target position indicated by the moving operation.

The interaction method in this implementation can move the virtual input options to the target position according to the user's needs, thereby improving the flexibility of the layout of the virtual input options in the virtual input device.

In some optional implementations of this embodiment, the interaction method further includes: deselecting the virtual input option in response to a user input exit operation for the selected operation.

In this implementation, if the user selects a virtual input option due to misoperation and a movable logo of one or all virtual input options is presented, the user can enter an exit operation for the selected operation to deselect the virtual input option. The exit operation here is a predefined operation used to deselect the virtual input option.

The interactive method for multiple screens in the above embodiments of the present disclosure can project the current local display interface to the target screen to realize information sharing, and use the local screen to display the operation interface including at least one virtual input option, without displaying the projected The current display interface reduces the heat and energy consumption of the electronic device where the local screen is located, and improves the accuracy of input information on the local operation interface.

The following describes an exemplary application scenario of the interactive method for multiple screens of the present disclosure with reference to FIG. 3 .

As shown in FIG. 3 , FIG. 3 shows an exemplary application scenario of the interaction method for multiple screens according to the present disclosure.

As shown in Figure 3, the interaction method 300 for multi-screen runs in the electronic device 320 and may include:

First, in response to receiving the user's selection operation 302 on the multi-screen interaction option 301, project the current display interface 303 to the target screen 304;

Thereafter, an option to select an input device is presented 305;

Thereafter, receiving a selection operation 306 for selecting an option 305 of the input device;

Afterwards, in response to the selection operation 306 indicating that the virtual input device 307 is used, based on the current display interface 303 projected to the target screen 304, an operation interface 309 including at least one virtual input option 308 is generated, and the operation interface 309 is displayed on the local screen 310. The current display interface 303 projected to the target screen 304 is displayed.

It should be understood that the application scenario of the interactive method for multiple screens shown in FIG. 3 is only an exemplary description of the interactive method for multiple screens and does not represent a limitation of the method. For example, each step shown in Figure 3 above can be further implemented in more detailed ways. It is also possible to further add interactive steps for multiple screens on the basis of Figure 3 above.

Referring further to FIG. 4a , FIG. 4a shows a schematic flowchart of one embodiment of a method for uploading files in a multi-screen interactive method according to the present disclosure.

As shown in Figure 4a, the method 400 for uploading files in the multi-screen interaction method of this embodiment may include the following steps:

Step 401: In response to receiving the user's selection operation on the multi-screen interaction option, project the current display interface to the target screen.

In this embodiment, the execution subject of the multi-screen interaction method (for example, the terminal shown in FIG. 1 ) can receive the user's selection operation of the multi-screen interaction option through the input device.

Among them, the multi-screen interaction option refers to the option of projecting the current display interface of the local screen to other screens. The selection operation may be a preset operation that can be recognized as a selection action. The target screen is the screen indicated by the multi-screen interaction option and has a projection relationship with the local screen.

Step 402: An option to select an input device is presented.

In this embodiment, when the execution subject projects the current display interface to the target screen, the option of selecting the input device may be presented on the local screen and/or the target screen. For example, an option to select a virtual input device and an option to select a physical input device are presented.

Step 403: Receive a selection operation for selecting an option of the input device.

In this embodiment, when the above execution body presents the options of the input device, the user's selection operation on the options of the input device can be received.

Step 404: In response to the selection operation instruction to use the physical input device, based on the preset corresponding relationship between the local physical input device and the application software or system software to which the current display interface belongs, start using the local physical input device and blacken the local screen.

In this embodiment, the physical input device may be a physical button provided on the screen or casing of the electronic device. When the physical button is turned on, the electronic device is completely used as a physical interactive handle. At the same time, the design of the physical interactive handle can have concave, convex or flat sensing modes.

Exemplarily, FIG. 4 b , FIG. 4 c and FIG. 4 d respectively show the situation in which the actual input options in the actual interactive handle are convex mode, concave mode and flat sensing mode respectively.

If the operation instruction is selected to use a physical input device, then the execution subject can activate the use of the local physical input device according to the preset corresponding relationship between the local physical input device and the current display interface, so as to improve the accuracy and precision of the input. Furthermore, the above-mentioned execution subject can also black out the local screen to reduce the energy consumption of the above-mentioned execution subject.

In some optional implementations of this embodiment, the above-mentioned activation using a local entity input device includes popping up an entity interactive handle.

In this implementation, if the local entity input device is activated, the entity interaction handle can pop up to improve the user's interaction efficiency with the current display interface projected to the target screen.

The interactive method for multiple screens in the embodiment shown in Figure 4a of the present disclosure is different from the virtual input device in Figure 2a. In Figure 4a, a physical input device is used, which can improve the user's operating efficiency.

Those skilled in the art can understand that steps 401, 402 and 403 in the above-mentioned Figure 4a correspond to steps 201, 202 and 203 respectively. Therefore, the operations and features described for steps 201, 202 and 203 in the above-mentioned Figure 2a , the same applies to steps 401, 402 and 403, which will not be described again here.

As an implementation of the methods shown in the above figures, the embodiment of the present disclosure provides an embodiment of an interactive device for multiple screens. The device embodiment corresponds to the method embodiment shown in Figures 2a-4d. , this device can be specifically applied to the terminal or server shown in Figure 1 above.

As shown in FIG. 5 , the interactive device 500 for multi-screen in this embodiment may include: a display interface projection unit 510 configured to project the current display interface to the target in response to receiving the user's selection operation on the multi-screen interaction option. screen; the selection option presentation unit 520 is configured to present the option of selecting the input device; the selection operation receiving unit 530 is configured to receive the selection operation of the option of selecting the input device; the operation interface display unit 540 is configured to respond to the selection The operation instruction uses a virtual input device to generate an operation interface including at least one virtual input option based on the current display interface projected to the target screen, and displays the operation interface on the local screen without displaying the current display interface projected to the target screen.

In some embodiments, the display interface projection unit is further configured to: in response to receiving the local user's selection operation on the multi-screen interaction option, project the current display interface including the local user's interactive operation and the remote user's interactive operation to the target screen. .

In some embodiments, the operation interface display unit is further configured to: use AI big data to predict the user's user portrait of the current display interface projected to the target screen; and generate an operation including at least one virtual input option based on the predicted user portrait. interface.

In some embodiments, the operation interface display unit is further configured to: generate an operation interface including a preset background and at least one virtual input option based on the current display interface projected to the target screen, wherein the preset background includes a default background or a user The background is set based on the background settings option.

In some embodiments, the virtual input options in the operation interface display unit include: virtual interactive handles.

In some embodiments, the interactive device further includes (not shown in the figure): an option vibration presentation unit configured to respond to the user's selection operation on the virtual input option, and present a vibration frequency corresponding to the selected virtual input option. shock.

In some embodiments, the interactive device further includes (not shown in the figure): a mobile identification presentation unit configured to present a movable identification of the virtual input option in response to the user's selected operation on the generated virtual input option; input The option moving unit is configured to move the virtual input option indicated by the move operation to a target position indicated by the move operation in response to the user's move operation on the virtual input option having the movable identifier; the option selection release unit is configured Unselecting the virtual input option in response to user input exiting the selected operation.

In some embodiments, the interaction device further includes (not shown in the figure): an input device activating unit configured to use the physical input device in response to the selection operation instruction, based on the correspondence between the preset local physical input device and the current display interface. relationship, start using the local entity input device and blacken the local screen.

In some embodiments, the input device initiating unit is further configured to: pop up the physical interaction handle.

In some embodiments, the size of the target screen in the display interface projection unit is larger than the local screen.

In some embodiments, the display interface projection unit is further configured to: in response to receiving the user's selection operation on the multi-screen interaction option, project the current display interface to the TV screen via the adapter of the external device in the TV.

It should be understood that each unit recorded in the device 500 corresponds to each step recorded in the method described with reference to Figures 2a-4d. Therefore, the operations and features described above for the method are also applicable to the device 500 and the various units included therein, and will not be described again here.

Referring now to FIG. 6 , a schematic structural diagram of an electronic device (such as the server or terminal device in FIG. 1 ) 600 suitable for implementing embodiments of the present disclosure is shown. Terminal devices in embodiments of the present disclosure may include, but are not limited to, laptop computers, desktop computers, etc. The terminal device/server shown in FIG. 6 is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present disclosure.

As shown in FIG. 6, the electronic device 600 may include a processing device (eg, central processing unit, graphics processor, etc.) 601, which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 602 or from a storage device 608. The program in the memory (RAM) 603 executes various appropriate actions and processes. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 607 such as a computer; a storage device 608 including a magnetic tape, a hard disk, etc.; and a communication device 609. Communication device 609 may allow electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG. 6 illustrates electronic device 600 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided. Each block shown in Figure 6 may represent one device, or may represent multiple devices as needed.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such embodiments, the computer program may be downloaded and installed from the network via communication device 609, or from storage device 608, or from ROM 602. When the computer program is executed by the processing device 601, the above-described functions defined in the method of the embodiment of the present disclosure are performed. It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to receiving the user's selection operation of the multi-screen interactive option, projects the current display interface. to the target screen; presenting an option for selecting the input device; receiving a selection operation for the option for selecting the input device; employing the virtual input device in response to the selection operation indication, and generating, based on the current display interface projected to the target screen, including at least one virtual input option. The operation interface is displayed on the local screen and the current display interface projected to the target screen is not displayed.

Computer program code for performing operations of embodiments of the present disclosure may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, or a combination thereof, Also included are conventional procedural programming languages—such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider). connected via the Internet).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. The described unit can also be provided in a processor. For example, it can be described as: a processor includes a display interface projection unit, a selection option presentation unit, a selection operation receiving unit, and an operation interface display unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the display interface projection unit can also be described as "in response to receiving the user's selection operation of the multi-screen interaction option, converting the current display interface into The unit that projects to the target screen".

The above description is only a description of the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to technical solutions formed by a specific combination of the above technical features, but should also cover any solution consisting of the above technical features or without departing from the above inventive concept. Other technical solutions formed by any combination of equivalent features. For example, a technical solution is formed by replacing the above features with technical features with similar functions disclosed in this disclosure (but not limited to).

Claims (14)

1. An interaction method for multiple screens, including: In response to receiving the user's selection operation on the multi-screen interaction option, project the current display interface to the target screen; Presenting options for selecting an input device includes: presenting options for selecting a virtual input device and options for selecting a physical input device; receiving a selection operation for an option of the selection input device; In response to the selection operation instruction, a virtual input device is used to generate an operation interface including at least two virtual input options based on the current display interface projected to the target screen, including: using AI big data to predict the user's current display interface projected to the target screen. User portrait of the display interface; based on the predicted user portrait and the current display interface, generate an operation interface including a preset background and at least two virtual input options, where the user portrait includes: the user category to which the current user belongs is in the current interactive interface The layout of the virtual input options in the corresponding virtual input device, the default background includes the background set by the user based on the background setting options, and the operation interface is displayed on the local screen, and the current display interface projected to the target screen is not displayed. ; In response to the selection operation instruction to use the physical input device, based on the preset corresponding relationship between the local physical input device and the current display interface, start using the local physical input device and blacken the local screen, the local physical input device having Raised mode, concave mode and flat sensing mode. 2. The interaction method according to claim 1, wherein the projecting the current display interface to the target screen in response to receiving the user's selection operation on the multi-screen interaction option includes: In response to receiving the local user's selection operation on the multi-screen interaction option, the current display interface including the local user's interaction operation and the remote user's interaction operation is projected to the target screen. 3. The interaction method according to any one of claims 1-2, wherein the virtual input options include: virtual interaction handles. 4. The interaction method according to claim 1, wherein the interaction method further includes: In response to the user's selection operation on the virtual input option, vibrations with a vibration frequency corresponding to the selected virtual input option are presented. 5. The interactive method according to claim 1, wherein the interactive method further includes: responsive to a user's selection of the generated virtual input option, presenting a movable identification of the virtual input option; In response to the user's movement operation on the virtual input option having the movable identification, moving the virtual input option indicated by the movement operation to the target position indicated by the movement operation; The virtual input option is deselected in response to a user input exit operation for the selected operation. 6. The interaction method according to claim 1, wherein initiating the use of a local entity input device includes popping up an entity interaction handle. 7. The interaction method according to claim 1, wherein the size of the target screen is larger than the local screen. 8. The interaction method according to claim 1, wherein the projecting the current display interface to the target screen in response to receiving the user's selection operation on the multi-screen interaction option includes: In response to receiving the user's selection operation on the multi-screen interaction option, the current display interface is projected to the TV screen via an adapter of the external device in the TV. 9. An interactive device for multiple screens, including: The display interface projection unit is configured to project the current display interface to the target screen in response to receiving the user's selection operation on the multi-screen interaction option; a selection option presentation unit configured to present options for selecting an input device, including: presenting options for selecting a virtual input device and options for selecting a physical input device; a selection operation receiving unit configured to receive a selection operation for an option of the selection input device; The operation interface display unit is configured to use a virtual input device in response to the selection operation instruction, and generate an operation interface including at least two virtual input options based on the current display interface projected to the target screen, including: using AI big data, predicting A user portrait of the user on the current display interface projected to the target screen; based on the predicted user portrait and the current display interface, an operation interface including a preset background and at least two virtual input options is generated, where the user portrait includes: the current user The layout of the virtual input options in the virtual input device corresponding to the current interactive interface of the user category to which the user belongs. The default background includes the background set by the user based on the background setting options, and the operation interface is displayed on the local screen or not. Project to the current display interface of the target screen; The input device activating unit is configured to use the physical input device in response to the selection operation instruction, start using the local physical input device based on the preset correspondence relationship between the local physical input device and the current display interface, and blacken the local screen. , the local entity input device has a convex mode, a concave mode and a flat sensing mode. 10. The interactive device according to claim 9, wherein the display interface projection unit is further configured to: In response to receiving the local user's selection operation on the multi-screen interaction option, the current display interface including the local user's interaction operation and the remote user's interaction operation is projected to the target screen. 11. The interactive device according to claim 9, wherein the interactive device further includes: a mobile logo presentation unit configured to present a movable logo of the virtual input option in response to a user's selected operation on the generated virtual input option; an input option moving unit configured to, in response to a user's movement operation on a virtual input option having a movable identification, move the virtual input option indicated by the movement operation to a target position indicated by the movement operation; An option selection deselection unit is configured to deselect the virtual input option in response to a user input exit operation for the selection operation. 12. The interactive device according to claim 9, wherein the display interface projection unit is further configured to: In response to receiving the user's selection operation on the multi-screen interaction option, the current display interface is projected to the TV screen via an adapter of the external device in the TV. 13. An electronic device, including: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any one of claims 1-8. 14. A computer-readable medium having a computer program stored thereon, which implements the method according to any one of claims 1-8 when executed by a processor.