CN116880946A

CN116880946A - Remote control method, device, equipment and medium based on mobile terminal

Info

Publication number: CN116880946A
Application number: CN202310891325.2A
Authority: CN
Inventors: 吴汉彬; 廖唯心
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-07-19
Filing date: 2023-07-19
Publication date: 2023-10-13

Abstract

The application discloses a remote control method, a device, equipment and a medium based on a mobile terminal. Belonging to the field of man-machine interaction. The method is performed by a mobile terminal and comprises the following steps: responding to remote connection operation between the mobile terminal and the remote terminal, and displaying a remote operation interface of the remote terminal in a display screen of the mobile terminal; responding to the triggering operation, displaying an amplifying assembly in a display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplification factors, and N is more than 1; in the magnification component, magnification interface content corresponding to the trigger operation is displayed, the magnification interface content being associated with an operation position indicated by the trigger operation. The scheme can be applied to a remote control scene, so that the operation flow of amplifying the interface content of the remote terminal is simplified, and the man-machine interaction efficiency is improved.

Description

Remote control method, device, equipment and medium based on mobile terminal

Technical Field

The present application relates to the field of man-machine interaction, and in particular, to a remote control method, apparatus, device and medium based on a mobile terminal.

Background

The remote connection is that a manager connects the controlled terminal through means such as remote dialing of a computer network or network access of both parties, an operation interface of the controlled terminal is displayed on the control terminal of the manager, and remote control, configuration, software installation program, modification and other works are carried out on the controlled terminal through the local control terminal.

In the related art, a manager manually amplifies an operation interface of a controlled terminal in a double-finger manner at the operation interface of the control terminal, and performs accurate selection in the amplified operation interface, thereby realizing accurate control of the controlled terminal.

However, this approach is cumbersome to operate, resulting in inefficient human-machine interaction.

Disclosure of Invention

The application provides a remote control method, a device, equipment and a medium based on a mobile terminal. The technical scheme is as follows:

according to an aspect of the present application, there is provided a mobile terminal-based remote control method, the method being performed by a mobile terminal, the method comprising:

responding to remote connection operation between the mobile terminal and a remote terminal, and displaying a remote operation interface of the remote terminal in a display screen of the mobile terminal;

responding to triggering operation, displaying an amplifying assembly in the display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplifying power, and N is larger than 1;

And displaying the amplified interface content corresponding to the triggering operation in the amplifying component, wherein the amplified interface content is associated with the operation position indicated by the triggering operation.

According to another aspect of the present application, there is provided a mobile terminal-based remote control apparatus, the apparatus comprising:

the display module is used for responding to remote connection operation between the mobile terminal and a remote terminal and displaying a remote operation interface of the remote terminal in a display screen of the mobile terminal;

the display module is used for responding to the triggering operation, displaying an amplifying assembly in the display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplification factors, and N is larger than 1;

the display module is used for displaying the amplified interface content corresponding to the triggering operation in the amplifying assembly, and the amplified interface content is associated with the operation position indicated by the triggering operation.

According to another aspect of the present application, there is provided a computer apparatus including: a processor and a memory storing a computer program that is loaded and executed by the processor to implement a mobile terminal based remote control method as described above.

According to another aspect of the present application, there is provided a computer-readable storage medium storing a computer program loaded and executed by a processor to implement the mobile terminal-based remote control method as described above.

According to another aspect of the present application, there is provided a computer program product comprising computer instructions stored in a computer readable storage medium, from which a processor retrieves the computer instructions, causing the processor to load and execute to implement a mobile terminal based remote control method as described above.

The technical scheme provided by the embodiment of the application has the beneficial effects that at least:

the mobile terminal responds to remote connection operation between the mobile terminal and the remote terminal, and a remote operation interface of the remote terminal is displayed in a display screen of the mobile terminal; responding to the triggering operation, displaying an amplifying assembly in a display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplification factors, and N is more than 1; in the magnification component, magnification interface content corresponding to the trigger operation is displayed, the magnification interface content being associated with an operation position indicated by the trigger operation. By adopting the scheme of the embodiment of the application, under the scene of adopting the mobile terminal to remotely control the remote terminal, a user does not need to amplify the display content of the remote terminal in a mode of manually amplifying by using two fingers, the user can trigger the mobile terminal to automatically display the amplifying assembly in the display screen and display the amplifying interface content by triggering operation on the mobile terminal, the operation flow of amplifying the interface content of the remote terminal is simplified, and the man-machine interaction efficiency is improved. Remote control is carried out on the mobile terminal based on the content of the amplified interface, so that the false touch condition during remote control can be reduced, and the accuracy of remote control is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 illustrates a block diagram of a computer system provided by an exemplary embodiment;

fig. 2 shows a schematic diagram of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 3 shows a flowchart of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 4 shows a schematic diagram of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 5 shows a flowchart of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 6 shows a schematic diagram of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 7 shows a flowchart of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 8 shows a flowchart of a mobile terminal-based remote control method provided by an exemplary embodiment;

Fig. 9 shows a flowchart of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 10 shows a flowchart of a mobile terminal-based remote control method provided by the exemplary embodiment;

fig. 11 is a schematic diagram of a remote control method based on a mobile terminal according to an exemplary embodiment;

fig. 12 is a schematic diagram of a remote control method based on a mobile terminal according to an exemplary embodiment;

fig. 13 shows a flowchart of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 14 shows a flowchart of a mobile terminal-based remote control method provided by the exemplary embodiment;

fig. 15 shows a schematic diagram of a mobile terminal-based remote control method provided by an exemplary embodiment;

fig. 16 is a block diagram showing a structure of a remote control apparatus based on a mobile terminal according to an exemplary embodiment;

fig. 17 shows a block diagram of a computer device provided by an exemplary embodiment.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be noted that, before collecting relevant data of a user and a terminal (such as remote connection data between a mobile terminal and a remote terminal, screen video data of a remote terminal, etc.), and during collecting relevant data of a user, the present application may display a prompt interface, a popup window, or output voice prompt information, where the prompt interface, the popup window, or the voice prompt information is used to prompt the user to collect relevant data currently, so that the present application only starts to execute a relevant step of acquiring relevant data of the user after acquiring a confirmation operation sent by the user to the prompt interface or the popup window, or otherwise (i.e. when a confirmation operation sent by the user to the prompt interface or the popup window is not acquired), ends a relevant step of acquiring relevant data of the user, i.e. does not acquire relevant data of the user. In other words, all user data collected by the present application is collected with the consent and authorization of the user, and the collection, use and processing of relevant user data requires compliance with relevant laws and regulations and standards of the relevant country and region.

It should be understood that, although the terms first, second, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter, without departing from the scope of the application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

First, the nouns relating to the present embodiment will be described.

Floating layer: a control that is positioned overhead of the visual topmost layer.

Remote operation of the mobile phone: and a mode of remotely controlling the computer through the mobile terminal of the mobile phone. The operation mode of the remote control computer includes one of a touch screen mode and a pointer mode.

Touch screen mode: the operation mode of simulating mouse click is directly clicked by a finger.

Pointer mode: an operation mode of controlling the movement of the mouse pointer by finger drag.

FIG. 1 illustrates a block diagram of a computer system provided in accordance with an exemplary embodiment of the present application. The computer system may be implemented as a system architecture for a mobile terminal-based remote control method. The computer system 100 includes: mobile terminal 110 and remote terminal 120.

The mobile terminal 110, the remote terminal 120 may be electronic devices such as a mobile phone, a tablet computer, a vehicle-mounted terminal (car), a wearable device, a PC (Personal Computer ), an unmanned reservation terminal, etc., respectively. Alternatively, the device types of the mobile terminal 110 and the remote terminal 120 may be the same or different. For example, the mobile terminal 110 is a cellular phone, the remote terminal 120 is a computer, and the screen size of the display screen of the mobile terminal 110 is smaller than the screen size of the display screen of the remote terminal 120. The application is not limited in this regard.

The mobile terminal 110 and the remote terminal 120 may respectively install a client for running a target application program, where the target application program may be a remote connection and remote control application program, or may be other application programs that provide functions of remote connection and remote control, such as: chat applications, gaming applications, etc. that provide remote connectivity and remote control functionality, as the application is not limited in this regard. The form of the target Application program is not limited to the present application, and may be a web page, including but not limited to an App (Application), applet, etc. installed in the mobile terminal 110 and the remote terminal 120.

The mobile terminal 110 and the remote terminal 120 can establish remote connection through at least one of wired or wireless network, computer network remote dialing, network access by both parties, remote connection software and the like, thereby realizing remote connection and remote control between the mobile terminal 110 and the remote terminal 120.

According to the remote control method based on the mobile terminal, an execution main body of each step can be computer equipment, and the computer equipment is electronic equipment with data calculation, processing and storage capacity.

Taking the implementation environment of the scheme shown in fig. 1 as an example:

alternatively, when the mobile terminal 110 is used as a control terminal and the remote terminal 120 is used as a controlled terminal, the mobile terminal 110 may perform a remote control method based on the mobile terminal (for example, a remote control method based on the mobile terminal is performed based on a client of a target application program installed and running in the mobile terminal 110), amplify interface contents of the remote terminal 120, and remotely control the remote terminal 120.

Alternatively, when the remote terminal 120 is used as a control terminal and the mobile terminal 110 is used as a controlled terminal, the remote control method based on the mobile terminal (for example, the remote control method based on the mobile terminal is performed by a client terminal of the target application program installed and operated in the remote terminal 120) may be performed by the remote terminal 120, so as to amplify the interface content of the mobile terminal 110 and remotely control the mobile terminal 110.

Those skilled in the art will appreciate that the number of mobile terminals 110, remote terminals 120 may be greater or lesser, respectively. Such as: the number of mobile terminals 110 may be only one, and the number of remote terminals 120 may be only one, so that one mobile terminal 110 is used to remotely connect and remotely control one remote terminal 120. Alternatively, there may be only one mobile terminal 110, and there may be a plurality of remote terminals 120, and one mobile terminal 110 is used to remotely connect and remotely control a plurality of remote terminals 120. The number of the mobile terminals 110 may be plural, and the number of the remote terminals 120 may be only one, so that one remote terminal 120 is remotely connected and remotely controlled by using a plurality of mobile terminals 110. The number and device types of the mobile terminal 110 and the remote terminal 120 are not limited in the embodiment of the present application.

Fig. 2 is a schematic diagram illustrating a remote control method based on a mobile terminal according to an exemplary embodiment of the present application. In the case where the control terminal is the mobile terminal 110 and the controlled terminal is the remote terminal 120, the mobile terminal 110 remotely controls the remote terminal 120, and the method is performed by the mobile terminal 110 as an example. The steps are briefly described as follows:

1. the mobile terminal responds to remote connection operation between the mobile terminal and the remote terminal, and a remote operation interface of the remote terminal is displayed in a display screen of the mobile terminal.

And the mobile terminal responds to the selection operation of the remote connection control, and responds to the selection operation of the remote terminal, so that the remote connection between the mobile terminal and the remote terminal is realized, and a remote operation interface of the remote terminal is displayed in a display screen of the mobile terminal. Wherein the screen size of the display screen of the mobile terminal is smaller than the screen size of the display screen of the remote terminal. Therefore, in the process of remotely controlling the remote terminal by the mobile terminal, the remote operation interface of the remote terminal displayed in the display screen of the mobile terminal needs to be enlarged.

Illustratively, as shown by interface 132 of (1) in fig. 2, interface 132 is a remote operation interface of remote terminal 120 displayed in a display screen of mobile terminal 110. In the related art, a user manually enlarges the remote operation interface of the remote terminal 120 on the display screen of the mobile terminal 110 through the double fingers 134, as shown in an interface 136 of (2) in fig. 2, the interface 136 is the enlarged remote operation interface of the remote terminal 120 displayed on the display screen of the mobile terminal 110, and a manager performs precise selection through the interface 136, thereby realizing precise control of the remote terminal 120.

2. And the mobile terminal responds to the triggering operation, and displays an amplifying assembly in the display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplifying power, and N is more than 1.

3. In the magnification component, magnification interface content corresponding to the trigger operation is displayed, the magnification interface content being associated with an operation position indicated by the trigger operation.

Illustratively, as shown by interface 140 in (3) of fig. 2, interface 140 is a remote operation interface of remote terminal 120 displayed in a display screen of mobile terminal 110, and may further include an edge display area 141 and an operation control display area 143 in the display screen of mobile terminal 110. Nothing may be displayed in the edge display area 141, and an amplification component may be displayed in the edge display area 141 in this embodiment. The operation control display area 143 generally displays operation controls for performing remote control of the remote terminal 120, and the operation controls shown in fig. 2 (3) include an up-down movement control and a right key selection control.

Illustratively, the user initiates the trigger operation 142 and the mobile terminal 110 displays the magnification component 144 in the display screen in response to the trigger operation 142, with the magnification component display area of the magnification component 144 being located to the right of the location 142. The magnified interface content corresponding to the triggering operation 142 is displayed in the magnification component 144, where the magnified interface content is associated with the operational location indicated by the triggering operation 142. Such as: the operation position indicated by the triggering operation 142 corresponds to the display content "XXXXXXX", and the enlarged display content in which the display content "XXXXXXX" is enlarged with N-fold magnification is displayed in the enlargement component 144, N being greater than 1.

Optionally, the mobile terminal 110 also switches an amplifying assembly display area of the display screen in response to the moving operation, and displays an amplifying assembly in the switched amplifying assembly display area, and displays the content of the amplifying interface corresponding to the triggering operation in the amplifying assembly.

As shown in interface 150 of (4) in fig. 2, interface 150 is a remote operation interface of remote terminal 120 displayed in a display screen of mobile terminal 110, in which an edge display area 151 and an operation control display area 153 may also be included. Nothing may be displayed in the edge display area 151, and an amplifying component may also be displayed in the edge display area 151 in the present embodiment. The operation control display area 153 generally displays operation controls for performing remote control of the remote terminal 120, and the operation controls shown in (4) of fig. 2 include an up-down movement control and a right key selection control.

Illustratively, after the user initiates the triggering operation 142, the finger of the user continues to move, and in response to the moving operation 152, the mobile terminal 110 switches the display area of the amplifying component of the display screen, and switches the display area of the amplifying component from the right display corresponding to the triggering operation 142 to the left display corresponding to the moving operation 152. The magnification component 154 is displayed in the magnification component display area after switching, and the magnification interface content corresponding to the trigger operation 152 is displayed in the magnification component 154, wherein the magnification interface content is associated with the operation position indicated by the movement operation 152. Such as: the operation position indicated by the movement operation 152 corresponds to the display content "yyyyyy", and the enlarged display content in which the display content "yyyyyy" is enlarged with N times of magnification is displayed in the enlargement component 154, N being greater than 1.

In summary, in the scenario that the mobile terminal performs remote control on the remote terminal, the user does not need to amplify the display content of the remote terminal in a mode of manually amplifying by using two fingers, and the mobile terminal can automatically display the amplifying assembly and display the amplified interface content by triggering operation, so that the operation flow of amplifying the interface content of the remote terminal is simplified, and the man-machine interaction efficiency is improved. Further, the mode of this embodiment makes the user can not be sheltered from by the finger when carrying out remote control operation to the second terminal, reduces the circumstances that the mistake touched when the user used for the user carries out the remote selection and remote control operation to the second terminal on mobile terminal accurately, can realize carrying out accurate window drag, file drag etc. remote control operation to the second terminal, improves the accuracy of carrying out remote control to the second terminal.

For example, fig. 3 shows a flowchart of a mobile terminal based remote control method provided by an exemplary embodiment of the present application, to control a terminal to be the mobile terminal 110 in fig. 1 and a controlled terminal to be the remote terminal 120 in fig. 1, the method is illustrated by the mobile terminal 110, and the method may include steps 220, 240 and 260:

And 220, in response to the remote connection operation between the mobile terminal and the remote terminal, displaying a remote operation interface of the remote terminal in a display screen of the mobile terminal.

The mobile terminal refers to a control terminal. The remote terminal refers to a controlled terminal.

Alternatively, the mobile terminal may be a mobile terminal, such as: the mobile terminal is a mobile phone or a smart watch. The screen size of the display screen of the mobile terminal is smaller than the screen size of the display screen of the remote terminal. Therefore, in the process of remotely controlling the remote terminal by the mobile terminal, the remote operation interface of the remote terminal displayed in the display screen of the mobile terminal needs to be enlarged.

The remote connection operation refers to an operation of remotely connecting the mobile terminal to the remote terminal by selecting or clicking a remote connection control.

In some embodiments, the mobile terminal and the remote terminal each support a remote connection function. The remote connection function can be a system self-contained function of the mobile terminal and the remote terminal, and can also be realized by remote connection software.

Optionally, the mobile terminal and the remote terminal can be connected remotely through at least one of a wired or wireless network, a computer network for remote dialing, network access by both parties, remote connection software and the like, so as to realize remote connection and remote control between the mobile terminal and the remote terminal.

The remote operation interface is a remote operation interface of a remote terminal displayed in a display screen of the mobile terminal.

Optionally, the interface content of the remote operation interface displayed in the display screen of the mobile terminal is the interface content currently displayed in the display screen of the remote terminal. The remote terminal can be remotely controlled in real time through the mobile terminal.

Alternatively, the remote operation interface may be an operating system interface running on the remote terminal, or an application program interface corresponding to an application program running in the foreground of the remote terminal.

Such as: the display screen of the remote terminal stays on a desktop, the desktop comprises two icons of a recycle bin and a my computer, the mouse pointer stays at the position of the recycle bin, and after the mobile terminal establishes remote connection with the remote terminal, a remote operation interface displayed in the display screen of the mobile terminal is the desktop of the remote terminal. By moving and clicking the mouse pointer by the mobile terminal, the recycle bin of the remote terminal can be opened just as if the recycle bin were opened directly by the remote terminal. Such as: and when the chat software is operated in the foreground of the display screen of the remote terminal, displaying a chat software interface of the chat software on the display screen of the mobile terminal after the remote connection.

In some embodiments, the screen size of the display screen of the mobile terminal is greater than or equal to the interface size of the remote operation interface. That is, in the display screen of the mobile terminal, the remote operation interface occupies at least a part of the display area. In this embodiment, at least a part of the display area occupied by the remote operation interface is referred to as a local display area. Generally, a remote operation interface is displayed at the very middle of a display screen of a mobile terminal.

Optionally, an edge display area and an operation control display area may be further included in the display screen of the mobile terminal, and reference may be made to the edge display area 141 and the operation control display area 143 in fig. 2.

The edge display area refers to at least a portion of the display area located at the edge of the display screen. Illustratively, the edge display area is generally referred to as a black screen area.

The operation control display area is a display area where the operation control in the display screen is located.

Optionally, the edge display area does not overlap with a local display area corresponding to the remote operation interface. The operation control display area and the edge display area or the local display area corresponding to the remote operation interface may be partially overlapped or not overlapped.

Such as: the operation control display area may be located within the edge display area, or the operation control display area may be located outside the edge display area, or located within a local display area corresponding to the remote operation interface, or a portion of the operation control display area may be located within a local display area corresponding to the remote operation interface.

Illustratively, in the case where the screen size of the display screen is larger than the interface size of the remote operation interface, at least one edge display area is included in the display screen. Wherein, the edge display area can not display any content, and the edge display area in the embodiment can also optionally display an amplifying component.

The operation control display area generally displays an operation control for performing remote control on the remote terminal, wherein the operation control comprises at least one of an up-down movement control, a right key selection control, a keyboard selection control, a shortcut key selection control, a horizontal screen display mode selection control, a vertical screen display mode selection control, a virtual button control, a touch screen mode selection control, a pointer mode selection control and an operation guide selection control.

Optionally, the mobile terminal responds to the up-down movement operation of the up-down movement control, and displays a remote operation interface after up-down movement. And the mobile terminal responds to the selection operation of the mode selection control on the transverse screen, and switches the current display mode of the display screen into the transverse screen display mode. And the mobile terminal responds to the selection operation of the mode selection control on the vertical screen, and switches the current display mode of the display screen into the vertical screen display mode. And the mobile terminal responds to the selection operation of the pointer mode selection control, and the current operation mode of the mobile terminal is switched to the pointer mode. And the mobile terminal responds to the selection operation of the touch screen mode selection control, and the current operation mode of the mobile terminal is switched to the touch screen mode. The mobile terminal displays an operation guide in a display screen of the mobile terminal in response to a selection operation at the operation guide selection control.

It should be noted that, the positions of the edge display area and the operation control display area of the display screen of the mobile terminal in the display screen are not limited in this embodiment. Such as: the edge display area may be at least one of an upper edge, a lower edge, a left edge, and a right edge of the display screen. The operation control display area may be located at least one of the upper, lower, left, and right positions of the display screen.

In one embodiment, a remote connection procedure between a mobile terminal and a remote terminal is described. For example, as shown in fig. 4, the mobile terminal is a mobile phone, and the remote terminal is a computer. As shown in fig. 4 (1), selecting the remote control function 221, clicking the remote control computer portal 222 selects a computer that needs remote connection, and realizes viewing/manipulating the computer desktop in the mobile phone. As shown in fig. 4 (2), a computer 223 requiring remote connection is selected, and a computer a224 is selected. As shown in fig. 4 (3), after the password is input in the password area 225, the user connects to the loading page 226 of the remote connection computer shown in fig. 4 (4), and after the remote connection is successful, the user can see the desktop of the computer on the display screen of the mobile phone as shown in fig. 4 (5). An edge display area 227, a partial display area 228, and an operation control display area 229 are included in the display screen. Wherein, the edge display area 227 temporarily does not display any content, the local display area 228 displays a computer desktop, and the operation control display area 229 displays an operation control.

And 240, in response to the triggering operation, displaying an amplifying assembly in the display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplifying power, and N is larger than 1.

The triggering operation refers to an operation of triggering the display amplifying assembly.

The amplifying component is used for previewing the interface content of the remote operation interface by adopting N times of amplifying power, wherein N is larger than 1. The magnifying element corresponds to a magnifying lens.

Optionally, the content type of the interface content includes at least one of text, image, icon, element.

Alternatively, the shape of the magnifying element may be one of a circle, a polygon, an irregular pattern. The amplifying assembly is in a suspension state and is suspended on a display interface of the display screen, and the amplifying assembly is also called a floating layer.

In some embodiments, the size of the amplifying component may be a fixed preset size, may be adjusted according to the screen size of the display screen and the interface size of the remote operation interface, or may be user-defined.

Such as: the shape of the amplifying assembly is a circle with a radius of 100 pixels (Device Independent Pixels, DP), wherein the pixels, also referred to as device independent pixels, are a unit of length. Or, the shape of the magnifying member is a circle, and the radius is determined according to a height difference between a screen height of the display screen and an interface height of the remote operation interface. Or, the size of the magnification component is custom determined by the user.

Illustratively, the mobile terminal displays the magnification component in the display screen in response to a triggering operation.

In some embodiments, the mobile terminal plays the pop-up special effect of the amplifying assembly in response to the triggering operation, and displays the amplifying assembly in the display screen after the pop-up special effect is played.

Alternatively, the pop-up effect of the amplifying assembly may be at least one of an animation effect, a music effect, and a vibration effect. The pop-up special effect can be set by user definition, and can also be set according to the shape of the amplifying assembly or the content of an amplifying interface displayed in the amplifying assembly or the display area of the amplifying assembly where the amplifying assembly is located. Through multiple reminding modes, the user is reminded to trigger the amplifying assembly, and the man-machine interaction efficiency can be effectively improved.

For example, the pop-up effect of the user-defined amplifying assembly is a vibration effect, the number of vibration times is one, the mobile terminal responds to the triggering operation and vibrates once, and the amplifying assembly is displayed in the display screen after the vibration is finished. For example, the shape of the amplifying component of the mobile terminal defaults to a circular shape, and the corresponding pop-up special effect is that the music accompanies vibration once, then the mobile terminal plays the music accompanies vibration once in response to the trigger operation, and the amplifying component is displayed in the display screen after the music accompanies vibration is finished. For example, the content of the amplifying interface displayed in the amplifying assembly is text content, the pop-up special effect corresponding to the text content is firework animation, the mobile terminal responds to the triggering operation to play the firework animation, and the amplifying assembly is displayed in the display screen after the firework animation is played.

In step 260, in the magnification component, magnification interface content corresponding to the trigger operation is displayed, the magnification interface content being associated with the operation position indicated by the trigger operation.

The interface content is amplified by adopting N amplified times.

Illustratively, the enlarged interface content is changed from the original interface content only in terms of the interface content size, which includes text size, image size, icon size, element size, etc.

In some embodiments, based on the content type of the interface content of the remote operation interface, at least one magnification corresponding to the interface content is determined, and in the magnification component, the magnified interface content corresponding to the triggering operation is displayed according to the at least one magnification of the interface content.

Optionally, there is a one-to-one correspondence between content types and magnification of interface content, and magnification corresponding to different content types is different. The corresponding relation can be set by user definition or set according to the shape and the size of the amplifying component. The correspondence may be expressed as follows:

for example, the content type of the interface content includes both text and icons, wherein the text corresponds to a first magnification and the icon corresponds to a second magnification, and the first magnification is different from the second magnification in value. And displaying the amplified characters according to the first amplification factors corresponding to the characters and displaying the amplified icons according to the second amplification factors corresponding to the icons in the amplifying assembly. Accordingly, different content types can be displayed in an enlarged manner adaptively,

In some embodiments, at least one magnification corresponding to the interface content is determined based on the respective original sizes of the different content types of the interface content of the remote operation interface, and in the magnification component, the magnified interface content corresponding to the trigger operation is displayed according to the at least one magnification of the interface content. Wherein, the amplified sizes of the different content types in the amplified interface content are consistent.

Alternatively, the size after amplification may be set according to actual technical needs, for example, according to the shape and size of the amplifying assembly, or user-defined. By the method, different content types of interface contents of the remote operation interface can be amplified to be uniform in size.

For example, the content type of the interface content includes text and an icon, where the text corresponds to a first original size, the icon corresponds to a second original size, and the first original size is smaller than the second original size, and the value of the first magnification corresponding to the text is larger than the value of the second magnification corresponding to the icon. And displaying the amplified characters according to the first amplification factors corresponding to the characters and displaying the amplified icons according to the second amplification factors corresponding to the icons in the amplifying assembly, so that the amplified characters are consistent with the amplified icons in size.

In some embodiments, in the magnification component, magnification interface content corresponding to the triggering operation is displayed. The magnified interface content is associated with an operational location that triggers the operational indication. The content of the amplifying interface may be determined according to the operation position indicated by the trigger operation, or based on the region to be amplified corresponding to the operation position indicated by the trigger operation, or based on the shape and size of the amplifying component.

Such as: the amplifying component is circular, the radius is 100DP, the interface content on the remote terminal corresponding to the operation position of the triggering operation instruction is 'this is an lovely cat', the amplifying display content displayed in the amplifying component can be 'this is an lovely cat' due to the limitation of the shape and the size of the amplifying component, and the amplifying display content displayed in the amplifying component can become 'lovely cat' along with the movement of the position of the triggering operation. Accordingly, the user can precisely see all interface contents in the remote terminal.

In some embodiments, after displaying the magnification interface content corresponding to the trigger operation, the mobile terminal displays the magnification-adjusted magnification interface content in the magnification assembly in response to the magnification adjustment operation.

The magnification adjustment operation is an operation of adjusting the magnification of the content of the magnification interface after the content of the magnification interface corresponding to the trigger operation is displayed in the magnification assembly.

Alternatively, the magnification adjustment operation may be implemented based on a sliding control. Taking the up-down sliding control as an example, the up-down sliding control can increase the magnification of the content of the amplified interface when sliding upwards, and can decrease the magnification of the content of the amplified interface when sliding downwards. Through the mode, the content of the amplifying interface displayed in the amplifying assembly can be readjusted, and readjustment of the amplifying factor is realized.

In some embodiments, the mobile terminal displays, in the amplifying component, the content of the amplifying interface corresponding to the triggering operation, and at the same time, the mobile terminal further displays an area frame corresponding to the triggering operation, where the area frame is used to indicate an operation position corresponding to the triggering operation. Wherein the region frame is changed with a change in the operation position of the trigger operation.

Optionally, the border color, shape, size of the region frame may be user-defined or set according to the content type and original size of the interface content. Therefore, the user can know the current position of the finger when performing the triggering operation, and the accuracy of operation position selection and remote control can be improved.

In some embodiments, the mobile terminal may further perform content recognition on the content of the zoom-in interface, and display recommended content corresponding to the content of the zoom-in interface in the zoom-in component. Therefore, recommended content can be generated for the user according to the content of the amplified interface, and the remote control mode is enriched.

Optionally, the mobile terminal stores at least one content recognition model, which is used for recognizing content of at least one content type of characters, images, icons and elements in the content of the enlarged interface, so as to obtain a content recognition result. And the mobile terminal determines recommended content corresponding to the amplified interface content and a link of the recommended content according to the content identification result, and displays the recommended content corresponding to the amplified interface content by displaying the link of the recommended content.

For example, after determining the recommended content of the amplified interface content, the mobile terminal displays the amplified interface content in the central display area of the amplifying assembly, and displays the specific content or link corresponding to the recommended content of the amplified interface content in the edge display area of the amplifying assembly. The link may be a web page link.

Optionally, the mobile terminal links the recommended content corresponding to the amplified interface content with the amplified interface content to obtain a hyperlink corresponding to the amplified interface content, and the mobile terminal may further display the amplified interface content carrying the hyperlink in the amplifying component.

Optionally, the mobile terminal further responds to a triggering operation on the recommended content in the amplifying assembly, and the recommended content corresponding to the content of the amplifying interface is displayed in the amplifying assembly.

In summary, in a scenario in which the mobile terminal performs remote control on the remote terminal, the mobile terminal responds to a remote connection operation between the mobile terminal and the remote terminal, and a remote operation interface of the remote terminal is displayed in a display screen of the mobile terminal; responding to the triggering operation, displaying an amplifying assembly in a display screen, wherein the amplifying assembly is used for previewing the interface content of the remote operation interface by adopting N times of amplification factors, and N is more than 1; in the magnification component, magnification interface content corresponding to the trigger operation is displayed, the magnification interface content being associated with an operation position indicated by the trigger operation. According to the scheme, a user does not need to amplify the display content of the remote terminal in a manual amplification mode by using two fingers, the user can trigger the automatic display amplification assembly in the display screen of the mobile terminal through triggering operation of the mobile terminal, and display amplified interface content, so that the operation flow of amplifying the interface content of the remote terminal is simplified, and the man-machine interaction efficiency is improved.

In some embodiments, the triggering operation includes a drag operation, which refers to an operation that the mobile terminal receives a movement event after receiving a pressing event. The press event includes a press by a finger touch, and the move event includes a move by a finger touch.

Alternatively, the drag operation may be implemented based on a touch screen mode, or based on a pointer mode. The drag operation based on the touch screen mode is that the user performs the operation by clicking and dragging with a finger in the touch screen mode, and the drag operation based on the pointer mode is that the user performs the operation by clicking and dragging with a finger in the pointer mode.

Fig. 5 is a flowchart illustrating a remote control method based on a mobile terminal according to an exemplary embodiment of the present application. Step 240 is optionally implemented as step 242:

in response to the drag operation, the magnification component is displayed in the display screen, step 242.

And the mobile terminal responds to the dragging operation to trigger the amplifying assembly, and the amplifying assembly is displayed in the display screen.

In this embodiment, the triggering operation includes a drag operation, and the mobile terminal may directly respond to the drag operation to display the amplifying assembly in the display screen, where the triggering mode of the amplifying assembly is simple, and only one finger is required to implement the triggering operation without additional operation steps, thereby simplifying the operation flow and improving the man-machine interaction efficiency.

In some embodiments, the display screen of the mobile terminal includes a display area of the magnifying element, where the display area of the magnifying element is the display area where the magnifying element is located. Step 240 is optionally implemented as step 244:

step 244, in response to the drag operation, displaying the amplifying assembly in an amplifying assembly display area of the display screen; wherein the magnification assembly display area does not intersect with the operation position of the drag operation.

The mobile terminal displays the amplifying assembly in an amplifying assembly display area of the display screen in response to the drag operation.

Optionally, the magnification component display area does not intersect the operational position of the drag operation. For avoiding the magnification assembly from blocking the drag operation.

Optionally, the display screen of the mobile terminal includes one of a landscape display mode and a portrait display mode.

The transverse screen display mode refers to that a display screen of the mobile terminal is in a transverse state, at the moment, the occupied area of a local display area of a remote operation interface on the display screen is larger, and the occupied areas of an edge display area and an operation control display area on the display screen are smaller. The landscape display mode may refer to 4 kinds of schematic diagrams shown in fig. 2.

The vertical screen display mode refers to that a display screen of the mobile terminal is in a vertical state, at the moment, the occupied area of a local display area of a remote operation interface on the display screen is small, and the occupied areas of an edge display area and an operation control display area on the display screen are large. The portrait display mode may refer to the schematic diagram of (5) in fig. 4.

In some embodiments, a sensor is provided in the mobile terminal for monitoring a screen direction change angle of the mobile terminal, and determining a current display mode of the mobile terminal according to the screen direction change angle.

In particular, the sensor may be at least one of a gravity sensor, an angle sensor, a magnetic azimuth sensor. The value of the screen direction change angle obtained by the sensor is 0-359 degrees, if the value is 0 degrees, the mobile terminal is in a positive vertical screen display mode, if the value is 180 degrees, the mobile terminal is in a reverse vertical screen display mode, if the value is 90 degrees, the mobile terminal is in a left-side upward horizontal screen display mode, and if the value is 270 degrees, the mobile terminal is in a right-side upward horizontal screen display mode.

In some embodiments, step 240 specifically includes step 245 or step 246 based on different display modes of the display screen of the mobile terminal:

and step 245, in response to the drag operation of the display screen in the transverse screen display mode, displaying the amplifying assembly in a local display area corresponding to the remote operation interface in the display screen.

Alternatively, the amplifying component may be located in the local display area corresponding to the remote operation interface in the display screen, and may also be located in part in the local display area corresponding to the remote operation interface in the display screen.

In other embodiments, in the case where the display screen is in the landscape display mode, the amplifying component may be displayed in the border display area in the display screen when the area of the border display area is sufficiently large.

For example, as shown in (3) in fig. 2, the display screen of the mobile terminal is in a landscape display mode, and in response to a drag operation 142 in which the display screen is in the landscape display mode, the mobile terminal displays the amplifying component 144 in the local display area 140 corresponding to the remote operation interface in the display screen, where the amplifying component 144 is located completely within the local display area 144. As shown in (4) of fig. 2, the display screen of the mobile terminal is in a landscape display mode, and in response to a drag operation 152 in the landscape display mode, the mobile terminal displays the amplifying component 154 in a local display area 154 corresponding to the remote operation interface in the display screen, where a part of the amplifying component 154 is located in the local display area 150 and a part of the amplifying component is located in the edge display area 151 beyond the local display area 150.

In response to the drag operation in the portrait display mode, the magnifying assembly is displayed in an edge display area of the display screen other than the remote operation interface, step 246.

Alternatively, the amplifying assembly may be located entirely in the edge display area in the display screen, or may be located partially in the edge display area in the display screen.

In other embodiments, when the display screen is in the portrait display mode and the area of the local display area corresponding to the remote operation interface is sufficiently large, the amplifying component may also be displayed in the local display area corresponding to the remote operation interface in the display screen.

For example, in the process of remote control of the mobile terminal by the remote terminal, as shown in fig. 6, the display screen of the mobile terminal is in a portrait display mode, and the display screen includes an edge display area 261, a local display area 262 corresponding to the remote operation interface, and an operation control display area 263. Wherein the edge display region 261 is located directly above the partial display region 262 and the operation control display region 263 is located directly below the partial display region 262. In response to a drag operation 264 in the vertical screen display mode, the mobile terminal displays an amplifying component 265 in an edge display area 261 except for a local display area 262 corresponding to a remote operation interface in the display screen.

In the embodiment, the display modes of the amplifying assembly are enriched by displaying the amplifying assembly in different display areas of the amplifying assembly under different display modes of the display screen, so that the display flexibility of the amplifying assembly is improved.

Fig. 7 shows a flowchart of a mobile terminal-based remote control method according to an exemplary embodiment of the present application. In some embodiments, the magnifying element may automatically avoid the current operation position, so as to avoid the blocking of the operation area, i.e. the specific position of the display area of the magnifying element may be moved. Such as: the finger is touched on the left side of the display screen, the amplifying component is displayed on the right side of the finger, and the finger is touched on the right side of the display screen, the amplifying component is displayed on the left side of the finger.

Illustratively, after step 260, the method may optionally further include steps 282 and 284:

in step 282, in response to the movement operation, the display area of the amplifying assembly of the display screen is switched.

Step 284, displaying the amplifying assembly in the switched amplifying assembly display area; wherein the magnification assembly display area does not intersect the operation position of the movement operation.

Alternatively, the movement operation may be implemented based on a touch screen mode, or based on a pointer mode. The movement operation based on the touch screen mode refers to an operation of moving by a user through a finger in the touch screen mode, and the drag operation based on the pointer mode refers to an operation of moving by a user through a finger in the pointer mode.

Optionally, after the mobile terminal displays the amplifying assembly, the amplifying assembly display area of the display screen may be further switched in response to the moving operation. And displaying the amplifying assembly in the display area of the amplifying assembly after switching.

Optionally, the magnification assembly display area does not intersect the operational position of the movement operation. For avoiding the magnification assembly from blocking the movement operation.

In some embodiments, the move operation and the drag operation may be two operations. Such as: the user first performs a drag operation, the mobile terminal responds to the drag operation and displays the amplifying assembly, then, the user performs a moving operation, the mobile terminal responds to the moving operation, switches the displaying area of the amplifying assembly along with the operating position of the moving operation, and displays the amplifying assembly in the displaying area of the amplifying assembly after the switching.

In other embodiments, the movement operation and the drag operation may be the same operation, for example, the movement operation may be a second half of the drag operation. Such as: the drag operation corresponds to a press event and a move event, and the move operation may be an operation corresponding to the move event. And the user executes the drag operation, the mobile terminal responds to the drag operation and displays the amplifying assembly, then the mobile terminal switches the display area of the amplifying assembly along with the operation position of the drag operation, and the amplifying assembly is displayed in the display area of the amplifying assembly after the switch.

For example, referring to (3) (4) of fig. 2, the user initiates the trigger operation 142 at the position 142, and then the finger is moved from the position 142 to the position 152, and the mobile terminal 110 switches the magnification assembly display area from the right side of the position 142 to the left side of the position 152 in response to the movement operation. The amplification unit 154 is displayed in the switched amplification unit display area, and accordingly, the content of the amplification interface is also switched, and the amplified display content in which the display content "YYYyy" is amplified by N times of amplification is displayed in the amplification unit 154, where N is greater than 1.

In this embodiment, the display area of the amplifying assembly can be switched according to the movement of the finger of the user, so that the amplifying assembly can not shield the current operation area of the user, and the user can reduce the misoperation touch during remote operation, thereby improving the accuracy of remote control.

In some embodiments, according to the screen size of the display screen of the mobile terminal, the display areas of the display screen are divided in advance, so that at least two display areas are obtained by dividing. It should be noted that at least two display areas divided as described above are not displayed on the display screen. Such as: the display area of the display screen is divided into 2 areas or 4 areas based on the center point of the display screen in advance, and the division granularity is not limited in the embodiment, and can be determined according to the required operation precision. Then, the mobile terminal can switch the display area of the amplifying assembly where the amplifying assembly is located according to the current operation area where the mobile operation is located, so as to avoid the amplifying assembly from shielding the operation area.

Fig. 8 is a flowchart illustrating a remote control method based on a mobile terminal according to an exemplary embodiment of the present application. Step 282 described above may be optionally implemented as step 320 and step 340:

in step 320, in response to the movement operation, a current operation area in which the movement operation is located is determined.

Step 340, switching the display area of the amplifying assembly where the amplifying assembly is located based on the current operation area and at least two display areas obtained by dividing the display screen; the display area of the amplifying assembly and the current operation area are different display areas of the display screen.

In an exemplary embodiment, the mobile terminal responds to the mobile operation, and determines a current operation area where the mobile operation is located in real time based on a coordinate position corresponding to the mobile operation. And switching the display area of the amplifying assembly where the amplifying assembly is positioned based on the current operation area and at least two display areas obtained by dividing the display screen in advance.

In some embodiments, the magnification assembly display area and the current operating area are different display areas of the display screen.

Taking the example of equally dividing the display area of the display screen into 2 areas, for example: the display screen is divided into a left area and a right area of the same size in advance based on the center point of the display screen. And the mobile terminal responds to the moving operation, determines that the current operation area where the moving operation is positioned is a left area, and displays the amplifying assembly in the right area if the amplifying assembly display area is a right area. And as the position indicated by the moving operation is changed, the current display area where the moving operation is positioned is moved to the right area, the display area of the amplifying assembly is the left area, and the amplifying assembly is displayed in the left area.

Taking the example of equally dividing the display area of the display screen into 4 areas, for example: based on the center point of the display screen, the display screen is divided into an upper left side area, a lower left side area, an upper right side area and a lower right side area with the same size in advance. And the mobile terminal responds to the moving operation, and determines that the current operation area where the moving operation is positioned is a left lower side area, and the display area of the amplifying assembly can be one of the left upper side area, the right upper side area and the right lower side area, so that the amplifying assembly is displayed in the display area of the amplifying assembly. With the position of the movement operation instruction changed, assuming that the current display area where the movement operation is located moves to the upper right side area, the magnification assembly display area may be switched to one of the upper left side area, the lower left side area, and the lower right side area, and the magnification assembly may be displayed in the above-described switched magnification assembly display area.

In this embodiment, the display screen is divided into a plurality of display areas in advance, so that the current operation area of the user can be accurately determined, the display area of the amplifying assembly corresponding to the amplifying assembly avoids the current operation area, and shielding of the amplifying assembly is avoided. In addition, as the user may operate in other operation areas around the current operation area, the manner of the embodiment can also avoid the amplifying assembly from shielding the possible operation areas, so that the man-machine interaction efficiency is further improved.

In some embodiments, the user may employ a two-finger zoom-in operation interface, and when the mobile terminal displays the zoom-in component, the zoom-in component will automatically cancel if the user employs the two-finger zoom-in operation interface. For example, with continued reference to fig. 7, after step 260, or after step 284, the method may optionally further include step 286:

in step 286, the display magnification assembly is canceled in response to the double-finger magnification operation.

The double-finger amplifying operation is an operation of amplifying the remote operation interface by adopting double fingers.

The mobile terminal cancels the display amplifying assembly in response to the double-finger amplifying operation.

In this embodiment, when the user adopts the same manner as the related art and adopts the double-finger manual amplification remote operation interface, the mobile terminal cancels the display amplification assembly, so that the resource occupation of the mobile terminal can be reduced. The area occupied by the amplifying assembly in the display screen of the mobile terminal can be saved, and more interface contents of the remote operation display interface can be displayed.

In some embodiments, the coordinate system is established in advance from the display screen of the mobile terminal. Optionally, a coordinate system is established with the upper right corner of the display screen of the mobile terminal as the origin of coordinates, the width (width) of the display screen of the mobile terminal as the horizontal axis, and the height (height) of the display screen of the mobile terminal as the vertical axis. Wherein the unit of coordinates is a pixel. The triggering operation, the dragging operation and the moving operation can be processed according to the coordinate system.

In particular, fig. 9 shows a flowchart of a mobile terminal-based remote control method according to an exemplary embodiment of the present application. The background implementation of step 260 may be step 420, step 440, and step 460:

step 420, receiving a pressing event, and acquiring a first coordinate indicated by the pressing event.

The press event includes an event of pressing by a finger touch.

The coordinates of the position point at the pressing time when the pressing event is received are determined as first coordinates.

Illustratively, the mobile terminal receives a press event and obtains a first coordinate indicated by the press event.

Step 440, in case a movement event is received after the pressing event is received, obtaining a second coordinate indicated by the movement event.

In some embodiments, the mobile terminal determines that the mobile event is received after the push event is received when the duration of the push event exceeds a duration threshold and the coordinate position of the first coordinate is detected to be different from the coordinate position of the release.

The movement event refers to an event of movement by a finger touch.

The coordinates of the position point at the release time of the movement event are determined as second coordinates.

In an exemplary case, the mobile terminal receives the movement event after receiving the pressing event, acquires the second coordinate indicated by the movement event.

Since the finger has a certain area, the coordinates indicated by the pressing event and the moving event are generally determined based on the geometric center of a touch area of the finger.

Step 460, determining that a triggering operation is received and displaying the amplifying component in the display screen when the triggering condition is met between the first coordinate and the second coordinate.

The trigger condition refers to a condition that determines that the currently performed operation is a drag operation that needs to be satisfied.

The trigger condition is determined based on the first coordinates of the press event and the second coordinates of the move event.

In some embodiments, a distance between the first and second coordinates is determined based on the first and second coordinates, and the trigger operation is determined to be received if the distance is greater than or equal to a distance threshold.

For example, a first coordinate of the pressing event is expressed as (x 1, y 1), a second coordinate of the moving event received after the pressing event is expressed as (x 2, y 2), and (x 2-x 1) is satisfied ² +(y2-y1) ² >100, the mobile terminal determines that the receiving is a triggering operation, namely a dragging operation, and displays the amplifying component in the display screen, otherwise, the mobile terminal does not process to continuously receive the screen touch event.

In some embodiments, the touch event monitoring in the mobile terminal may take a system of touch events having an action value indicating what the touch event is specifically. In this embodiment, the ACTION value is concerned with action_up/action_down/action_cancel/action_move.

The action_up and the action_cancel can be regarded as movement behavior when the finger is released, the action_down is pressed, and the action_move is pressed.

In general, the behavior of a sliding screen is "press-move-release", and the behavior of a long press screen is "press-release". In this embodiment, since the finger slightly moves when a part of the user presses down, the movement is calculated by determining that the movement is greater than or equal to a certain pixel distance.

In the embodiment, the mobile terminal can accurately determine whether the current operation of the user is the triggering operation, so that the amplifying assembly can be displayed in the display screen, and the triggering accuracy of the amplifying assembly is improved.

In some embodiments, after determining that the triggering operation is received, before displaying the amplifying component in the display screen, in order to further ensure that the amplifying component needs to be displayed in the display screen in order to avoid a display error or a situation that display is not actually needed, the step 460 further includes steps 482 and 484:

step 482, obtaining a first zoom ratio of the remote operation interface in the display screen.

In step 484, the magnification component is displayed in the display screen if the first magnification is less than or equal to the magnification threshold.

The first scaling factor refers to the current scaling factor of a remote operation interface in a display screen of the mobile terminal after the mobile terminal determines that the triggering operation is received.

The magnification threshold is a parameter value set in advance, and is set to 1.0 in this embodiment.

The mobile terminal obtains a first zoom magnification of a remote operation interface in the display screen, and determines that the amplifying assembly is required to be displayed when the first zoom magnification is smaller than or equal to a magnification threshold value, and displays the amplifying assembly in the display screen.

In some embodiments, the zoom magnification is determined according to a ratio between a width and a height of a current screen of a remote operation interface of a display screen of the mobile terminal and a width and a height of a default screen. The default frame width and height need to be initialized according to the resolution of the mobile phone.

In this embodiment, whether the display amplifying assembly is needed or not may be further determined by the first zoom ratio, so as to avoid a display error or a situation that display is not needed in practice, and thus resources of the mobile terminal may be saved.

Fig. 10 shows a flowchart of a mobile terminal-based remote control method according to an exemplary embodiment of the present application. After step 260, the mobile terminal also needs to determine interface contents to be displayed in the zoom-in component before step 280, in order to display the zoomed-in interface contents in step 280. In some embodiments, referring to fig. 10, the method further comprises step 520 and step 540:

Step 520, a current display mode of the display screen is obtained.

The display mode of the display screen includes one of a landscape display mode and a portrait display mode.

Based on the foregoing embodiments, in different display modes, the display areas of the amplifying assemblies corresponding to the amplifying assemblies are different, and the sizes of the amplifying assemblies are different. Thus, the mobile terminal acquires the current display mode of the display screen.

Step 540, determining interface content displayed in the magnification component based on the display mode and at least one of the first and second coordinates of the trigger operation indication.

The mobile terminal determines interface contents displayed in the magnification component based on the display mode and at least one of the first coordinate and the second coordinate of the trigger operation instruction, and then displays the magnification component in the magnification component display area and displays the magnified interface contents in the magnification component.

The following embodiments are described with respect to different display modes and different types of trigger operations.

1. The display mode includes a landscape display mode, and the trigger operation includes a trigger operation based on a touch screen mode.

Illustratively, the step 540 includes a step 541:

in step 541, in the landscape display mode, the second coordinate indicated by the triggering operation is taken as the center position of the region to be amplified, the preset size is taken as the region size of the region to be amplified, and the interface content displayed in the amplifying assembly is determined.

Alternatively, the preset size may be set according to actual technical needs, and in this embodiment, the amplifying component is set to be circular, and the radius is 100DP. The second coordinate of the trigger operation is denoted as (x 2, y 2). The interface content in the amplifying assembly is the interface content corresponding to the area size of the area to be amplified with the radius of 100DP by taking (x 2, y 2) as the center position of the area to be amplified.

For example, as shown in fig. 11, the triggering operation is described by taking a case where the display screen of the mobile terminal is in a landscape display mode and the triggering operation is based on the triggering operation of the touch screen mode. The upper right corner of the display screen of the mobile terminal is taken as an origin of coordinates, the width (width) of the display screen of the mobile terminal is taken as a horizontal axis, and the height (height) of the display screen of the mobile terminal is taken as a vertical axis, so that a coordinate system is established. The rectangular ABCD in fig. 11 is a remote operation interface and the line segment EF is the midline of the rectangular ABCD, such that the display screen is divided into two equally sized display areas. The trigger operation is located at a position point P when pressed, i.e., a first coordinate is denoted as P (x 1, y 1), and the finger is dragged from the pressed position point P to a position point Q, i.e., a second coordinate is denoted as Q (x 2, y 2). The center position of the region to be enlarged is (x 2, y 2). The interface content to be drawn in the amplifying assembly is the interface content corresponding to the circle with the radius of 100DP by taking (x 2, y 2) as the center, and the amplifying assembly is filled with the amplified interface content which is amplified by N times. If the finger is dragged on the right side of the screen, the magnifying component is drawn on the left side of the screen, otherwise if the finger is dragged on the left side of the screen, the magnifying component is drawn on the right side of the screen. The abscissa x2 of the second coordinate is compared with the width of the display screen (i.e. the horizontal length of the display mode of the transverse screen), if x2> width, the screen is dragged on the right side, otherwise the screen is dragged on the left side.

2. The display mode includes a portrait display mode and the trigger operation includes a trigger operation based on a touch screen mode.

Illustratively, the step 540 includes steps 542 and 543:

step 542, in the vertical screen display mode, determining the area size of the area to be enlarged based on the screen size of the display screen and the area size of the local display area corresponding to the remote operation interface.

In the vertical screen display mode, since the magnifying member is displayed in the edge display area, it is necessary to determine the area size of the area to be magnified based on the size of the edge display area.

Optionally, determining the height between the top of the display screen and the top of the remote operation interface to obtain the height of the edge display area, and subtracting the height of the reserved space from the height of the edge display area to obtain the height of the area to be amplified. Wherein the reserved space height may be set to 10DP.

And step 543, taking the second coordinate of the triggering operation instruction as the center position of the region to be amplified, and determining the interface content displayed in the amplifying assembly based on the region size of the region to be amplified.

Optionally, the second coordinate of the trigger operation is denoted (x 2, y 2). The interface content in the amplifying assembly is the interface content corresponding to the region to be amplified by taking (x 2, y 2) as the central position of the region to be amplified and taking the calculated region size as the region size of the region to be amplified.

3. The display mode includes a landscape display mode, and the trigger operation includes a trigger operation based on a pointer mode.

Illustratively, step 540 specifically includes steps 544 and 545:

in step 544, in the landscape display mode, the center position of the region to be enlarged is determined based on the current position of the pointer, and the first and second coordinates of the trigger operation indication.

And adding the abscissa of the current position of the pointer to the abscissa of the second coordinate and subtracting the abscissa of the first coordinate to obtain the abscissa of the central position of the region to be amplified. And adding the ordinate of the current position of the pointer to the ordinate of the second coordinate and subtracting the ordinate of the first coordinate to obtain the ordinate of the central position of the region to be amplified, thereby obtaining the central position of the region to be amplified.

Alternatively, the current position of the pointer is denoted (x, y). The first coordinate is denoted (x 1, y 1) and the second coordinate is denoted (x 2, y 2). The center position of the region to be amplified is (x+x2-x 1, y+y2-y 1).

Step 545, determining interface content displayed in the amplifying assembly based on the center position of the region to be amplified by taking the preset size as the region size of the region to be amplified.

Alternatively, the preset size may be set according to actual technical needs, and in this embodiment, the amplifying component is set to be circular, and the radius is 100DP. The interface content in the amplifying assembly is the interface content corresponding to the region to be amplified with the radius of 100DP by taking (x+x2-x 1, y+y2-y 1) as the center position of the region to be amplified.

For example, as shown in fig. 12, the triggering operation is described by taking a case where the display screen of the mobile terminal is in the landscape display mode and the triggering operation based on the pointer mode as an example. The upper right corner of the display screen of the mobile terminal is taken as an origin of coordinates, the width (width) of the display screen of the mobile terminal is taken as a horizontal axis, and the height (height) of the display screen of the mobile terminal is taken as a vertical axis, so that a coordinate system is established. The rectangular ABCD in fig. 12 is a remote operation interface and the line segment EF is the midline of the rectangular ABCD, such that the display screen is divided into two equally sized display areas. The current position of the pointer is Z (x, y), and the trigger operation is located at a position point M when pressed, i.e., a first coordinate is denoted as M (x 1, y 1), and the finger is dragged from the pressed position point M to a position point N, i.e., a second coordinate is denoted as N (x 2, y 2). The center position of the region to be amplified is (x+x2-x 1, y+y2-y 1). The interface content to be drawn in the amplifying assembly is the interface content corresponding to the circle with the radius of 100DP and taking (x+x2-x 1, y+y2-y 1) as the center, and the amplifying assembly is filled with the amplified interface content which is amplified by N times. If the finger is dragged on the right side of the screen, the magnifying component is drawn on the left side of the screen, otherwise if the finger is dragged on the left side of the screen, the magnifying component is drawn on the right side of the screen.

4. The display mode includes a portrait display mode and the trigger operation includes a trigger operation based on a pointer mode.

Step 540 specifically includes step 546, step 547, and step 548:

step 546, in the vertical screen display mode, determining the center position of the region to be enlarged based on the current position of the pointer, and the first and second coordinates of the trigger operation indication.

Step 546 is determined in the same manner as step 544, and will not be described again.

Step 547, determining the area size of the area to be enlarged based on the screen size of the display screen and the area size of the local display area corresponding to the remote operation interface.

The determination in this step 547 is the same as that in step 542, and will not be described here again.

And step 548, determining the interface content displayed in the amplifying assembly according to the central position of the region to be amplified and the region size of the region to be amplified.

Optionally, the interface content in the amplifying assembly is the interface content corresponding to the region to be amplified by taking (x+x2-x 1, y+y2-y 1) as the central position of the region to be amplified and taking the calculated region size as the region size of the region to be amplified.

In some embodiments, the size of the region to be amplified may also be the size of the amplifying assembly.

In the above embodiment, in different display modes and different operation modes of the display screen, the amplifying assembly can be displayed at different positions, so that different interface contents can be displayed in the amplifying assembly, and the flexibility and the accuracy of displaying the amplifying assembly are improved.

Fig. 13 is a flowchart illustrating a mobile terminal-based remote control method according to an exemplary embodiment of the present application. Based on the above embodiments, after the step 280 displays the magnified interface content in the magnification component, the method further includes steps 620 and 640:

step 620, obtaining a second zoom magnification of the remote operation interface in the display screen.

And step 640, stopping displaying the amplifying component in the display screen and hiding the amplifying component when the second zoom magnification is larger than the magnification threshold.

The second zoom magnification is a current zoom magnification of a remote operation interface in a display screen of the mobile terminal after the content of the zoom interface is displayed in the zoom-in component.

The mobile terminal obtains a second zoom magnification of the display screen, and if the second zoom magnification is greater than the magnification threshold, the mobile terminal indicates that the user performs the double-finger zoom operation, and stops displaying the zoom-in component in the display screen and conceals the zoom-in component.

In this embodiment, whether the display amplifying component needs to be canceled or not may be further determined by the second zoom ratio, so that resources of the mobile terminal may be saved.

Fig. 14 shows a flowchart of a remote control method based on a mobile terminal according to an exemplary embodiment of the present application, and step 220 is implemented as step 720 and step 740 based on the above embodiments:

and step 720, responding to remote connection operation between the mobile terminal and the remote terminal, and acquiring screen video data corresponding to a remote operation interface of the remote terminal.

The remote terminal grabs screen video data in response to remote connection operation between the mobile terminal and the remote terminal, and transmits the screen video data to the mobile terminal through a video stream channel. The mobile terminal responds to remote connection operation between the mobile terminal and the remote terminal, and screen video data corresponding to a remote operation interface of the remote terminal is obtained.

Step 740, displaying the remote operation interface of the remote terminal in the display screen of the mobile terminal based on the screen video data corresponding to the remote operation interface of the remote terminal.

The mobile terminal renders the screen video data based on the screen video data corresponding to the remote operation interface of the remote terminal, draws the screen video data onto a bound Surface View, and displays the remote operation interface of the remote terminal in a display screen of the mobile terminal.

In this embodiment, the mobile terminal may successfully render and display the remote operation interface on the display screen of the mobile terminal by acquiring the screen video data of the remote terminal, thereby improving the display accuracy of the remote operation interface.

In some embodiments, fig. 15 shows a schematic diagram of a remote control method based on a mobile terminal according to an exemplary embodiment of the present application. The mobile terminal is used as a control terminal, in particular a mobile phone; the remote terminal is a controlled terminal, and in particular, a computer is illustrated as an example. The mobile terminal and the remote terminal are provided with the same client supporting the remote desktop function, and the mobile terminal can remotely control the remote terminal by simultaneously entering the remote desktop function of the client. The method provided in this embodiment is described in detail below with reference to the schematic diagram. The method comprises the following steps:

1. the mobile terminal and the remote terminal respectively enter the remote desktop function of the client.

2. The method comprises the steps of acquiring and processing pre-processing data, wherein the pre-processing data comprise screen video data of a remote terminal, screen orientation data of a mobile terminal, coordinate data of a trigger event and the like, and the acquisition and processing of the data specifically comprise the following sub-steps:

2-1, capturing screen video data by a remote terminal, and transmitting the screen video data to a client of a mobile terminal through a video stream channel; after the client of the mobile terminal acquires the screen video data, rendering the content of the whole computer screen of the remote terminal, and drawing the video data onto the bound Surface View;

2-2, the mobile terminal establishes a coordinate system based on a display screen, wherein the coordinate of the upper left corner of the display screen is (0, 0), and the coordinate of the lower right corner of the display screen is (screen width height);

2-3, the mobile terminal obtains the screen coordinates x and y (unit: pixel) of the current virtual pointer;

2-4, the mobile terminal obtains the current screen orientation of the display screen as a horizontal screen/vertical screen;

2-5, the mobile terminal acquires the trigger event of the current screen, if the touch press event is acquired for more than 500 milliseconds (ms), and the coordinates (x 2, y 2) of the touch movement event received after the press and the coordinates (x 1, y 1) acquired when the press are performed. If satisfied (x 2-x 1) ² +(y2-y1) ² >100, the mobile terminal judges that the user performs the drag operation currently, otherwise, the mobile terminal does not perform processing and continues to accept the touch event. And the virtual pointer is drawn to translate (x 2-x1, y2-y 1) during the triggering event.

3. The mobile terminal obtains the zoom ratio of the current video rendering interface; when the zoom magnification is 1.0, the magnifying glass (display magnifying component) can be accessed, step 5 is executed, when the zoom magnification is more than 1.0, the magnifying glass is hidden, and step 4 is executed.

4. The mobile terminal conceals the magnifying glass.

5. The mobile terminal judges whether the display screen is a horizontal screen or not; if yes, executing step 6, otherwise executing step 9.

6. The mobile terminal acquires a current control mode; step 7 is performed in pointer mode and step 8 is performed in touch screen mode.

7. The display screen of the mobile terminal displays the cross screen pointer magnifying glass, which comprises the following steps:

7-1, if the current display screen is in a horizontal screen state, judging the size between the coordinate x2 and the screen width (namely the horizontal length of the horizontal screen) of the mobile terminal in the horizontal screen state, if x2> width, dragging the mobile terminal on the right side of the screen, and if x2> width, dragging the mobile terminal on the left side of the screen.

7-2, in the horizontal screen pointer mode, the image content drawn in the magnifying glass is a circle with the radius of 100DP by taking the current position (x+x2-x 1, y+y2-y 1) of the pointer as the center, and the circle is filled with a picture rendered by video stream data of a corresponding screen area which is amplified N times. If the finger is dragged on the right side of the screen, drawing a magnifier image on the left side of the screen; otherwise, if the finger is dragged on the left side of the screen, drawing a magnifier image on the right side of the screen. The image content is rendered on a new View (View) and superimposed over the original Surface View.

8. The method for displaying the transverse screen touch screen magnifier in the display screen of the mobile terminal specifically comprises the following sub-steps:

8-1, the judgment content is the same as that of the step 7-1.

8-2 in the horizontal screen touch screen mode, the image content drawn in the magnifier is a circle with the radius of 100DP centered on the touch position (x 2, y 2), and the circle is filled with a picture rendered by video stream data of the corresponding screen area which is amplified N times. If the finger is dragged on the right side of the screen, drawing a magnifier image on the left side of the screen; otherwise, if the finger is dragged on the left side of the screen, drawing a magnifier image on the right side of the screen. The image content is rendered on a new View (View) and superimposed over the original Surface View.

9. The mobile terminal acquires a current control mode; step 10 is performed in pointer mode and step 11 is performed in touch screen mode.

10. Displaying a vertical screen pointer magnifying glass in a display screen of a mobile terminal, which comprises the following steps:

in the vertical screen pointer mode, the computer screen content of the currently drawn remote terminal is centered in the display screen, and the height3 for drawing the magnifier image is: the height of the height 1/2 of the Surface View of the drawing computer screen is subtracted from the screen top of the display screen to (the screen height of the display screen), the reserved gaps of 10DP up and down are subtracted, and the width3 of the drawing magnifier image is the same as the height 3. The image content drawn in the magnifier is a circle with the current position (x+x2-x 1, y+y2-y 1) of the pointer as the center and the radius of height3/2, and the circle is filled with a picture rendered by video stream data of a corresponding screen area with N times of magnification.

11. The vertical screen touch screen magnifier is displayed in a display screen of the mobile terminal, and specifically comprises the following steps:

in the vertical screen touch screen mode, the computer screen content of the remote terminal which is currently drawn is centered in the display screen, and the height3 for drawing the magnifier image is as follows: the height of the height 1/2 of the Surface View of the drawing computer screen is subtracted from the screen top of the display screen to (the screen height of the display screen), the reserved gaps of 10DP up and down are subtracted, and the width3 of the drawing magnifier image is the same as the height 3. The image content drawn in the magnifier is a circle with the current position (x 2, y 2) touched by the finger as the center and the radius of height3/2, and the circle is filled with a picture rendered by video stream data of the corresponding screen area which is amplified by N times.

12. And when the zoom magnification of the drawn computer screen content is detected to be larger than 1.0, stopping drawing the magnifier and executing the step 4 to hide the magnifier.

In summary, by adopting the method provided by the embodiment of the application, the user is not blocked by fingers when remotely operating, the operation step of amplifying the interface content is simplified, the man-machine interaction efficiency is improved, the situation of false touch when the user uses can be reduced, the accurate remote control operation can be realized, the operations such as window dragging and file dragging can be accurately performed, and the accuracy of remote control is improved.

Fig. 16 is a block diagram illustrating a remote control apparatus based on a mobile terminal according to an exemplary embodiment of the present application. A mobile terminal-based remote control 800 comprising:

and a display module 810 for displaying a remote operation interface of the remote terminal in a display screen of the mobile terminal in response to a remote connection operation between the mobile terminal and the remote terminal.

The display module 810 is configured to respond to a triggering operation, and display an amplifying component in the display screen, where the amplifying component is configured to preview the interface content of the remote operation interface with N times of magnification, where N is greater than 1.

The display module 810 is configured to display, in the amplifying component, an amplifying interface content corresponding to the triggering operation, where the amplifying interface content is associated with an operation position indicated by the triggering operation.

In some embodiments, the triggering operation includes a drag operation, where the drag operation refers to an operation that the mobile terminal receives a movement event after receiving a pressing event; the display module 810 is configured to:

and displaying the amplifying component in the display screen in response to the drag operation.

In some embodiments, the display screen includes an amplifying assembly display area, where the amplifying assembly display area refers to a display area where the amplifying assembly is located; the display module 810 is configured to:

displaying the amplifying assembly in the amplifying assembly display area of the display screen in response to the drag operation;

wherein the magnification assembly display area does not intersect with the operation position of the drag operation.

In some embodiments, the display module 810 is configured to:

and responding to the dragging operation of the display screen in a transverse screen display mode, and displaying the amplifying assembly in a local display area corresponding to the remote operation interface in the display screen.

In some embodiments, the display module 810 is configured to:

and responding to the drag operation of the display screen in a vertical screen display mode, and displaying the amplifying assembly in an edge display area except for the remote operation interface in the display screen.

In some embodiments, the display module 810 is configured to:

switching the display area of the amplifying assembly of the display screen in response to a moving operation;

displaying the amplifying assembly in the display area of the amplifying assembly after switching;

Wherein the magnification assembly display area does not intersect with the operation position of the movement operation.

In some embodiments, the display module 810 is configured to:

responding to the moving operation, and determining a current operation area in which the moving operation is positioned;

switching the display area of the amplifying assembly where the amplifying assembly is located based on the current operation area and at least two display areas obtained by dividing the display screen;

the display area of the amplifying assembly and the current operation area are different display areas of the display screen.

In some embodiments, the display module 810 is configured to:

and in response to the double-finger amplifying operation, the amplifying assembly is not displayed.

In some embodiments, the display module 810 is configured to:

receiving a pressing event, and acquiring a first coordinate indicated by the pressing event;

under the condition that a moving event is received after the pressing event is received, acquiring a second coordinate indicated by the moving event;

and under the condition that the triggering condition is met between the first coordinate and the second coordinate, determining that the triggering operation is received, and displaying the amplifying component in the display screen.

In some embodiments, the display module 810 is configured to:

acquiring a first zoom ratio of the remote operation interface in the display screen;

and displaying the amplifying component in the display screen under the condition that the first scaling multiplying power is smaller than or equal to a multiplying power threshold value.

In some embodiments, the display module 810 is configured to:

acquiring a current display mode of the display screen;

and determining interface content displayed in the amplifying component based on the display mode and at least one of the first coordinate and the second coordinate of the trigger operation indication.

In some embodiments, the display mode comprises a cross screen display mode, and the trigger operation comprises a touch screen mode-based trigger operation; the display module 810 is configured to:

and in the transverse screen display mode, taking the second coordinate indicated by the triggering operation as the center position of the area to be amplified, taking the preset size as the area size of the area to be amplified, and determining the interface content displayed in the amplifying assembly.

In some embodiments, the display mode includes a portrait display mode and the trigger operation includes a touch screen mode based trigger operation; the display module 810 is configured to:

In the vertical screen display mode, determining the area size of an area to be amplified based on the screen size of the display screen and the area size of a local display area corresponding to the remote operation interface;

and taking the second coordinate indicated by the triggering operation as the center position of the region to be amplified, and determining the interface content displayed in the amplifying assembly based on the region size of the region to be amplified.

In some embodiments, the display mode comprises a cross screen display mode, and the trigger operation comprises a pointer mode-based trigger operation; the display module 810 is configured to:

in the horizontal screen display mode, determining the central position of the region to be amplified based on the current position of the pointer and the first coordinate and the second coordinate indicated by the triggering operation;

and taking the preset size as the area size of the area to be amplified, and determining the interface content displayed in the amplifying assembly based on the central position of the area to be amplified.

In some embodiments, the display mode comprises a portrait display mode and the trigger operation comprises a pointer mode based trigger operation; the display module 810 is configured to:

In the vertical screen display mode, determining the central position of the region to be amplified based on the current position of the pointer and the first coordinate and the second coordinate indicated by the triggering operation;

determining the area size of the area to be amplified based on the screen size of the display screen and the area size of the local display area corresponding to the remote operation interface;

and determining interface content displayed in the amplifying assembly according to the central position of the region to be amplified and the region size of the region to be amplified.

In some embodiments, the display module 810 is configured to:

acquiring a second zoom ratio of the remote operation interface in the display screen;

and stopping displaying the amplifying component in the display screen and hiding the amplifying component under the condition that the second scaling multiplying power is larger than the multiplying power threshold value.

In some embodiments, the display module 810 is configured to:

responding to the remote connection operation between the mobile terminal and the remote terminal, and acquiring screen video data corresponding to the remote operation interface of the remote terminal;

and displaying the remote operation interface of the remote terminal in the display screen of the mobile terminal based on the screen video data corresponding to the remote operation interface of the remote terminal.

It should be noted that, the specific limitation in the embodiments of the one or more remote control devices based on mobile terminals provided above may be referred to above as limitation of the remote control method based on mobile terminals, which is not repeated herein. The modules of the above device may be implemented in whole or in part by software, hardware, or a combination thereof, and each module may be embedded in hardware form or independent of a processor of the computer device, or may be stored in a memory of the computer device in software form, so that the processor may call and execute operations corresponding to each module.

Fig. 17 is a block diagram showing a structure of a computer device according to an exemplary embodiment of the present application.

The computer device 1000 may be a portable mobile terminal, also referred to as a mobile terminal in this embodiment. Such as: smart phones, tablet computers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg 3), MP4 (Moving Picture Experts Group Audio Layer IV, mpeg 4) players. The computer device 1000 may also be referred to by other names of user devices, portable terminals, etc.

In general, the computer device 1000 includes: a processor 1001 and a memory 1002.

The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 1001 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 1001 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 1002 may include one or more computer-readable storage media, which may be tangible and non-transitory. Memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1002 is used to store at least one instruction for execution by processor 1001 to implement the mobile terminal-based remote control method provided in embodiments of the present application.

In some embodiments, the computer device 1000 may further optionally include: a peripheral interface 1003, and at least one peripheral. Specifically, the peripheral device includes: at least one of radio frequency circuitry 1004, touch display 1005, camera 1006, audio circuitry 1007, and power supply 1008.

Peripheral interface 1003 may be used to connect I/O (Input/Output) related at least one peripheral to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002, and peripheral interface 1003 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 1001, memory 1002, and peripheral interface 1003 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

Radio Frequency circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. Radio frequency circuitry 1004 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 1004 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, etc. Radio frequency circuitry 1004 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 1004 may also include NFC (Near Field Communication ) related circuitry, which is not limiting of the application.

The touch display 1005 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. The touch display 1005 also has the ability to capture touch signals at or above the surface of the touch display 1005. The touch signal may be input to the processor 1001 as a control signal for processing. Touch display 1005 is used to provide virtual buttons and/or virtual keyboards, also known as soft buttons and/or soft keyboards. In some embodiments, the touch display 1005 may be one, providing a front panel of the computer device 1000; in other embodiments, the touch display 1005 may be at least two, respectively disposed on different surfaces of the computer device 1000 or in a folded design; in some embodiments, touch display 1005 may be a flexible display disposed on a curved surface or a folded surface of computer device 1000. Even more, the touch display 1005 may be arranged in an irregular pattern other than rectangular, i.e., a shaped screen. The touch display 1005 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 1006 is used to capture images or video. Optionally, camera assembly 1006 includes a front camera and a rear camera. In general, a front camera is used for realizing video call or self-photographing, and a rear camera is used for realizing photographing of pictures or videos. In some embodiments, the number of the rear cameras is at least two, and the rear cameras are any one of a main camera, a depth camera and a wide-angle camera, so as to realize fusion of the main camera and the depth camera to realize a background blurring function, and fusion of the main camera and the wide-angle camera to realize a panoramic shooting function and a Virtual Reality (VR) shooting function. In some embodiments, camera assembly 1006 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

Audio circuitry 1007 is used to provide an audio interface between the user and computer device 1000. The audio circuit 1007 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and environments, converting the sound waves into electric signals, and inputting the electric signals to the processor 1001 for processing, or inputting the electric signals to the radio frequency circuit 1004 for voice communication. For purposes of stereo acquisition or noise reduction, the microphone may be multiple, each disposed at a different location of the computer device 1000. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, audio circuit 1007 may also include a headphone jack.

The power supply 1008 is used to power the various components in the computer device 1000. The power supply 1008 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1008 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the computer device 1000 also includes one or more sensors 1009. The one or more sensors 1009 include, but are not limited to: acceleration sensor 1010, gyro sensor 1011, pressure sensor 1012, optical sensor 1013, and proximity sensor 1014.

The acceleration sensor 1010 can detect the magnitudes of accelerations on three coordinate axes of a coordinate system established with the computer device 1000. For example, the acceleration sensor 1010 may be used to detect components of gravitational acceleration in three coordinate axes. The processor 1001 may control the touch display 1005 to display a user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 1010. The acceleration sensor 1010 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 1011 may detect a body direction and a rotation angle of the computer apparatus 1000, and the gyro sensor 1011 may collect a 3D motion of the user on the computer apparatus 1000 in cooperation with the acceleration sensor 1010. The processor 1001 may implement the following functions according to the data collected by the gyro sensor 1011: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

Pressure sensor 1012 may be disposed on a side frame of computer device 1000 and/or on an underlying layer of touch display 1005. When the pressure sensor 1012 is provided at a side frame of the computer device 1000, a grip signal of the computer device 1000 by a user may be detected, and left-right hand recognition or quick operation may be performed according to the grip signal. When the pressure sensor 1012 is disposed at the lower layer of the touch display screen 1005, control of the operability control on the UI interface can be achieved according to the pressure operation of the user on the touch display screen 1005. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The optical sensor 1013 is used to collect the intensity of the ambient light. In one embodiment, the processor 1001 may control the display brightness of the touch display 1005 based on the ambient light intensity collected by the optical sensor 1013. Specifically, when the intensity of the ambient light is high, the display brightness of the touch display screen 1005 is turned up; when the ambient light intensity is low, the display brightness of the touch display screen 1005 is turned down. In another embodiment, the processor 1001 may also dynamically adjust the shooting parameters of the camera module 1006 according to the ambient light intensity collected by the optical sensor 1013.

A proximity sensor 1014, also known as a distance sensor, is typically provided on the front of the computer device 1000. The proximity sensor 1014 is used to collect the distance between the user and the front of the computer device 1000. In one embodiment, when the proximity sensor 1014 detects a gradual decrease in the distance between the user and the front of the computer device 1000, the processor 1001 controls the touch display 1005 to switch from the bright screen state to the off screen state; when the proximity sensor 1014 detects that the distance between the user and the front surface of the computer device 1000 gradually increases, the touch display screen 1005 is controlled by the processor 1001 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the architecture shown in fig. 17 is not limiting of the computer device and may include more or fewer components than shown, or may combine certain components, or employ a different arrangement of components.

The application provides a chip which comprises programmable logic circuits and/or program instructions and is used for realizing the remote control method based on the mobile terminal provided by the embodiment of the method when the chip runs on computer equipment.

The present application provides a computer readable storage medium storing a computer program loaded and executed by a processor to implement the mobile terminal-based remote control method provided by the above method embodiments.

The present application provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the processor of the computer device loads and executes the computer instructions to implement the remote control method based on the mobile terminal provided by the method embodiment.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

Those of ordinary skill in the art will appreciate that all or a portion of the steps implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the above mentioned computer readable storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Those skilled in the art will appreciate that in one or more of the examples described above, the functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.

Claims

1. A mobile terminal-based remote control method, the method being performed by the mobile terminal, the method comprising:

2. The method according to claim 1, wherein the triggering operation includes a drag operation, the drag operation being an operation in which the mobile terminal receives a movement event after receiving a pressing event;

The responding to the triggering operation displays an amplifying component in the display screen, comprising:

3. The method of claim 2, wherein the display screen comprises an amplifying assembly display area, the amplifying assembly display area being a display area in which the amplifying assembly is located;

the displaying the amplifying component in the display screen in response to the drag operation includes:

4. The method of claim 3, wherein the displaying the magnification component in the magnification component display area of the display screen in response to the drag operation comprises:

5. The method of claim 3, wherein the displaying the magnification component in the magnification component display area of the display screen in response to the drag operation comprises:

6. The method according to any one of claims 1 to 5, further comprising:

7. The method of claim 6, wherein the switching the magnification component display area of the display screen in response to a movement operation comprises:

8. The method according to any one of claims 1 to 7, further comprising:

9. The method of any one of claims 1 to 7, wherein displaying an amplification assembly in the display screen in response to a triggering operation comprises:

10. The method according to claim 9, wherein the method further comprises:

11. The method according to any one of claims 1 to 7, further comprising:

Acquiring a current display mode of the display screen;

12. The method of claim 11, wherein the display mode comprises a landscape display mode and the triggering operation comprises a touch screen mode based triggering operation;

the determining interface content displayed in the amplifying component based on the display mode and at least one of the first coordinate and the second coordinate indicated by the trigger operation includes:

13. The method of claim 11, wherein the display mode comprises a portrait display mode and the trigger operation comprises a touch screen mode based trigger operation;

14. The method of claim 11, wherein the display mode comprises a cross-screen display mode and the trigger operation comprises a pointer mode-based trigger operation;

15. The method of claim 11, wherein the display mode comprises a portrait display mode and the trigger operation comprises a pointer mode based trigger operation;

16. The method according to any one of claims 1 to 15, wherein displaying a remote operation interface of the remote terminal in a display screen of the mobile terminal in response to a remote connection operation between the mobile terminal and the remote terminal, comprises:

17. A mobile terminal-based remote control device, the device comprising:

18. A computer device, the computer device comprising: a processor and a memory storing a computer program that is loaded and executed by the processor to implement the mobile terminal based remote control method of any of claims 1 to 16.

19. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which is loaded and executed by a processor to implement the mobile terminal based remote control method according to any of claims 1 to 16.

20. A computer program product, characterized in that it comprises computer instructions stored in a computer-readable storage medium, from which a processor obtains the computer instructions, such that the processor loads and executes to implement the mobile terminal based remote control method according to any of claims 1 to 16.