CN114510191B - Screen projection method and device - Google Patents
Screen projection method and device Download PDFInfo
- Publication number
- CN114510191B CN114510191B CN202210143178.6A CN202210143178A CN114510191B CN 114510191 B CN114510191 B CN 114510191B CN 202210143178 A CN202210143178 A CN 202210143178A CN 114510191 B CN114510191 B CN 114510191B
- Authority
- CN
- China
- Prior art keywords
- mouse
- screen
- data
- throwing
- sharing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Controls And Circuits For Display Device (AREA)
- Information Transfer Between Computers (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses a screen projection method which can be applied to shared equipment. In one example, a sharing device may collect screen flow data at a first frame rate and mouse data at a second frame rate, and then send the screen flow data and the mouse data to a projection device, respectively, to cause the projection device to render a projection interface based on the screen flow data and the mouse data. Because the sharing device sends the screen stream data and the mouse data to the screen throwing device respectively, the transmission of the mouse data is less affected by the bandwidth, so that the second frame rate can be higher, and correspondingly, when the screen throwing device renders the screen throwing interface based on the mouse data, the action fluency of the mouse can be higher.
Description
Technical Field
The application relates to the technical field of computers, in particular to a screen projection method and device.
Background
In a screen-cast scenario, a user may screen content displayed on a screen of a shared device to the screen of the screen-cast device. When the content displayed on the screen of the sharing device is projected onto the screen of the projection device to be displayed, the content displayed on the screen of the sharing device can be transmitted to the projection device as a video stream, and the projection device renders the received video stream.
Currently, in a screen-throwing scene, a mouse displayed on a screen of a screen-throwing device has the problem of unsmooth movements, such as the occurrence of a phenomenon of jamming and the like.
Therefore, a scheme is urgently needed to improve the action smoothness of the mouse displayed on the screen of the screen projection device.
Disclosure of Invention
The technical problems to be solved by the application are as follows: how to improve the action fluency of a mouse displayed on a screen of the screen throwing device, and a screen throwing method and a device are provided.
In a first aspect, an embodiment of the present application provides a screen projection method, which is applied to a sharing device, where the method includes:
collecting screen flow data at a first frame rate;
Collecting mouse data at a second frame rate;
and respectively sending the screen flow data and the mouse data to a screen throwing device so that the screen throwing device renders a screen throwing interface based on the screen flow data and the mouse data.
Optionally, the method further comprises:
Responding to a screen throwing operation triggered by a user, and sending a screen throwing request to the screen throwing equipment;
And receiving indication information sent by the screen throwing equipment aiming at the screen throwing request, wherein the indication information is used for indicating the screen throwing equipment to support to independently receive mouse data.
Optionally, the content displayed on the screen of the sharing device is projected to be displayed on a plurality of projection devices, and the plurality of projection devices support to receive the mouse data independently.
Optionally, the collecting mouse data includes:
And in response to detecting that the characteristics of the mouse on the screen of the sharing device are changed, acquiring the mouse data, wherein the characteristics of the mouse comprise: mouse position and/or mouse style.
Optionally, in a case where the position of the mouse is changed and the style of the mouse is not changed, the features of the mouse include the position of the mouse and do not include the style of the mouse.
Optionally, in a case where the position of the mouse is changed and the style of the mouse is changed, the features of the mouse include the position of the mouse and the style of the mouse.
Optionally, in a case where the position of the mouse is unchanged and the pattern of the mouse is changed, the feature of the mouse includes the pattern of the mouse and does not include the position of the mouse.
Optionally, the method further comprises:
receiving a first response sent by the screen throwing device aiming at the received mouse data;
And determining the second frame rate based on the sending information and the response information, wherein the sending information and the receiving information are used for indicating the time delay of the sharing device for sending the mouse data to the screen throwing device, or the sending information and the receiving information are used for indicating the packet loss condition of the sharing device for sending the mouse data to the screen throwing device.
Alternatively to this, the method may comprise,
The transmission information includes: the sharing device sends a first timestamp of the mouse data to the screen throwing device;
the response information includes: the sharing device receives a second timestamp of the first acknowledgement.
Alternatively to this, the method may comprise,
The transmission information includes: the sharing device sends the quantity of the mouse data to the screen throwing device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen throwing equipment.
Optionally, the sharing device supports the separate transmission of mouse data.
In a second aspect, an embodiment of the present application provides a method for processing mouse data based on screen projection, which is applied to a screen projection device, and the method includes:
receiving mouse data sent by a sharing device, wherein the mouse data is the mouse data of a mouse displayed on a screen of the sharing device, and the content displayed on the screen of the sharing device is displayed on the screen of the screen-throwing device;
Caching the mouse data;
And rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache.
Optionally, the mouse data includes: mouse position and/or mouse style.
Optionally, in a case where the mouse data does not include the mouse style, the method further includes:
Obtaining a cached mouse pattern;
The rendering of the mouse on the screen of the screen throwing device based on the mouse data read from the cache comprises the following steps:
and rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse pattern.
Optionally, in a case where the mouse data does not include the mouse position, the method further includes:
Acquiring a cached mouse position;
The rendering of the mouse on the screen of the screen throwing device based on the mouse data read from the cache comprises the following steps:
And rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse position.
Optionally, the method further comprises:
receiving a screen throwing request sent by the sharing equipment;
and sending indication information to the sharing equipment based on the screen throwing request, wherein the indication information is used for indicating that the screen throwing equipment supports independent receiving of mouse data.
In a third aspect, an embodiment of the present application provides a screen projection device, which is applied to a sharing device, where the device includes:
The first acquisition unit is used for acquiring screen flow data at a first frame rate;
the second acquisition unit is used for acquiring mouse data at a second frame rate;
And the first sending unit is used for respectively sending the screen flow data and the mouse data to the screen throwing equipment so that the screen throwing equipment renders a screen throwing interface based on the screen flow data and the mouse data.
Optionally, the apparatus further includes:
the second sending unit is used for responding to the screen throwing operation triggered by the user and sending a screen throwing request to the screen throwing equipment;
the first receiving unit is used for receiving indication information sent by the screen throwing device aiming at the screen throwing request, and the indication information is used for indicating the screen throwing device to support independent receiving of mouse data.
Optionally, the content displayed on the screen of the sharing device is projected to be displayed on a plurality of projection devices, and the plurality of projection devices support to receive the mouse data independently.
Optionally, the collecting mouse data includes:
And in response to detecting that the characteristics of the mouse on the screen of the sharing device are changed, acquiring the mouse data, wherein the characteristics of the mouse comprise: mouse position and/or mouse style.
Optionally, in a case where the position of the mouse is changed and the style of the mouse is not changed, the features of the mouse include the position of the mouse and do not include the style of the mouse.
Optionally, in a case where the position of the mouse is changed and the style of the mouse is changed, the features of the mouse include the position of the mouse and the style of the mouse.
Optionally, in a case where the position of the mouse is unchanged and the pattern of the mouse is changed, the feature of the mouse includes the pattern of the mouse and does not include the position of the mouse.
Optionally, the apparatus further includes:
The second receiving unit is used for receiving a first response sent by the screen throwing device aiming at the received mouse data;
The determining unit is configured to determine the second frame rate based on sending information and response information, where the sending information and the receiving information are used to instruct the sharing device to send a delay of mouse data to the screen throwing device, or the sending information and the receiving information are used to instruct the sharing device to send a packet loss condition of the mouse data to the screen throwing device.
Alternatively to this, the method may comprise,
The transmission information includes: the sharing device sends a first timestamp of the mouse data to the screen throwing device;
the response information includes: the sharing device receives a second timestamp of the first acknowledgement.
Alternatively to this, the method may comprise,
The transmission information includes: the sharing device sends the quantity of the mouse data to the screen throwing device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen throwing equipment.
Optionally, the sharing device supports the separate transmission of mouse data.
In a fourth aspect, an embodiment of the present application provides a device for processing mouse data based on screen projection, which is applied to a screen projection device, and the device includes:
The first receiving unit is used for receiving mouse data sent by the sharing equipment, wherein the mouse data are mouse data of a mouse displayed on a screen of the sharing equipment, and contents displayed on the screen of the sharing equipment are projected to the screen of the screen projecting equipment for display;
The buffer unit is used for buffering the mouse data;
And the rendering unit is used for rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache.
Optionally, the mouse data includes: mouse position and/or mouse style.
Optionally, in a case where the mouse data does not include the mouse pattern, the apparatus further includes:
the first acquisition unit is used for acquiring the cached mouse pattern;
The rendering unit is used for:
and rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse pattern.
Optionally, in a case where the mouse data does not include the mouse position, the apparatus further includes:
the second acquisition unit is used for acquiring the cached mouse position;
The rendering unit is used for:
And rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse position.
Optionally, the apparatus further includes:
the second receiving unit is used for receiving the screen throwing request sent by the sharing equipment;
The sending unit is used for sending indication information to the sharing equipment based on the screen throwing request, wherein the indication information is used for indicating that the screen throwing equipment supports independent receiving of mouse data.
In a fifth aspect, embodiments of the present application provide an apparatus comprising a processor and a memory;
The processor is configured to execute instructions stored in the memory to cause the apparatus to perform the method of any one of the first aspect above, or to cause the apparatus to perform the method of any one of the second aspect above.
In a sixth aspect, embodiments of the present application provide a computer-readable storage medium comprising instructions that instruct a device to perform the method of any one of the first aspects above, or that instruct a device to perform the method of any one of the second aspects above.
In a seventh aspect, embodiments of the present application provide a computer program product which, when run on a computer, causes the computer to perform the method of any one of the first aspects above, or causes the computer to perform the method of any one of the second aspects above.
Compared with the prior art, the embodiment of the application has the following advantages:
The embodiment of the application provides a screen projection method which can be applied to shared equipment. In one example, a sharing device may collect screen flow data at a first frame rate and mouse data at a second frame rate, and then send the screen flow data and the mouse data to a projection device, respectively, to cause the projection device to render a projection interface based on the screen flow data and the mouse data. Because the sharing device sends the screen stream data and the mouse data to the screen throwing device respectively, the transmission of the mouse data is less affected by the bandwidth, so that the second frame rate can be higher, and correspondingly, when the screen throwing device renders the screen throwing interface based on the mouse data, the action fluency of the mouse can be higher.
The embodiment of the application provides a screen projection method which can be applied to screen projection equipment. In one example, the screen casting device may receive mouse data sent by the sharing device and cache the received mouse data. And then, reading the mouse data from the cache and rendering the mouse based on the mouse data. By adopting the scheme, when the screen throwing device receives a plurality of mouse data sent by the sharing device at the same time, the plurality of mouse data can be first selected to be cached, instead of rendering a mouse based on one of the plurality of mouse data and discarding other mouse data in the plurality of mouse data, and the problem that the action of a mouse displayed on a screen of the screen throwing device is unsmooth due to discarding of the mouse data can be avoided.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to the drawings without inventive effort to those skilled in the art.
Fig. 1 is a signaling interaction diagram of a screen projection method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a method for rendering a mouse according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a screen projection device according to an embodiment of the present application;
Fig. 4 is a schematic structural diagram of a screen projection device according to an embodiment of the present application.
Detailed Description
In order to make the present application better understood by those skilled in the art, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The inventor of the present application has found through research that, in the conventional technology, when content displayed on a screen of a sharing device is projected onto the screen of the projection device for display, the content displayed on the screen of the sharing device can be transmitted as a video stream to the projection device according to a specific frame rate, and the received video stream is rendered by the projection device. Wherein the video stream includes a screen stream and mouse data. While the sharing device sends the video stream to the screening device, the particular frame rate is typically not too high, e.g., 15 frames per second, due to the network bandwidth. Because the specific frame rate is not too high, when the frequency of the mouse change of the shared device side is high, the problem of unsmooth mouse action can occur on the screen throwing device side.
The inventors of the embodiments of the present application have also found that in some scenarios there is little variation in the screen flow data of the shared device. For example, in a scenario of a slide show (PPT) presentation, the screen flow data of the sharing device may stay at a certain page of the PPT for a period of time. For this case, if the screen stream data and the mouse data are collected at the same frame rate, and the video stream including the screen stream data and the mouse data is synchronously transmitted to the screen throwing device, since the screen stream transmission is greatly affected by the bandwidth, the frame rate of collecting the screen stream will not be very high, and since the frame rate of collecting the mouse data is the same as the frame rate of collecting the screen stream data, the frame rate of collecting the mouse data will not be very high, and therefore, if the mouse is in the moving process, the problem that the action of the mouse will not be smooth will occur at the screen throwing device side.
In addition, since the screen stream data is the same for a period of time in this scene, the manner of transmitting the video stream including the screen stream and the mouse data to the screen throwing device may result in repeated transmission of the same screen stream data to the screen throwing device, resulting in unnecessary waste of bandwidth.
In view of this, the embodiment of the application provides a screen projection method and device.
Various non-limiting embodiments of the present application are described in detail below with reference to the attached drawing figures.
Exemplary method
Referring to fig. 1, the diagram is a signaling interaction diagram of a screen projection method provided by an embodiment of the present application. In this embodiment, the method may for example comprise the following steps S101-S105.
In the sharing device and the screen-throwing device shown in fig. 1, it should be noted that, the content displayed on the screen of the sharing device is thrown to the screen of the screen-throwing device for display. The embodiment of the application is not particularly limited to the sharing equipment and the screen throwing equipment. In one example, the sharing device and the screen-casting device may be both terminal devices. In one example, content displayed on a screen of the sharing device may be projected onto a projection device for display. In yet another example, for example, in a conference scenario, content that may be displayed on a screen of a sharing device (a device to which a presenter corresponds) may be projected onto a plurality of projection devices (devices to which a plurality of other conference participants correspond). The embodiment of the present application is not particularly limited.
S101: the sharing device collects screen flow data at a first frame rate.
S102: the sharing device collects mouse data at a second frame rate.
In the embodiment of the present application, the sharing device may collect the screen stream data at the first frame rate, which is not particularly limited in the embodiment of the present application. The screen flow data mentioned here may be, for example, screen flow data of the sharing device. In one example, the sharing device may determine whether the screen flow data is changed at a certain sampling interval, and if it is determined that the screen flow data is changed, collect the screen flow data. By adopting the method, the screen flow data can be prevented from being collected and sent to the screen throwing equipment when the screen flow data is not changed, so that repeated screen flow data is sent to the screen throwing equipment, and the bandwidth is saved.
In the embodiment of the present application, the sharing device collects the mouse data at the second frame rate, which is not particularly limited in the embodiment of the present application. The mouse data mentioned here refers to the mouse data of the sharing device.
The embodiment of the application is not particularly limited to the first frame rate and the second frame rate, and the first frame rate and the second frame rate may be the same or different. It will be appreciated that in different application scenarios, the first frame rate and/or the second frame rate may be set manually or adaptively according to the set frame rate adjustment mode, as required by the actual scenario. In one example, for example, in the scenario of the PPT explanation previously described, the second frame rate is greater than the first frame rate.
In one example, the "collect mouse data" in S102 may be acquired in a specific implementation in a case where a characteristic of a mouse on a screen of the sharing device is changed. By adopting the mode, repeated mouse data can be prevented from being collected and sent to the screen throwing equipment under the condition that the characteristics of the mouse are not changed, so that the bandwidth is saved.
In the embodiment of the application, the characteristics of the mouse can comprise the position and/or the style of the mouse. In one example, where the position of the mouse is changed and the style of the mouse is not changed, the characteristics of the mouse may include the mouse position and not the mouse style. In yet another example, where the position of the mouse changes and the style of the mouse changes, the characteristics of the mouse may include the mouse position and the mouse style. In another example, where the position of the mouse is unchanged and the style of the mouse is changed, the characteristics of the mouse include the mouse style and do not include the mouse position.
With respect to the mouse data, it should be noted that the mouse data refers to mouse-related data displayed on a screen of the sharing device. The mouse data includes a mouse pattern and/or a mouse position. In one example, the client may obtain the mouse style and mouse position using an application programming interface (Application Programming Interface, API) of its operating system.
In one example, the mouse data relates to a characteristic of a mouse. Specifically:
In one example, if the mouse feature includes the mouse position and does not include the mouse pattern, the mouse data may include the mouse position and does not include the mouse pattern.
In yet another example, if the characteristics of the mouse include the mouse position and the mouse style. The mouse data may include the mouse position and the mouse pattern. The mouse style may be, for example, a picture format.
In another example, if the characteristics of the mouse include the mouse pattern and do not include the mouse position, the mouse data may include the mouse pattern and do not include the mouse position.
S103: and the sharing equipment respectively sends the screen stream data and the mouse data to the screen throwing equipment.
In the embodiment of the application, the sharing device has the capability of independently sending mouse data, namely: the sharing device supports the separate transmission of mouse data. In other words, the sharing device does not necessarily have to collect the screen stream data and the mouse data at the same frame rate and transmit the video stream including the screen stream data and the mouse data to the screen throwing device, but may collect the screen stream data and the mouse data at different frame rates, respectively, and transmit the collected screen stream data and the mouse data to the screen throwing device at different frame rates. Because the sharing device has the capability of independently sending the mouse data, the sharing device can respectively send the screen stream data and the mouse data to the screen throwing device.
In one example, another condition for performing S103 is: the screen projection device has the capability of independently receiving mouse data, namely: the screen throwing device supports independent receiving of mouse data.
In one example, before executing S103, the sharing device may determine whether the screen casting device supports the separate reception of the mouse data, and if it is determined that the screen casting device supports the separate reception of the mouse data, the sharing device may execute S103. In one example, after the user triggers the screen-casting operation, the sharing device may send a screen-casting request to the screen-casting device in response to the user-triggered screen-casting operation. As an example, if the screen throwing device supports to separately receive the mouse data, after receiving the screen throwing request, the screen throwing device sends indication information to the sharing device, where the indication information is used to indicate that the screen throwing device supports to separately receive the mouse data. After the sharing device receives the indication information, it can be determined that the screen throwing device supports independent receiving of mouse data. The screen projection device mentioned herein supports the separate reception of mouse data, and means that the screen projection device supports the separate drawing of a mouse. As one example, a function module that draws a mouse alone may be included in the projection device. The function module of the independent drawing mouse can be executed after receiving the mouse data, and is not influenced by the screen flow data.
With respect to the user-triggered screen-throwing operation, the embodiment of the present application is not particularly limited, and in one example, the user may trigger the screen-throwing operation by using an Application (APP) installed on the sharing device for providing the screen-throwing service.
As previously described, content displayed on the screen of the sharing device may be projected onto one or more projection devices for display. When the content displayed on the screen of the sharing device is projected to a plurality of projection devices for display, the plurality of projection devices support to independently receive the mouse data. For this case, the sharing device may send the foregoing screen-throwing request to each screen-throwing device, and receive the foregoing indication information sent by each screen-throwing device.
S104: the screen throwing device receives screen stream data and mouse data sent by the sharing device.
S105: and the screen throwing device renders a screen throwing interface based on the screen flow data and the mouse data.
S105, in a specific implementation, may render page content in a screen-drop interface based on screen flow data, and render a mouse on a screen of the screen-drop device based on the mouse data.
In one example, after the screen casting device receives the mouse data, the mouse may be rendered on a screen of the screen casting device directly based on the mouse data.
As can be seen from the above description, by using the scheme provided by the embodiment of the present application, the sharing device can send the screen stream data and the mouse data to the screen throwing device respectively. In this way, in the scene explained by the PPT, the screen stream data and the mouse data are respectively collected according to different frame rates, and the frame rate for collecting the screen stream data can be reduced and the frame rate for collecting the mouse data can be properly improved because the screen stream data has small change. The frame rate of the screen flow data is reduced, so that the frequency of transmitting the same screen flow data to the screen throwing equipment can be reduced, and the network bandwidth is saved. In addition, the frame rate of the acquired mouse data is improved, so that the frame rate of the mouse data received by the screen throwing equipment side is correspondingly improved, and the action fluency of the mouse of the screen throwing equipment side can be improved.
In one example, as shown in FIG. 2, the screen casting device may render mouse data based on steps S201-S203 as follows
S201: and receiving mouse data sent by the sharing equipment, wherein the mouse data are the mouse data of a mouse displayed on a screen of the sharing equipment, and the content displayed on the screen of the sharing equipment is displayed on the screen of the screen-throwing equipment.
In one example, the sharing device may collect mouse data at a second frame rate. After the sharing device collects the mouse data, the mouse data can be sent to the screen throwing device, and correspondingly, the screen throwing device can receive the mouse data sent by the sharing device.
For a specific implementation in which the sharing device collects mouse data at the second frame rate, reference may be made to the relevant description section above, and the description will not be repeated here.
S202: and caching the mouse data.
S203: and rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache.
It is contemplated that in some scenarios, if a network between the sharing device and the screen-casting device shakes, the screen-casting device may receive multiple mouse data at the same time. Namely: the mouse data sent by the sharing device for multiple times are received by the screen throwing device at the same time. For this case, if the screen projection device renders a mouse based on one of the plurality of mouse data and discards the other mouse data of the plurality of mouse data, the action of the mouse on the screen of the screen projection device may be unsmooth. For this case, in order to make the mouse action on the screen of the screen throwing device more natural and smooth. In one example, the screen casting device may cache the received plurality of mouse data. For example, the screen-casting device may cache the plurality of mouse data in the first cache region. For another example, the screen-casting device may cache the plurality of mouse data in a cache queue. When the mouse is rendered, the cached mouse data can be sequentially read from the cache, and the mouse is rendered based on the read mouse data. For example, the screen-projecting device may read mouse data from the cache at a certain frame rate, and render a mouse based on the read mouse data.
In one example, when rendering a mouse on a screen of a screen-throwing device based on mouse data read from a cache, if the mouse data includes a mouse position but does not include a mouse pattern, the screen-throwing device may acquire the cached mouse pattern, and further, render a mouse on the screen of the screen-throwing device based on the mouse data read from the cache (i.e., the mouse position) and the cached mouse pattern. The cached mouse pattern referred to herein refers to the mouse pattern used by the screen throwing device to render the mouse last time. In other words, the last time the screen throwing device renders the mouse, the mouse style used by the rendering mouse may be cached in the cache (e.g., in the second cache region).
In one example, when rendering a mouse on a screen of a screen-throwing device based on cached mouse data, if the mouse data includes a mouse pattern but does not include a mouse position, the screen-throwing device may acquire the cached mouse position, and further, render the mouse on the screen of the screen-throwing device based on the mouse data (i.e., the mouse pattern) read from the cache and the cached mouse position. Reference herein to cached mouse position refers to the position of the mouse that was used by the screen casting device to render the mouse last time. In other words, the last time the screen-throwing device renders the mouse, the mouse position used by the rendering mouse may be cached in the cache (e.g., in the third cache region).
In one example, if the mouse position used by the screen throwing device to render the mouse currently is obtained from the cache (for example, the first cache region), it indicates that the position of the mouse is changed compared with the last time of rendering the mouse, and in this case, the screen throwing device may replace the mouse position stored in the third cache region with the mouse position used by the mouse currently, so that when the mouse is rendered next time, if the mouse position is not changed but the mouse style is changed, the mouse may be rendered based on the mouse position in the third cache region.
In addition, in one example, if the mouse pattern used by the screen throwing device for rendering the mouse currently is obtained from the cache (for example, the first cache region), it is indicated that the mouse pattern is changed compared with the previous rendering of the mouse, and for this case, the screen throwing device may replace the mouse pattern stored in the second cache region with the mouse pattern used by the current rendering mouse, so that when the mouse is rendered next time, if the mouse pattern is not changed but the mouse position is changed, the mouse may be rendered based on the mouse pattern in the second cache region.
In one example, after the sharing device sends the mouse data to the screen casting device, the screen casting device receives the mouse data, and may send a response to the mouse data to the sharing device. Further, the sharing device may determine the aforementioned second frame rate based on the transmission information and the response information. Wherein:
The sending information and the receiving information are used for indicating time delay of the sharing device sending the mouse data to the screen throwing device, or the sending information and the receiving information are used for indicating packet loss condition of the sharing device sending the mouse data to the screen throwing device.
In one example, when the sending information and the receiving information are used to instruct the sharing device to send the time delay of the mouse data to the screen throwing device, the sending information includes: the sharing device sends a first timestamp of the mouse data to the screen throwing device. The response information includes: the sharing device receives a second timestamp of the first acknowledgement.
As one example, the sharing device may record the first timestamp when sending mouse data to the screen casting device. Correspondingly, the sharing device can record a second timestamp when receiving the first response sent by the screen throwing device. Based on the second timestamp and the first timestamp, the time delay of the sharing device for sending mouse data to the screen throwing device can be calculated. For example, the delay is 0.5× (second timestamp-first timestamp). After calculating the delay, the sharing device may determine the aforementioned second frame rate based on the delay. For example, the second frame rate may be the inverse of the time delay, or the second frame rate may be slightly less than the inverse of the time delay.
In one example, when the sending information and the receiving information are used to instruct the sharing device to send the packet loss condition of the mouse data to the screen throwing device, the sending information includes: the sharing device sends the quantity of the mouse data to the screen throwing device; the response information includes: and the sharing equipment receives the number of the responses sent by the screen throwing equipment.
In one example, the sharing device may adjust the second frame rate during interaction with the screen-casting device, where the initial value of the second frame rate may be a default value, for example. Specifically:
The sharing device may send mouse data to the screen-throwing device at a second frame rate, and then adjust the second frame rate based on the number of mouse data sent by the sharing device to the screen-throwing device and the number of responses sent by the screen-throwing device received by the sharing device. For example, if the number of mouse data sent by the sharing device to the screen throwing device is greater than the number of responses sent by the screen throwing device received by the sharing device, which indicates that a packet loss occurs in the mouse data sent by the sharing device to the screen throwing device, the sharing device may reduce the second frame rate. For another example, if the number of mouse data sent by the sharing device to the screen throwing device is equal to the number of responses sent by the sharing device to the screen throwing device, it is indicated that no packet loss occurs in the mouse data sent by the sharing device to the screen throwing device, and then the sharing device may increase the second frame rate or keep the second frame rate unchanged.
Exemplary apparatus
Based on the method provided by the embodiment, the embodiment of the application also provides a device, and the device is described below with reference to the accompanying drawings.
Referring to fig. 3, the structure of a screen projection device according to an embodiment of the present application is shown. The apparatus 300 may be applied to a shared device, and the apparatus 300 may specifically include, for example: a first acquisition unit 301, a second acquisition unit 302 and a first transmission unit 303.
A first acquisition unit 301 for acquiring screen flow data at a first frame rate;
a second acquisition unit 302, configured to acquire mouse data at a second frame rate;
And a first sending unit 303, configured to send the screen flow data and the mouse data to a screen throwing device, so that the screen throwing device renders a screen throwing interface based on the screen flow data and the mouse data.
Optionally, the apparatus further includes:
the second sending unit is used for responding to the screen throwing operation triggered by the user and sending a screen throwing request to the screen throwing equipment;
the first receiving unit is used for receiving indication information sent by the screen throwing device aiming at the screen throwing request, and the indication information is used for indicating the screen throwing device to support independent receiving of mouse data.
Optionally, the content displayed on the screen of the sharing device is projected to be displayed on a plurality of projection devices, and the plurality of projection devices support to receive the mouse data independently.
Optionally, the collecting mouse data includes:
And in response to detecting that the characteristics of the mouse on the screen of the sharing device are changed, acquiring the mouse data, wherein the characteristics of the mouse comprise: mouse position and/or mouse style.
Optionally, in a case where the position of the mouse is changed and the style of the mouse is not changed, the features of the mouse include the position of the mouse and do not include the style of the mouse.
Optionally, in a case where the position of the mouse is changed and the style of the mouse is changed, the features of the mouse include the position of the mouse and the style of the mouse.
Optionally, in a case where the position of the mouse is unchanged and the pattern of the mouse is changed, the feature of the mouse includes the pattern of the mouse and does not include the position of the mouse.
Optionally, the apparatus further includes:
The second receiving unit is used for receiving a first response sent by the screen throwing device aiming at the received mouse data;
The determining unit is configured to determine the second frame rate based on sending information and response information, where the sending information and the receiving information are used to instruct the sharing device to send a delay of mouse data to the screen throwing device, or the sending information and the receiving information are used to instruct the sharing device to send a packet loss condition of the mouse data to the screen throwing device.
Alternatively to this, the method may comprise,
The transmission information includes: the sharing device sends a first timestamp of the mouse data to the screen throwing device;
the response information includes: the sharing device receives a second timestamp of the first acknowledgement.
Alternatively to this, the method may comprise,
The transmission information includes: the sharing device sends the quantity of the mouse data to the screen throwing device;
the response information includes: and the sharing equipment receives the number of the responses sent by the screen throwing equipment.
Optionally, the sharing device supports the separate transmission of mouse data.
Since the apparatus 300 is an apparatus corresponding to the method provided in the above method embodiment, the specific implementation of each unit of the apparatus 300 is the same as the above method embodiment, and therefore, with respect to the specific implementation of each unit of the apparatus 300, reference may be made to the description part of the steps executed by the above method embodiment for the sharing device, which is not repeated herein.
Referring to fig. 4, the structure of a screen projection device according to an embodiment of the present application is shown. The apparatus 400 may be applied to a screen projection device, and the apparatus 400 may specifically include, for example: a first receiving unit 401, a buffering unit 402, and a rendering unit 403.
A first receiving unit 401, configured to receive mouse data sent by a sharing device, where the mouse data is mouse data of a mouse displayed on a screen of the sharing device, and content displayed on the screen of the sharing device is projected onto a screen of the projection device for display;
a buffer unit 402, configured to buffer the mouse data;
and a rendering unit 403, configured to render a mouse on a screen of the screen-throwing device based on the mouse data read from the cache.
Optionally, the mouse data includes: mouse position and/or mouse style.
Optionally, in a case where the mouse data does not include the mouse pattern, the apparatus further includes:
the first acquisition unit is used for acquiring the cached mouse pattern;
The rendering unit 403 is configured to:
and rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse pattern.
Optionally, in a case where the mouse data does not include the mouse position, the apparatus further includes:
the second acquisition unit is used for acquiring the cached mouse position;
The rendering unit 403 is configured to:
And rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse position.
Optionally, the apparatus further includes:
the second receiving unit is used for receiving the screen throwing request sent by the sharing equipment;
The sending unit is used for sending indication information to the sharing equipment based on the screen throwing request, wherein the indication information is used for indicating that the screen throwing equipment supports independent receiving of mouse data.
Since the apparatus 400 is an apparatus corresponding to the method provided by the above method embodiment, the specific implementation of each unit of the apparatus 400 is the same as the above method embodiment, and therefore, with respect to the specific implementation of each unit of the apparatus 400, reference may be made to the description part of the steps executed by the above method embodiment for the screen projection device, which is not repeated herein.
The embodiment of the application provides equipment, which comprises a processor and a memory;
The processor is configured to execute the instructions stored in the memory, so that the device performs the method performed by the sharing device or the screen-casting device according to any of the above method embodiments.
Embodiments of the present application provide a computer-readable storage medium comprising instructions that instruct a device to perform a method performed by a sharing device or a screen-casting device of any of the above method embodiments.
Embodiments of the present application provide a computer program product which, when run on a computer, causes the computer to perform the method of any of the above method embodiments performed by a sharing device or a screening device.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the application are intended to be included within the scope of the application.
Claims (15)
1. A method of dropping a screen, applied to a shared device that supports separate transmission of mouse data, the method comprising:
Responding to a screen throwing operation triggered by a user, and sending a screen throwing request to screen throwing equipment;
Receiving indication information sent by the screen throwing device for the screen throwing request, wherein the indication information is used for indicating the screen throwing device to support independent receiving of mouse data;
Judging whether screen flow data are changed or not according to a certain sampling interval, and if the screen flow data are determined to be changed, acquiring the screen flow data at a first frame rate;
Collecting mouse data at a second frame rate;
respectively sending the screen flow data and the mouse data to a screen throwing device so that the screen throwing device renders a screen throwing interface based on the screen flow data and the mouse data;
the second frame rate is determined by:
receiving a first response sent by the screen throwing device aiming at the received mouse data;
And determining the second frame rate based on the sending information and the response information, wherein the sending information and the response information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen throwing equipment.
2. The method of claim 1, wherein content displayed on a screen of the shared device is projected onto a plurality of projected devices, each of the plurality of projected devices supporting separate receipt of mouse data.
3. The method of claim 1, wherein the acquiring mouse data comprises:
And in response to detecting that the characteristics of the mouse on the screen of the sharing device are changed, acquiring the mouse data, wherein the characteristics of the mouse comprise: mouse position and/or mouse style.
4. The method of claim 3, wherein the characteristics of the mouse include the mouse position and do not include the mouse pattern in the event that the position of the mouse changes and the pattern of the mouse does not change.
5. A method according to claim 3, wherein the characteristics of the mouse include the mouse position and the mouse pattern in the event that the position of the mouse changes and the pattern of the mouse changes.
6. The method of claim 3, wherein the characteristics of the mouse include the mouse pattern and do not include the mouse position in the event that the position of the mouse is unchanged and the pattern of the mouse is changed.
7. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The transmission information includes: the sharing device sends a first timestamp of the mouse data to the screen throwing device;
the response information includes: the sharing device receives a second timestamp of the first acknowledgement.
8. A method for processing mouse data based on screen projection, which is characterized by being applied to screen projection equipment, the method comprising:
After triggering screen throwing operation on a sharing device by a user, receiving a screen throwing request sent by the sharing device, wherein the sharing device supports independent sending of mouse data;
Transmitting indication information to the sharing equipment based on the screen throwing request, wherein the indication information is used for indicating that the screen throwing equipment supports independent receiving of mouse data;
Receiving screen flow data and mouse data sent by a sharing device, wherein the mouse data is mouse data of a mouse displayed on a screen of the sharing device, content displayed on the screen of the sharing device is projected to the screen of the screen projecting device for display, the screen flow data is acquired at a first frame rate under the condition that the screen flow data is determined to be changed by the sharing device according to a certain sampling interval;
Caching the mouse data;
Rendering the screen stream and a mouse on a screen of the screen throwing device based on the screen stream data and the mouse data read from the cache;
The mouse data are acquired by the sharing equipment at a second frame rate and sent to the screen throwing equipment;
The sharing device determines the second frame rate by:
receiving a first response sent by the screen throwing device aiming at the received mouse data;
And determining the second frame rate based on the sending information and the response information, wherein the sending information and the response information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen throwing equipment.
9. The method of claim 8, wherein the mouse data comprises: mouse position and/or mouse style.
10. The method of claim 9, wherein in the event that the mouse data does not include the mouse pattern, the method further comprises:
Obtaining a cached mouse pattern;
The rendering of the mouse on the screen of the screen throwing device based on the mouse data read from the cache comprises the following steps:
and rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse pattern.
11. The method of claim 9, wherein in the event that the mouse data does not include the mouse position, the method further comprises:
Acquiring a cached mouse position;
The rendering of the mouse on the screen of the screen throwing device based on the mouse data read from the cache comprises the following steps:
And rendering a mouse on a screen of the screen throwing device based on the mouse data read from the cache and the cached mouse position.
12. A screen projection apparatus for use with a shared device that supports separate transmission of mouse data, the apparatus comprising:
the second sending unit is used for responding to the screen throwing operation triggered by the user and sending a screen throwing request to the screen throwing equipment;
the first receiving unit is used for receiving indication information sent by the screen throwing equipment aiming at the screen throwing request, and the indication information is used for indicating the screen throwing equipment to support independent receiving of mouse data;
The first acquisition unit is used for judging whether the screen flow data is changed according to a certain sampling interval, and if the screen flow data is determined to be changed, acquiring the screen flow data at a first frame rate;
the second acquisition unit is used for acquiring mouse data at a second frame rate;
The first sending unit is used for respectively sending the screen flow data and the mouse data to the screen throwing equipment so that the screen throwing equipment renders a screen throwing interface based on the screen flow data and the mouse data;
the second frame rate is determined by:
receiving a first response sent by the screen throwing device aiming at the received mouse data;
And determining the second frame rate based on the sending information and the response receiving information, wherein the sending information and the response receiving information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen throwing equipment.
13. A screen-based mouse data processing apparatus, characterized in that it is applied to a screen-casting device, the apparatus comprising:
The second receiving unit is used for receiving a screen throwing request sent by the sharing equipment after a user triggers the screen throwing operation on the sharing equipment, and the sharing equipment supports independent sending of mouse data;
The sending unit is used for sending indication information to the sharing equipment based on the screen throwing request, wherein the indication information is used for indicating that the screen throwing equipment supports independent receiving of mouse data;
the first receiving unit is used for receiving screen flow data and mouse data sent by the sharing equipment, wherein the mouse data is mouse data of a mouse displayed on a screen of the sharing equipment, content displayed on the screen of the sharing equipment is projected to the screen of the screen projecting equipment for display, the screen flow data is obtained by judging whether the screen flow data is changed or not by the sharing equipment according to a certain sampling interval, and the screen flow data is collected at a first frame rate under the condition that the screen flow data is determined to be changed;
The buffer unit is used for buffering the mouse data;
a rendering unit for rendering the screen stream and the mouse on a screen of the screen throwing device based on the screen stream data and the mouse data read from the cache;
The mouse data are acquired by the sharing equipment at a second frame rate and sent to the screen throwing equipment;
The sharing device determines the second frame rate by:
receiving a first response sent by the screen throwing device aiming at the received mouse data;
And determining the second frame rate based on the sending information and the response information, wherein the sending information and the response information are used for indicating the time delay of the sharing equipment for sending the mouse data to the screen throwing equipment.
14. An electronic device comprising a processor and a memory;
The processor is configured to execute instructions stored in the memory to cause the electronic device to perform the method of any one of claims 1 to 11.
15. A computer readable storage medium comprising instructions that instruct a device to perform the method of any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210143178.6A CN114510191B (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210143178.6A CN114510191B (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114510191A CN114510191A (en) | 2022-05-17 |
| CN114510191B true CN114510191B (en) | 2024-10-18 |
Family
ID=81552376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210143178.6A Active CN114510191B (en) | 2022-02-16 | 2022-02-16 | Screen projection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114510191B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115052186B (en) * | 2022-07-12 | 2023-09-15 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115037978B (en) * | 2022-07-13 | 2023-08-25 | 北京字跳网络技术有限公司 | Screen projection method and related equipment |
| CN115209117B (en) * | 2022-07-20 | 2024-06-18 | 北京字跳网络技术有限公司 | Screen projection method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155606A (en) * | 2015-04-07 | 2016-11-23 | 中国移动通信集团公司 | A kind of multi-screen interaction method and device |
| CN106375841A (en) * | 2015-07-23 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Wireless screen projection data processing method and device, video data display method and device, and electronic device |
| CN107317655A (en) * | 2017-06-06 | 2017-11-03 | 努比亚技术有限公司 | Transfer control method, system and the readable storage medium storing program for executing of screen prjection |
| CN110519632A (en) * | 2019-07-30 | 2019-11-29 | 华为技术有限公司 | Throw screen method and apparatus |
| CN111541919A (en) * | 2020-05-13 | 2020-08-14 | 北京百度网讯科技有限公司 | Video frame transmission method and device, electronic equipment and readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112399215A (en) * | 2020-10-26 | 2021-02-23 | 努比亚技术有限公司 | Screen projection parameter regulation and control method and device and computer readable storage medium |
-
2022
- 2022-02-16 CN CN202210143178.6A patent/CN114510191B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106155606A (en) * | 2015-04-07 | 2016-11-23 | 中国移动通信集团公司 | A kind of multi-screen interaction method and device |
| CN106375841A (en) * | 2015-07-23 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Wireless screen projection data processing method and device, video data display method and device, and electronic device |
| CN107317655A (en) * | 2017-06-06 | 2017-11-03 | 努比亚技术有限公司 | Transfer control method, system and the readable storage medium storing program for executing of screen prjection |
| CN110519632A (en) * | 2019-07-30 | 2019-11-29 | 华为技术有限公司 | Throw screen method and apparatus |
| CN111541919A (en) * | 2020-05-13 | 2020-08-14 | 北京百度网讯科技有限公司 | Video frame transmission method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114510191A (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114510191B (en) | Screen projection method and device | |
| JP7110234B2 (en) | Video playback methods, devices and systems | |
| US20230188578A1 (en) | Data transmission method and apparatus | |
| EP1566038B1 (en) | Methods and systems for transferring events including multimedia data | |
| US10924782B2 (en) | Method of providing streaming service based on image segmentation and electronic device supporting the same | |
| US20100293226A1 (en) | Web conference performance monitoring system | |
| US8725947B2 (en) | Cache control for adaptive stream player | |
| JP2001094625A (en) | Data communication unit, data communication method and storage medium | |
| US9813742B2 (en) | Method, device and system for evaluating user experience value of video quality | |
| US20150134846A1 (en) | Method and apparatus for media segment request retry control | |
| CN109089131A (en) | A kind of record screen live broadcasting method, device, equipment and storage medium based on IOS system | |
| KR101795537B1 (en) | System and method for providing of cloud streaming service which is adapting to network environment | |
| CN113068001A (en) | Data processing method, device, equipment and medium based on cascade camera | |
| US11463651B2 (en) | Video frame-based media stream bandwidth reduction | |
| WO2017150792A1 (en) | Apparatus and method for providing contents using web-based virtual desktop protocol | |
| US10681400B2 (en) | Method and device for transmitting video | |
| US11070607B2 (en) | Dynamic behavior modification for content download and playback | |
| KR101668540B1 (en) | Distribution network and server, and distribution method | |
| US11438545B2 (en) | Video image-based media stream bandwidth reduction | |
| WO2024169650A1 (en) | Method and apparatus for acquiring screen-mirroring frame rate, and computer device and storage medium | |
| CN109729375B (en) | A kind of interactive data distribution control method, device, device and storage medium | |
| EP4020931A1 (en) | Managing data flow between source node and recipient node | |
| US20240305862A1 (en) | Method and system for requesting tile streams | |
| US20130297745A1 (en) | Method and device for receiving multimedia data | |
| CN110113641B (en) | Video data transmission method, device, edge service node and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |