CN115904290A

CN115904290A - Display device, communication terminal and screen projection control method

Info

Publication number: CN115904290A
Application number: CN202211513148.6A
Authority: CN
Inventors: 马晓燕; 庞秀娟; 宋子全
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2022-11-29
Filing date: 2022-11-29
Publication date: 2023-04-04

Abstract

The application provides a display device, a communication terminal and a screen projection control method, and the method can judge whether the current display device can project a screen or not by detecting the screen end state of the display device when a screen projection request sent by the communication terminal is received. And then sending a notification message containing the screen end state to the communication terminal, wherein if the screen end state is a busy state, the communication terminal can display a prompt message after receiving the notification message, and if the screen end state is an idle state, the communication terminal can establish screen projection connection with the display equipment after receiving the notification message. The method can be used for carrying out screen projection display processing through interaction among multiple devices when screen projection is carried out in a single-path mode or multiple-path mode, and user experience is improved.

Description

Display device, communication terminal and screen projection control method

Technical Field

The application relates to the technical field of display equipment, in particular to display equipment, a communication terminal and a screen projection control method.

Background

The display device is a terminal device capable of outputting a specific display picture, and may be a terminal device such as a smart television, a communication terminal, a smart advertisement screen, and a projector. Taking an intelligent television as an example, the intelligent television is based on an Internet application technology, is provided with an open operating system and a chip, is provided with an open application platform, can realize a bidirectional man-machine interaction function, integrates multiple functions of video, entertainment, data and the like, and is a television product for meeting diversified and personalized requirements of users.

The display device can also establish a screen-casting connection relation with the communication terminal through communication modes such as a wireless local area network, namely the display device can receive screen-casting data shared by the communication terminal and display the received screen-casting data in real time, namely the current picture of the communication terminal or a media asset picture played on the communication terminal is displayed. However, a scene of single-path or multi-path corresponding multiple devices or multi-protocol screen projection interaction may be encountered in screen projection, for example, under single-path or multi-path screen projection, a communication terminal is already projected onto a display device at present, and at this time, how screen projection processing should be performed by the display device after receiving screen projection requests of other communication terminals is received by the display device.

Disclosure of Invention

The application provides a display device, a communication terminal and a screen projection control method, and aims to solve the problem of interaction between the traditional display device and the communication terminal.

In a first aspect, the present application provides a display device comprising: a display, a communicator, and a controller. Wherein the display is configured to display a screen projection interface; the communicator is configured to establish a screen-casting connection with a communication terminal; the controller is configured to perform the following program steps:

receiving a screen projection request sent by a target communication terminal;

responding to the screen projection request, and detecting the screen end state of the display equipment, wherein the screen end state is an idle state or a busy state; the idle state is used for representing that the current maximum screen-throwing quantity of the display equipment is more than or equal to 1 and a target display position is available; the busy state is used for indicating that the current maximum screen-throwing quantity of the display equipment is less than 1 and/or a target display position is unavailable;

if the screen end state is an idle state, screen projection connection is established with the target communication terminal;

and if the screen end state is a busy state, sending a notification message to the target communication terminal, wherein the notification message is used for enabling the target communication terminal to display first prompt information.

In a second aspect, the present application further provides a communication terminal, including: a display unit, a communication unit and a processor. The display unit is configured to display a terminal interface; the communication unit is configured to establish a screen-casting connection with a display device; the processor is configured to perform the following program steps:

sending a screen projection request to display equipment, and receiving a notification message fed back by the display equipment, wherein the notification message comprises a screen end state of the display equipment; the screen end state is an idle state or a busy state; the idle state is used for representing that the current maximum screen-throwing quantity of the display equipment is more than or equal to 1 and a target display position is available; the busy state is used for indicating that the current maximum screen-throwing quantity of the display equipment is less than 1 and/or a target display position is unavailable;

if the screen end state is an idle state, screen projection connection is established with the display equipment;

and if the screen end state is a busy state, receiving prompt information fed back by the display equipment, and controlling a display unit to display the prompt information.

In a third aspect, the present application further provides a screen projection control method, including:

the method comprises the steps that a display device responds to a screen projection request sent by a target communication terminal, and the screen end state of the display device is detected, wherein the screen end state is an idle state or a busy state;

the display equipment sends a notification message to the target communication terminal, wherein the notification message comprises the screen end state;

if the screen end state is a busy state, the target communication terminal receives a notification message sent by the display equipment and displays a prompt message;

and if the screen end state is an idle state, the target communication terminal receives a notification message sent by the display equipment and establishes screen projection connection with the display equipment.

According to the technical scheme, the display device, the communication terminal and the screen projection control method can judge whether the current display device can project the screen or not by detecting the screen end state of the display device when the screen projection request sent by the communication terminal is received. And then sending a notification message containing the screen end state to the communication terminal, wherein if the screen end state is a busy state, the communication terminal can display prompt information after receiving the notification message, and if the screen end state is an idle state, the communication terminal can establish screen projection connection with the display equipment after receiving the notification message. The method can be used for carrying out screen projection display processing through interaction among multiple devices when screen projection is carried out in a single-path mode or multiple-path mode, and user experience is improved.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments are briefly described below, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus in an embodiment of the present application;

fig. 2 is a block diagram of a hardware configuration of a display device in an embodiment of the present application;

FIG. 3 is a schematic diagram of a software configuration of a display device in an embodiment of the present application;

fig. 4 is a schematic display diagram of an icon control interface of an application program of a display device in an embodiment of the present application;

fig. 5 is a schematic structural diagram of a communication terminal in an embodiment of the present application;

fig. 6 is a schematic software architecture diagram of a communication terminal according to an embodiment of the present application;

FIG. 7 is a schematic view of a screen projection operation in the embodiment of the present application;

FIG. 8 is a schematic diagram of a screen projection effect in the embodiment of the present application;

fig. 9 is a schematic diagram of screen projection connection of a multi-communication terminal in an embodiment of the present application;

FIG. 10 is a flowchart illustrating a screen projection control method according to an embodiment of the present application;

FIG. 11 is a flowchart illustrating the process of determining the current maximum number of screens to be projected according to an embodiment of the present application;

FIG. 12 is a schematic view illustrating a process of detecting a state of an interface of a target display location according to an embodiment of the present disclosure;

FIG. 13 is a schematic diagram of creating a drop window in an embodiment of the present application;

fig. 14 is a schematic diagram illustrating display effects of two communication terminals in the embodiment of the present application;

fig. 15 is a schematic diagram of display effects of three communication terminals in the embodiment of the present application;

fig. 16 is a schematic diagram of a communication terminal corresponding to a display position of a detection target in an embodiment of the present application;

fig. 17 is a schematic diagram illustrating display effects of three communication terminals in the embodiment of the present application;

fig. 18 is a schematic view of an interface for displaying a first prompt message by the communication terminal in the embodiment of the present application;

fig. 19 is a schematic diagram of the embodiment of the present application after replacing the communication terminal;

fig. 20 is a schematic diagram of a communication terminal acquiring a screen end state in the embodiment of the present application.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the functionality associated with that element.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and the display device 200 is controlled by a wireless or wired method. The user may input a user command through a key on a remote controller, a voice input, a control panel input, etc. to control the display apparatus 200.

In some embodiments, a smart device 300 (e.g., a communication terminal, a tablet, a computer, a laptop, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received through a module configured inside the display device 200 to obtain a voice command, or may be received through a voice control device provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, serving as an interaction intermediary between the user and the display device 200.

Fig. 3 illustrates a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments the controller comprises a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving image display, a component for receiving an image signal from the controller output, performing display of video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of a control signal and a data signal with the external control apparatus 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals for controlling the apparatus 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other operable control. The operations related to the selected object are: displaying an operation of connecting to a hyperlink page, document, image, etc., or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors. E.g., comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit which carries out operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on the direct display device 200.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module, such as an image synthesizer, is used for performing superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphics generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain an audio signal that can be played in the speaker.

In some embodiments, the user may input a user command on a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, the system of the display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (windows) programs carried by an operating system, system setting programs, clock programs or the like; or may be an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application program. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the location Manager (location Manager) is used for providing the system service or application with the access of the system location service; a Package Manager (Package Manager) for retrieving various information related to an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications and the usual navigation fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of the display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, shaking the display, distorting and deforming the display, and the like).

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

Fig. 5 shows a schematic configuration of the communication terminal 300.

The following specifically describes an embodiment by taking communication terminal 300 as an example. It should be understood that the communication terminal 300 shown in fig. 5 is only an example, and the communication terminal 300 may have more or less components than those shown in fig. 5, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

As shown in fig. 5, the communication terminal 300 includes: radio Frequency (RF) circuitry 310, memory 320, display unit 330, camera 340, sensor 350, audio circuitry 360, wireless Fidelity (Wi-Fi) circuitry 370, processor 380, bluetooth circuitry 381, and power supply 390.

The RF circuit 310 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink data of a base station and then send the downlink data to the processor 380 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

Memory 320 may be used to store software programs and data. The processor 380 executes various functions of the communication terminal 300 and data processing by executing software programs or data stored in the memory 320. The memory 320 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 320 stores an operating system that enables the communication terminal 300 to operate. The memory 320 may store an operating system and various application programs, and may also store codes for performing the methods described in the embodiments of the present application.

The display unit 330 may be used to receive input numeric or character information and generate signal input related to user settings and function control of the communication terminal 300, and particularly, the display unit 330 may include a touch screen 331 disposed on the front of the communication terminal 300 and collecting touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like.

The display unit 330 may also be used to display information input by the user or information provided to the user, and a Graphical User Interface (GUI) of various menus of the communication terminal 300. Specifically, the display unit 330 may include a display screen 332 disposed on the front surface of the communication terminal 300. The display screen 332 may be configured in the form of a liquid crystal display, a light emitting diode, or the like. The display unit 330 may be used to display various graphical user interfaces described herein.

The touch screen 331 may cover the display screen 332, or the touch screen 331 and the display screen 332 may be integrated to implement the input and output functions of the communication terminal 300, and after integration, the touch screen may be referred to as a touch display screen for short. In the present application, the display unit 330 may display the application programs and the corresponding operation steps.

Camera 340 may be used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensitive elements convert the optical signals to electrical signals which are then passed to the processor 380 for conversion to digital image signals.

The communication terminal 300 may further comprise at least one sensor 350, such as an acceleration sensor 351, a distance sensor 352, a fingerprint sensor 353, a temperature sensor 354. The communication terminal 300 may also be provided with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, motion sensor, and the like.

The audio circuitry 360, speaker 361, microphone 362 may provide an audio interface between a user and the communication terminal 300. The audio circuit 360 may transmit the electrical signal converted from the received audio data to the speaker 361, and the audio signal is converted by the speaker 361 and output. The communication terminal 300 may be further provided with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 362 converts the collected sound signals into electrical signals, which are received by the audio circuit 360 and converted into audio data, which are then output to the RF circuit 310 for transmission to, for example, another terminal or to the memory 320 for further processing. In this application, the microphone 362 may capture the voice of the user.

Wi-Fi is a short-range wireless transmission technology, and the communication terminal 300 can help a user send and receive e-mails, browse webpages, access streaming media and the like through the Wi-Fi circuit 370, and provides wireless broadband Internet access for the user.

The processor 380 is a control center of the communication terminal 300, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the communication terminal 300 and processes data by running or executing software programs stored in the memory 320 and calling data stored in the memory 320. In some embodiments, processor 380 may include one or more processing units; the processor 380 may also integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a baseband processor, which primarily handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 380. In the present application, the processor 380 may run an operating system, an application program, a user interface display, and a touch response, and the processing method described in the embodiments of the present application. Further, the processor 380 is coupled with the input unit 330 and the display unit 340.

And the bluetooth circuit 381 is used for performing information interaction with other bluetooth devices with bluetooth circuits through a bluetooth protocol. For example, the communication terminal 300 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) also equipped with a bluetooth circuit through the bluetooth circuit 381, so as to perform data interaction.

The communication terminal 300 also includes a power source 390 (e.g., a battery) that provides power to the various components. The power supply may be logically coupled to the processor 380 through a power management system to manage charging, discharging, and power consumption functions through the power management system. The communication terminal 300 may also be configured with power buttons for powering the terminal on and off, and for locking the screen.

Fig. 6 is a block diagram of a software configuration of the communication terminal 300 according to the embodiment of the present invention.

The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, an application layer, an application framework layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom.

The application layer may include a series of application packages.

As shown in fig. 6, the application package may include applications such as camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 6, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

Content providers are used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide a communication function of the communication terminal 300. Such as management of call status (including on, off, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top status bar of the system, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, text information is presented in the status bar, a warning tone is given, the communication terminal vibrates, and an indicator light flashes.

The Android Runtime comprises a core library and a virtual machine. The Android runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. And executing java files of the application program layer and the application program framework layer into a binary file by the virtual machine. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The following exemplifies the workflow of the software and hardware of the communication terminal 300 in connection with capturing a photographing scene.

When the touch screen 331 receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, a time stamp of the touch operation, and other information). The raw input events are stored at the kernel layer. And the application program framework layer acquires the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and taking a control corresponding to the click operation as a control of a camera application icon as an example, the camera application calls an interface of an application framework layer, starts the camera application, further starts a camera drive by calling a kernel layer, and captures a still image or a video through the camera 340.

Based on the display device 200 and the communication terminal 300, the display device 200 can display the content in the communication terminal 300 by establishing a screen-projecting connection, for example, a user projects a video of interest onto the display device 200 for large-screen playing through some application programs with a screen-projecting function. In some embodiments, as shown in fig. 7, to establish the screen-casting connection relationship, the user may issue a screen-casting connection request through the communication terminal 300 and transmit the screen-casting connection request to the display device 200 through the WiFi network. The display device 200 completes the configuration of the transmission protocol according to the screen-projecting connection request, thereby establishing a transmission channel of the screen-projecting data with the communication terminal 300.

In some embodiments, different device screen projection applications for different screen projection protocols are provided in the display device 200, so that a user can perform screen projection display by using screen projection devices of different screen projection protocols. The multiple communication terminals 300 can be used for projecting screens to the same display device 200 based on the same screen projecting protocol or different screen projecting protocols, and screen projecting interfaces of the multiple communication terminals 300 can be displayed on the same user interface synchronously.

In some embodiments, controller 250 is configured to perform a control process for a multi-device on-screen display. The screen-casting connection can be established with the plurality of communication terminals 300 through one screen-casting protocol, and the screen-casting connection can also be established with the plurality of communication terminals 300 through a plurality of screen-casting protocols respectively. For example, the controller 250 establishes a screen-casting connection with a plurality of communication terminals 300 supporting different screen-casting protocols, respectively, using a plurality of screen-casting services, so that the display device 200 receives screen-casting contents of a plurality of screen-casting devices. The display device 200 receives screen-casting requests of the plurality of communication terminals 300 through different screen-casting services, and the communication terminal a and the communication terminal B establish related protocol connection with the display device 200 through different screen-casting protocols to perform screen-casting related operations. Through protocol layer convention, relevant protocol information is carried in the communication process, and the communication terminal 300 can obtain device information, network information and the like of the display device 200.

In some embodiments, the display device 200 receives, in response to the screen projection request, a screen projection request for monitoring the communication terminals 300 supporting different screen projection protocols through screen projection services of the different screen projection protocols, and monitors the connection of the communication terminals 300 by using the screen projection services to build an instruction channel and a data channel, wherein the instruction channel is used for instruction interaction between the communication terminals 300 and the display device 200, and the data channel is used for data interaction between the communication terminals 300 and the display device 200. For example, the screen-casting service a in the display device 200 receives a screen-casting request of the communication terminal a, the screen-casting service B receives a screen-casting request of the communication terminal B, and the screen-casting service is responsible for monitoring the connection of the communication terminal and establishing an instruction channel and a data channel with the communication terminal. Through the instruction channel, the display apparatus 200 can receive a request transmitted by a user through the communication terminal, and through the data channel, the display apparatus 200 can receive related video data and the like transmitted by the communication terminal. After the command channel and the data channel are established, the communication terminal 300 establishes a screen-casting connection with the display device 200.

In some embodiments, the display device 200 performs discovery of the communication terminal 300 through screen projection services of different screen projection protocols, and performs screen projection connection after discovering the communication terminal 300. For example, when there are a plurality of Communication terminals 300 that respectively support NFC (Near Field Communication), airplay (wireless projection technology), and DLNA (Digital Living Network Alliance) projection protocols, the controller 250 receives a projection request of the Communication terminal 300 that supports the NFC protocol by using the NFC protocol service, and listens for connection of the Communication terminal 300 by using the NFC protocol service. For another example, the controller 250 receives a screen-casting request of the communication terminal 300 supporting the Airplay protocol by using the Airplay protocol service, and listens for a connection of the communication terminal 300 by using the Airplay protocol service. Further, for example, the controller 250 receives a screen-casting request of the communication terminal 300 supporting the DLNA protocol using the DLNA protocol service, and listens for the connection of the communication terminal 300 using the DLNA protocol service. The NFC screen projection is based on NFC protocol to discover and connect equipment in a two-way mode, the Airplay screen projection is based on Airplay protocol to discover and connect equipment in a two-way mode, the DLNA push is used for discovering and connecting equipment through DLNA protocol, and other protocols are also used for discovering and connecting equipment through appointments of respective protocols.

The display device 200 may establish a screen-projecting connection with the communication terminal 300 based on any one of the screen-projecting manners provided in the above-described embodiments. That is, the display apparatus 200 establishes a screen-projection connection with only one communication terminal 300, as shown in fig. 8, screen-projection data can exclusively display a screen. In some embodiments, the display device 200 may further establish a screen-projection connection relationship with a plurality of communication terminals through one or more of the screen-projection manners described above. For example, the display apparatus 200 may establish a screen-projection connection with two communication terminals 300 at the same time and divide a display area in a display screen for displaying screen-projection screens shared by the two communication terminals 300, respectively. That is, as shown in fig. 9, a screen shot shared by the communication terminal a may be displayed on the left side of the user interface displayed by the display device 200, and a screen shot shared by the communication terminal B may be displayed on the right side of the user interface.

In order to be able to display a screen-projected screen shared by a plurality of communication terminals 300 on the display apparatus 200, in some embodiments, when the display apparatus 200 establishes a screen-projected connection with a plurality of communication terminals 300, the display apparatus 200 may perform area division, such as halving, trisecting, quartering, etc., on a user interface for screen-projected display. After the plurality of display areas obtained by the area division are obtained, the display apparatus 200 displays the screen projection screen in each divided area, respectively.

In addition, the communication terminal 300 may be placed in different positions according to different usage modes, for example, for a communication terminal 300 such as a smart phone, the communication terminal may be in a vertical screen position and a horizontal screen position during usage. The vertical screen state refers to a state that the width of the displayed picture is smaller than the height, and the horizontal screen state refers to a state that the width of the displayed picture is larger than the height. In different placement states, the communication terminal 300 can present different screen layout manners. Based on this, when the display apparatus 200 displays the screen shots of the plurality of communication terminals 300 by the area division manner, each display area appears as a different screen scale because of a difference in the placement state of the communication terminals 300.

For an application scenario in which the display device is accessed to a single-channel screen projection or a multi-channel screen projection, one screen projection window or a plurality of screen projection windows may be displayed on the screen interface, each screen projection window corresponds to one communication terminal 300 with a successfully connected screen projection, screen projection data sent by the communication terminal 300 is displayed, and a user may project content on the communication terminal 300 onto the display device 200 and view the screen projection content through the display device 200. However, when the display device 200 has been connected to a plurality of communication terminals and the threshold of the number of screen-shot connections of the display device 200 has been reached, and other communication terminals also wish to be connected to the display device 200, it may occur that the display device 200 cuts off the already connected communication terminals and displays the content on the communication terminal newly connected to the display device 200 on the display interface. In this way, the connected communication terminal is suddenly disconnected, the display content on the display device 200 suddenly changes, and the user experience is reduced, or the display device 200 does not respond to the screen-casting connection request of another communication terminal, so that the user cannot determine whether the screen-casting connection is successfully established, and cannot perform subsequent operations.

Therefore, some embodiments of the present application further provide a screen projection control method, which is applied to the display device 200, and when a single-channel screen projection or multiple-channel screen projection is performed, screen projection display processing is performed through interaction among multiple devices, so that the viewing experience of a user is improved. The display device 200 to which the screen projection control method can be applied includes: a display 260, a communicator 220, and a controller 250. The display 260 is configured to display a screen projection interface. The communicator 220 is configured to establish a screen-cast connection with the communication terminal 300. The controller 250 is configured to execute program steps corresponding to the above-mentioned screen projection control method, as shown in fig. 10, which specifically includes the following contents:

and S100, receiving a screen projection request sent by a target communication terminal.

The target communication terminal is the communication terminal 300 which has not established a screen-projection connection with the display device 200. In order to establish a screen-casting connection with the communication terminal 300 supporting different screen-casting protocols, the display device 200 configures a plurality of screen-casting services, monitors screen-casting requests sent by the communication terminal 300 supporting different screen-casting protocols through the different screen-casting services, and can perform protocol interaction with the communication terminal 300 after the screen-casting connection is established.

Accordingly, the user may send a screen-projection request to the display device 200 through the communication terminal 300, wherein the processor 380 of the communication terminal 200 may receive various control instructions input by the user during the use process of the user, and different control instructions have different functions correspondingly. Wherein part of the control instructions may be used to establish a screen-projection connection with the display device 200. The processor 380 may first obtain control instructions input by the user through different interaction manners, and analyze the control instructions to obtain screen projection information in the control instructions, and then generate a screen projection request according to the screen projection information, and send the screen projection request to the display device 200.

For example, a display interface of the communication terminal 300 is controlled to display a screen projection menu page through touch operation of a user on the display screen of the communication terminal 300, the screen projection menu interface includes the searched screen projection devices, the screen projection devices are displayed in a list form on the screen projection page, the user inputs a control instruction for establishing screen projection connection with the target screen projection device to the communication terminal 300 by pressing a selected option of the target screen projection device, a screen projection connection action of the communication terminal 300 is triggered, and a screen projection request is sent to the target screen projection device.

S200, responding to the screen projection request, and detecting the screen end state of the display equipment, wherein the screen end state is an idle state or a busy state; the idle state is used for representing that the current maximum screen-throwing quantity of the display equipment is more than or equal to 1 and a target display position is available; the busy state is used to indicate that the current maximum number of screen throwable of the display device is less than 1 and/or that a target display location is unavailable.

After acquiring the screen projecting request sent by the target communication terminal, the controller 250 may determine the screen end state of the display device 200 by detecting the current maximum screen projectable number and the interface state of the target display position, and determine whether the display device 200 can project the screen according to the screen end state. If the screen end state is the idle state, it indicates that the display device 200 can screen, and if the screen end state is the busy state, it indicates that the display device 200 cannot screen.

In some embodiments, the number of connections of the current communication terminal 300 may be detected. The connection number is compared with a screen-projection connection number threshold of the display device 200, and the current maximum screen-projection number of the display device 200 is determined. As shown in fig. 11, the controller 250 is further configured to execute the following program steps:

and acquiring a screen projection connection quantity threshold of the display equipment. The screen projection connection number threshold is the maximum number of communication terminals allowed to be connected by the display device 200, that is, the number of screen projection paths of the display device 200. The display device 200 performs single screen projection, the number of screen projection paths is 1, the threshold of the screen projection connection number of the display device 200 is 1, the display device 200 performs multi-screen projection, the number of screen projection paths is multiple, for example, the display device 200 performs N screen projections, and the threshold of the screen projection connection number of the display device 200 is N.

The number of connections of the current communication terminal is detected. Since the connection number of the communication terminals may dynamically change along with the connection and disconnection of the screen-projecting connection, the display device 200 may detect the number of the communication terminals 300 establishing the screen-projecting connection with the display device 200 in real time during the operation process, and the connection number of the communication terminals 300 may be obtained by performing respective detection and judgment on each communication terminal in the wireless local area network where the display device 200 is located. For example, when the connection number of the communication terminals 300 needs to be detected, the controller 250 of the display device 200 may call all the communication terminals 300 currently accessed in the wireless network through the communicator 220, and detect whether the communication terminal 300 currently establishes a communication connection with the display device 200 by capturing transmission data of each communication terminal and identifying a screen-shot data flag therefrom.

In some embodiments, the connection number of the communication terminals 300 may be stored in the display device 200 by forming a connection device list in which information of the connected communication terminals 300 is recorded and recording the connection number of all the connected communication terminals 300, updating the connection device list when it is detected that the display device 200 establishes a screen-projected connection with one communication terminal 300, adding one to the connection number of the communication terminals 300, and updating the connection device list when it is detected that the screen-projected connection of the display device 200 with one communication terminal 300 is disconnected, and reducing one to the connection number of the communication terminals 300. In this embodiment, obtaining the connection number of the current communication terminal 300 may be implemented by calling the connection device list and searching the connection number recorded in the list.

After the screen projection connection quantity threshold and the connection quantity of the current communication terminal 300 are obtained, if the connection quantity is smaller than the screen projection connection quantity threshold, it is determined that the current maximum screen projection quantity of the display device is larger than or equal to 1. And if the connection number is equal to the screen projection connection number threshold value, determining that the current maximum screen projection number of the display equipment is less than 1. That is, if the connection number of the current communication terminal 300 reaches the screen-throwing connection number threshold, it indicates that the display apparatus 200 is not screen-throwable, that is, the screen-side state of the display apparatus 200 is a busy state. If the connection number of the current communication terminal 300 does not reach the screen-casting connection number threshold, whether the target display position is available or not can be further detected, and if the target display position is available, screen casting of the display device 200 is indicated, that is, the screen end state of the display device 200 is an idle state.

For example, the threshold of the number of connection screens dropped by the display device 200 is 5, and if it is detected that the number of connection screens of the current communication terminal 300 is 5, the current maximum number of screen drops that can be displayed by the display device 200 is 0, and it is determined that the screen side state of the display device 200 is a busy state. If it is detected that the number of connections of the current communication terminal 300 is 3, the current maximum number of screens projectable of the display device 200 is 2, and further detecting whether the target display position is available, and if the target display position is available, determining that the screen end state of the display device 200 is an idle state.

In some embodiments, the display apparatus 200 may perform single-pass screen projection or multi-pass screen projection, and thus, the screen projection mode of the display apparatus 200 includes a single-pass screen projection mode and a multi-pass screen projection mode, and the user may cause the display apparatus 200 to perform single-pass screen projection or multi-pass screen projection by setting the screen projection mode of the display apparatus 200. When the screen projection mode of the display device 200 is the one-way screen projection mode, the controller 250 is further configured to perform the following program steps:

the number of connections of the current communication terminal is detected.

And if the connection number is 0, determining that the current maximum screen-projectable number of the display equipment is equal to 1.

And if the connection number is 1, determining that the current maximum screen-projectable number of the display equipment is equal to 0.

In this embodiment, when the screen projection mode of the display device 200 is the single-channel screen projection mode, the threshold of the screen projection connection number is 1. That is, the number of connections of the communication terminal 300 is only two values, the display device 200 has established a screen-cast connection with the communication terminal 300, and the number of connections of the communication terminal 300 is 1, or the display device 200 has not established a screen-cast connection with the communication terminal 300, and the number of connections of the communication terminal 300 is 0. Therefore, in order to increase the response speed, the current maximum number of projectable screens of the display apparatus 200 can be directly judged by the detected number of connections of the current communication terminal 300.

For example, when the screen projection mode of the display device 200 is the single-channel screen projection mode, it is detected that the connection number of the current communication terminal is 1, and it is determined that the screen end state of the display device 200 is a busy state. If it is detected that the number of connections of the current communication terminal 300 is 1, it is further detected whether the target display position is available, and if the target display position is available, it is determined that the screen end state of the display device 200 is an idle state.

In some embodiments, the target display position may be a user interface of the display device 200 or a display area in the user interface of the display device 200 designated by the user. The user can perform the screen splitting process on the user interface during the use of the display device 200, that is, the display device 200 can perform the area division, such as halving, trisecting, quartering, etc., on the user interface for screen projection display. By obtaining a plurality of display areas through area division, the display apparatus 200 can display a screen projection picture in each divided display area, and each window can independently display different contents, so that a user can view different program pictures at the same time. The controller 250 is further configured to perform the following program steps:

and detecting the interface state of the target display position in the screen projection interface, wherein the interface state comprises a screen projection state and a non-screen projection state.

And if the interface state is a screen projection state, determining that the target display position is unavailable.

And if the interface state is a non-screen-projection state, determining that the target display position is available.

When it is determined whether the target display position is available, an interface state of the target display position in the screen projection interface of the display device 200 is detected. As shown in fig. 12, the display device 200 performs quarter area division on the user interface for screen projection display into four display areas, namely, an area one, an area two, an area three, and an area four. And if the target display position in the received screen projection request is the second area, detecting the interface state of the second area, and if the interface state is the screen projection state, determining that the target display position is unavailable. And if the interface state is a non-screen-projection state, determining that the target display position is available.

It is to be understood that, if the user does not specify the target display position, when the screen end state of the display device 200 is detected, the idle state is used to indicate that the current maximum screen-projectable number of the display device 200 is greater than or equal to 1, and the busy state is used to indicate that the current maximum screen-projectable number of the display device 200 is less than 1.

S300, if the screen end state is an idle state, screen projection connection is established with the target communication terminal. When the screen end state of the display device 200 is detected to be an idle state, it is indicated that the display device 200 can project a screen, the display device 200 can directly establish screen projection connection with a target communication terminal, and after the screen projection establishment is successful, media data can be projected to the display device 200 from the target communication terminal to be played.

In some embodiments, when the controller 250 performs the screen-projecting connection with the target communication terminal, a screen-projecting data channel is established with the target communication terminal, and after the screen-projecting data channel is successfully established, a screen-projecting window is created at the target display position, where the screen-projecting window is used for displaying the screen-projecting data received through the screen-projecting data channel. For example, as shown in fig. 12, when the current maximum number of screen-throwable screens of the display device 200 is greater than or equal to 1, and it is detected that the target display position is available, that is, the area two in the display interface is available, and after the display device 200 establishes a screen-throwing data channel with the target communication terminal, the controller 250 may create a screen-throwing window in the area two, and display the screen-throwing picture of the target communication terminal through the screen-throwing window.

In some embodiments, when it is detected that the screen end state of the display apparatus 200 is the idle state, the screen end state of the display apparatus 200 may be sent to the target communication terminal, and the target communication terminal, receiving the screen end state of the display apparatus 200, may control the display unit to display a prompt message, where the prompt message is used to prompt the user that the display apparatus 200 may throw the screen, so that the user may prepare for screen throwing data.

For example, the threshold of the number of screen-shot connections of the display device 200 is 3, and taking the user as an example of not specifying the target display position, the display device 200 and the communication terminal A1 have already established the screen-shot connection, and the screen-shot data transmitted by the communication terminal A1 is displayed through the display 260 of the display device 200, as shown in fig. 13. At this time, a screen-casting request which carries terminal information such as a terminal device name and a terminal ip and is sent by the communication terminal A2 is received, and the screen-casting request may represent specific request content by defining a message field, that is, the screen-casting request may include the following content: "cmd" is used to indicate the content of the request command, such as "cmd": "CAST _ CONNECT _ REQUEST" indicates that the current REQUEST is a screen-casting connection REQUEST; "mobile" is used to denote a communication terminal name, such as "mobile": "A2"; "ip" represents a network address; "status" indicates the screen-projection connection state.

The display device 200 receives a screen projection request sent by the communication terminal A2, detects a screen end state of the display device 200, detects that the connection number of the current communication terminal 300 of the display device 200 is 1, and does not reach a screen projection connection number threshold of the display device 200, determines that the screen end state of the display device 200 is an idle state, which indicates that the display device 200 can project a screen currently, establishes screen projection connection between the display device 200 and the communication terminal A2, and sends reply information containing the screen end state to the communication terminal A2, where the reply information can also represent specific information content by defining a message field, that is, the reply information can include the following content: "cmd" is used to indicate that the content of the request is returned, e.g., "cmd": "CAST _ CONNECT _ ACK" indicates that the current reply request content is a reply screen-CAST connection request; "mobile" is used to denote a communication terminal name, such as "mobile": "A2"; "ip" represents a network address; "status" indicates a screen-projection connection state, such as "status": "idle" indicates an idle state.

The communication terminal A2 receives the reply message sent by the display device 200, and displays the prompt message for prompting the user that the display device 200 can make a screen projection and prepare screen projection data.

Meanwhile, when determining that the current screen end state is the idle state, the display device 200 may establish a screen projection data channel with the communication terminal A2 by using a screen projection protocol supported by the communication terminal A2, and create a screen projection window after the screen projection data channel is successfully established, as shown in fig. 14, so as to display screen projection data received through the screen projection data channel through the screen projection window, thereby achieving an effect of synchronous display with the communication terminal A2.

When the display device 200 includes 1 screen projection window, the display control may be displayed in a full screen manner. When the display device 200 includes 2 screen projection windows, the two screen projection windows are in the form of two windows in a left-right vertical layout. As shown in fig. 14, the screen projection data of the communication terminal A1 is displayed in the screen projection window on the right, and the screen projection data of the communication terminal A2 is displayed in the screen projection window on the left. When the number of connections of the communication terminal 300 is changed, the display device 200 may dynamically adjust the display layout of the screen projection screen, and thus, the controller 250 is further configured to perform the steps of:

and after the screen projection data channel is established, updating the connection quantity of the current communication terminal.

And acquiring media configuration information of the target communication terminal, wherein the media configuration information comprises the width-height ratio of a screen projection picture of the target communication terminal. I.e. the ratio of the lateral width to the longitudinal height when displaying the video. In some embodiments, the display screen direction can be identified through the width-height ratio, and when the width-height ratio is greater than or equal to 1, namely the width is greater than or equal to the height, a landscape display mode is presented; when the width-height ratio is less than 1, namely the width is less than the height, a vertical screen display mode is presented. The creation of the screen projection window can be restricted according to the width-height ratio and the screen direction indicated in the media configuration information, so that the screen projection window and the corresponding communication terminal 300 keep the same width-height ratio and display direction, and the problem of confusion of media related configuration after screen projection can be avoided.

Planning the layout mode of the screen projection window and creating the screen projection window matched with the media configuration information according to the updated connection number and the media configuration information.

For different types of display devices 200, the screen size information of the display 260 is different, and the layout manner conforming to the screen specification is also different. Therefore, in the display apparatus 200 or the server 400, a database may be maintained, and layout templates corresponding to the screen size specification and the different connection numbers of the display apparatus 200 may be stored in the database. Therefore, when the layout mode of the screen projection window is planned according to the connection number and the media configuration information, the display device 200 may determine the optimal layout mode adapted to the current display state by matching the layout template in the database according to the connection number and the media configuration information.

For example, as shown in fig. 14, the display device 200 currently has two screen-projecting windows for displaying screen-projecting data of the communication terminal A1 and the communication terminal A2, at this time, the display device 200 establishes a screen-projecting connection with the communication terminal A3, the current connection number of the communication terminal is updated to 3, the media configuration information of the communication terminal A3 is acquired, the layout mode is planned according to the width-height ratio and the screen direction and the connection number indicated in the media configuration information, as shown in fig. 15, a vertical display area layout mode and two horizontal display area layout modes may be adapted, and the screen-projecting windows are created in the display area according to the media configuration information.

S400, if the screen end state is a busy state, sending a notification message to the target communication terminal, wherein the notification message is used for enabling the target communication terminal to display first prompt information. When the screen end state of the display device 200 is detected to be a busy state, it indicates that the display device 200 is not capable of screen projection, the display device 200 may notify the screen end state of the display device 200 to the target communication terminal, and the target communication terminal receives a notification message including the screen end state, and may control the display unit to display a first prompt message for prompting the user that the display device 200 is not capable of screen projection currently.

For example, the display device 200, the communication terminal A1, the communication terminal A2, and the communication terminal A3 have already established screen-casting connections, and reach a threshold of the screen-casting connection number of the display device 200, and if a screen-casting request of the communication terminal A4 is received at this time, it is detected that the screen-side state of the display device 200 is a busy state, the display device 200 may notify the screen-side state to the communication terminal A4, and the notification message may indicate specific request content by defining a message field, that is, the notification message may include the following content: "cmd" is used to indicate the content of a reply notification message, such as "cmd": "CAST _ CONNECT _ ACK" indicates that the current notification message is the screen-CAST connection notification content; "mobile" is used to denote a communication terminal name, such as "mobile": "A4"; "ip" represents a network address; "status" indicates a screen-casting connection state, such as "status": "busy" indicates that the screen end state is busy.

The communication terminal A4 receives the notification message including the screen end state, and may control the display unit to display first prompt information for prompting the user that the display device 200 is currently not capable of screen projection.

In some embodiments, before sending the screen projection request to the display device 200 through the communication terminal 300, the user may first send a search request to the searched display device 200 through the communication terminal 300, determine the screen end state of the display device 200 according to a reply received from the display device 200, and send the screen projection request when determining that the screen end state of the display device 200 is the idle state, and accordingly, the controller 250 is further configured to execute the following program steps:

and receiving a search request sent by the target communication terminal.

And responding to the search request, and detecting the screen end state of the display equipment.

And sending the screen end state of the display equipment to the target communication terminal so as to enable the target communication terminal to display third prompt information.

In this embodiment, after receiving a search request sent by a target communication terminal, the display device 200 may send device information including a screen end state to the target communication terminal, where the device information may include a device name, a device id, a function name, an ip, a display device end state, and the like of the display device 200, and the device information may indicate specific request content by defining a message field, that is, the device information may include the following content: "devicename" is used to denote a device name, such as "devicename": "Living room television" means that the current device is a living room television; "deviceid" is used to denote a device id; "featured" is used to denote the attribute id; "featurename" means a function name, such as "featurename": "screen projection"; "ip" represents a network address; "status" indicates the screen-projection connection state. Such as "status": "idle" indicates an idle state.

The target communication terminal acquires that the screen end state of the display device 200 is an idle state, may display third prompt information for inquiring of a user whether to perform screen projection, and establishes screen projection connection with the display device 200 after the user confirms.

In some embodiments, the user may replace the screen-shot content that is not desired to be continuously played in the display interface according to his/her own intention, that is, the user may specify a target display position in the display interface, and replace the communication terminal 300 corresponding to the screen-shot data in the target display position with another communication terminal 300, and therefore, as shown in fig. 16, the controller 250 is further configured to execute the following program steps:

and if the screen end state is a busy state, detecting the communication terminal corresponding to the target display position.

And sending an exit instruction to the communication terminal corresponding to the target display position, and receiving a response message fed back by the communication terminal corresponding to the target display position, wherein the response message comprises an exit agreement and an exit disagreement.

If the response message is that quitting is approved, the communication terminal corresponding to the target display position is disconnected from screen-casting connection, and after the screen-casting connection is disconnected, screen-casting connection is established with the target communication terminal.

And if the response message is not approved to quit, sending the response message to the target communication terminal so that the target communication terminal displays second prompt information.

As shown in fig. 17, the threshold of the number of screen-projection connections of the display device 200 is 4, the display device 200 has already established a screen-projection connection with the communication terminal A1, the communication terminal A2, the communication terminal A3, and the communication terminal A4, at this time, when receiving a screen-projection request sent by the communication terminal A5, the display device 200 sends notification information including a screen-end state to the communication terminal A5, and the communication terminal A5 displays first prompt information for prompting a user that the display device 200 is currently not capable of projecting a screen.

As shown in fig. 18, the communication terminal A5 receives notification information including a screen end state sent by the display device 200, and displays first prompt information, where the first prompt information may further include a replacement option, where the replacement option includes a display area of the display device 200, which is an area one, an area two, an area three, and an area four, each display area corresponds to the communication terminal A1, the communication terminal A2, the communication terminal A3, and the communication terminal A4, respectively, and a user may input a replacement instruction through the communication terminal A5, where the replacement instruction includes a target display area to be replaced.

When the user selects the second replacement area, that is, the communication terminal A2 is replaced with the communication terminal A5, the display device 200 receives a replacement instruction sent by the communication terminal A5, and sends a quit instruction to the communication terminal A2, where the quit instruction indicates specific request content by defining a message field, that is, the quit instruction may include the following content: "cmd" is used to denote the contents of an instruction, such as "cmd": "CAST _ FINISH" indicates that the current instruction is an exit instruction; "mobile" is used to denote a communication terminal name, such as "mobile": "A2"; "ip" represents a network address.

The communication terminal A2 receives the quit instruction sent by the display device 200, sends a response message to the display device 200, disconnects the screen-casting connection with the communication terminal A2 and cancels the screen-casting window of the communication terminal A2 if the response message is that quit is approved, establishes the screen-casting connection with the communication terminal A5 after disconnecting the screen-casting connection, creates a screen-casting window in the area two as shown in fig. 19, and displays the screen-casting content of the communication terminal A5 in the newly created screen-casting window.

And if the response message is not approved to quit, sending the response message to the communication terminal A5, receiving the response message by the communication terminal A5, and displaying second prompt information for prompting the user that the response message cannot be replaced.

Based on the display device 200, in some embodiments of the present application, there is further provided a communication terminal 300, including: a display unit 330, a communication unit, and a processor 380. Wherein the display unit 330 is configured to display a terminal interface, and the communication unit is configured to establish a screen-casting connection with the display device 200; as shown in fig. 20, the processor 380 is configured to perform the following program steps:

Based on the display device 200 and the communication terminal 300, some embodiments of the present application further provide a screen projection control method, which includes the following steps:

and if the screen end state is a busy state, the target communication terminal receives the notification message sent by the display equipment and displays prompt information.

According to the technical scheme, the screen projection control method provided by the embodiment can judge whether the current display equipment can perform screen projection or not by detecting the screen end state of the display equipment when receiving the screen projection request sent by the communication terminal. And then sending a notification message containing the screen end state to the communication terminal, wherein if the screen end state is a busy state, the communication terminal can display a prompt message after receiving the notification message, and if the screen end state is an idle state, the communication terminal can establish screen projection connection with the display equipment after receiving the notification message. The method can perform screen projection display processing through interaction among multiple devices when screen projection is performed in a single-path mode or multiple-path mode, and improves user experience.

The embodiments provided in the present application are only a few examples of the general concept of the present application, and do not limit the scope of the present application. Any other embodiments extended according to the scheme of the present application without inventive efforts will be within the scope of protection of the present application for a person skilled in the art.

Claims

1. A display device, comprising:

a display configured to display a screen projection interface;

a communicator configured to establish a screen-casting connection with a communication terminal;

a controller configured to:

receiving a screen projection request sent by a target communication terminal;

2. The display device of claim 1, wherein the controller performs detecting a screen-side status of the display device, and is further configured to:

acquiring a screen projection connection quantity threshold of the display equipment;

detecting the connection quantity of the current communication terminal;

if the connection number is smaller than the screen projection connection number threshold value, determining that the current maximum screen projection number of the display equipment is larger than or equal to 1;

and if the connection number is equal to the screen projection connection number threshold value, determining that the current maximum screen projection number of the display equipment is less than 1.

3. The display device of claim 1, wherein the screen projecting modes of the display device comprise a single-way screen projecting mode and a multi-way screen projecting mode, and when the screen projecting mode of the display device is the single-way screen projecting mode, the controller is further configured to:

detecting the connection quantity of the current communication terminal;

if the connection number is 0, determining that the current maximum screen-projectable number of the display equipment is equal to 1;

4. The display device of claim 1, wherein the controller is further configured to:

detecting interface states of the target display position in the screen projection interface, wherein the interface states comprise a screen projection state and a non-screen projection state;

if the interface state is a screen projection state, determining that the target display position is unavailable;

5. The display device according to claim 1, wherein the controller is further configured to:

if the screen end is in a busy state, detecting a communication terminal corresponding to the target display position;

sending an exit instruction to the communication terminal corresponding to the target display position, and receiving a response message fed back by the communication terminal corresponding to the target display position, wherein the response message comprises an exit agreement and an exit disagreement;

if the response message is an exit agreement, the communication terminal corresponding to the target display position is disconnected from screen-casting connection, and after the screen-casting connection is disconnected, screen-casting connection is established with the target communication terminal;

and if the response message does not agree with exit, sending the response message to the target communication terminal so that the target communication terminal displays second prompt information.

6. The display device according to claim 1, wherein the controller is further configured to:

receiving a search request sent by the target communication terminal;

responding to the search request, and detecting the screen end state of the display equipment;

7. The display device of claim 1, wherein the controller performs establishing a screen-cast connection with the target communication terminal, and is further configured to:

establishing a screen projection data channel with the target communication terminal;

and after the screen projection data channel is successfully established, establishing a screen projection window at the target display position, wherein the screen projection window is used for displaying the screen projection data received through the screen projection data channel.

8. The display device of claim 7, wherein the controller is further configured to:

after the screen projection data channel is established, updating the connection quantity of the current communication terminal;

acquiring media configuration information of the target communication terminal, wherein the media configuration information comprises the width-height ratio of a screen projection picture of the target communication terminal;

9. A communication terminal, comprising:

a display unit configured to display a terminal interface;

a communication unit configured to establish a screen-projection connection with a display device;

a processor configured to:

10. A screen projection control method is characterized by comprising the following steps: