CN111931692A - Display device and image recognition method - Google Patents

Abstract

The present application discloses a display device and an image recognition method. When image recognition is required, the display device controller acquires the image to be processed; when the to-be-processed image and the first target image meet preset recognition conditions, acquires the recognition based on the first target image As a result, the first target image is the original image corresponding to the target area in the image to be processed; when the image to be processed and the first target image do not meet the preset recognition conditions, the recognition result based on the second target image is obtained, and the second target image is based on The first target image is obtained and meets the preset recognition conditions; finally, the recognition result is displayed in the user interface. After the image to be processed is acquired, it is first determined whether the image to be processed satisfies the preset recognition conditions. If the image to be processed does not meet the preset recognition conditions, the image to be processed is processed into an image that satisfies the preset recognition conditions, and then the recognition is performed. , so as to improve the accuracy of person recognition in the image.

Description

Display device and image recognition method

technical field

The present application relates to the technical field of display devices, and in particular, to a display device and an image recognition method.

Background technique

The display device can provide the user with playback pictures such as audio, video, and pictures. Today, display devices can not only provide users with the content of live TV programs received through data broadcasting, but also provide users with various application and service content such as online video and online games.

Applications that support image recognition in display devices can generally implement face recognition. For example, when a user uses video playback software to watch a video program, by operating the buttons provided in the playback interface, the display device can be instructed to identify the person in the currently playing video screen.

However, affected by the size of the face area and the size of the image resolution, if the size of the face area in the image is small or the image resolution is low, it is easy to fail to identify or to identify errors due to insufficient feature extraction points.

SUMMARY OF THE INVENTION

The present application provides a display device and an image recognition method to solve the problem that in a face recognition scene, if the size of the face area in the image is small or the image resolution is low, it is easy to fail to recognize or to recognize errors due to insufficient feature extraction points.

In a first aspect, the present application provides a display device, including:

a display for displaying the user interface;

Controller, configured as:

Get the image to be processed;

When the to-be-processed image and the first target image satisfy a preset recognition condition, acquiring a recognition result based on the first target image, where the first target image is the original image corresponding to the target area in the to-be-processed image;

When the to-be-processed image and the first target image do not meet the preset recognition conditions, obtain a recognition result based on the second target image, the second target image is generated according to the first target image and satisfies the preset recognition condition;

The recognition result is displayed in the user interface.

In a second aspect, the present application further provides an image recognition method applied to a display device, the method comprising:

Get the image to be processed;

When the to-be-processed image and the first target image satisfy a preset recognition condition, acquiring a recognition result based on the first target image, where the first target image is the original image corresponding to the target area in the to-be-processed image;

When the to-be-processed image and the first target image do not meet the preset recognition conditions, obtain a recognition result based on the second target image, the second target image is generated according to the first target image and satisfies the preset recognition condition;

The recognition result is displayed.

In a third aspect, the present application also provides a server, including a processor and a memory, the processor is configured to:

Receive data corresponding to the to-be-processed image sent by the display device;

Determine the to-be-processed image and the first target image according to the data corresponding to the to-be-processed image, where the first target image is the original image corresponding to the target area in the to-be-processed image;

judging whether the to-be-processed image and the first target image satisfy a preset recognition condition;

When the to-be-processed image and the first target image meet preset recognition conditions, acquiring a recognition result based on the first target image;

When the to-be-processed image and the first target image do not meet the preset recognition conditions, obtain a recognition result based on the second target image, the second target image is generated according to the first target image and satisfies the preset recognition condition;

Return the identification result to the display device.

In a fourth aspect, the present application further provides an image recognition method, which is applied to a server, and the method includes:

Receive data corresponding to the to-be-processed image sent by the display device;

Determine the to-be-processed image and the first target image according to the data corresponding to the to-be-processed image, where the first target image is the original image corresponding to the target area in the to-be-processed image;

judging whether the to-be-processed image and the first target image satisfy a preset recognition condition;

When the to-be-processed image and the first target image meet preset recognition conditions, acquiring a recognition result based on the first target image;

When the to-be-processed image and the first target image do not meet the preset recognition conditions, obtain a recognition result based on the second target image, the second target image is generated according to the first target image and satisfies the preset recognition condition;

Return the identification result to the display device.

It can be seen from the above technical solutions that the present application provides a display device and an image recognition method. After obtaining the image to be processed, the method first determines whether the image to be processed meets the preset identification conditions, and if the image does not meet the preset identification conditions condition, the to-be-processed image is processed into an image that satisfies the preset recognition condition, and then the recognition is performed, thereby improving the accuracy of the person recognition in the image.

Description of drawings

In order to illustrate the technical solutions of the present application more clearly, the accompanying drawings that need to be used in the embodiments will be briefly introduced below. Other drawings can also be obtained from these drawings.

FIG. 1 exemplarily shows a schematic diagram of an operation scene between a display device and a control apparatus according to some embodiments;

FIG. 2 exemplarily shows a hardware configuration block diagram of a display device 200 according to some embodiments;

FIG. 3 exemplarily shows a hardware configuration block diagram of the control device 100 according to some embodiments;

FIG. 4 exemplarily shows a schematic diagram of software configuration in the display device 200 according to some embodiments;

FIG. 5 exemplarily shows a schematic diagram of displaying an icon control interface of an application in the display device 200 according to some embodiments;

6 is a flowchart of an image recognition method according to an exemplary embodiment of the present application;

7 is a schematic diagram of the first target image and the second target image exemplarily shown in the application;

FIG. 8 is a flowchart of another image recognition method according to an exemplary embodiment of the present application.

Detailed ways

Embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following examples are not intended to represent all implementations consistent with this application. are merely exemplary of systems and methods consistent with some aspects of the present application as recited in the claims.

Based on the exemplary embodiments described in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the appended claims of this application. Furthermore, although the disclosures in this application have been presented in terms of illustrative example or instances, it should be understood that various aspects of this disclosure may also constitute a complete embodiment in isolation.

It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the embodiments described below, rather than intended to limit the embodiments of the present application. Unless otherwise specified, these terms are to be understood according to their ordinary and ordinary meanings.

The terms "first", "second", "third", etc. in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit specific Order or sequence, unless otherwise indicated. It should be understood that the terms so used are interchangeable under appropriate circumstances, eg, can be implemented in an order other than those given in the illustration or description of the embodiments of the present application.

Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover but not exclusively include, for example, a product or device incorporating a series of components is not necessarily limited to those explicitly listed, but may include No other components are expressly listed or inherent to these products or devices.

The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware or/and software code capable of performing the function associated with that element.

As used in this application, the term "remote control" refers to a component of an electronic device, such as the display device disclosed in this application, that can wirelessly control the electronic device, usually over a short distance. Generally, infrared and/or radio frequency (RF) signals and/or Bluetooth are used to connect with electronic devices, and functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors may also be included. For example, a hand-held touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

The term "gesture" used in this application refers to a user's behavior that is used by a user to express an expected thought, action, purpose/or result through an action such as a change of hand shape or hand movement.

FIG. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , a user may operate the display apparatus 200 through the mobile terminal 300 and the control apparatus 100 .

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., and controls the display device 200 by wireless or other wired methods. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like. For example, the user can control the display device 200 by inputting corresponding control commands through the volume up/down key, channel control key, up/down/left/right movement keys, voice input key, menu key, power-on/off key, etc. on the remote control. function.

In some embodiments, mobile terminals, tablet computers, computers, notebook computers, and other smart devices may also be used to control the display device 200 . For example, the display device 200 is controlled using an application running on the smart device. The app can be configured to provide users with various controls in an intuitive user interface (UI) on the screen associated with the smart device.

In some embodiments, the mobile terminal 300 may install a software application with the display device 200 to implement connection communication through a network communication protocol, so as to achieve the purpose of one-to-one control operation and data communication. For example, a control command protocol can be established between the mobile terminal 300 and the display device 200, the remote control keyboard can be synchronized to the mobile terminal 300, and the function of controlling the display device 200 can be realized by controlling the user interface on the mobile terminal 300. The audio and video content displayed on the mobile terminal 300 may also be transmitted to the display device 200 to implement a synchronous display function.

As also shown in FIG. 1 , the display device 200 also performs data communication with the server 400 through various communication methods. The display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 . For example, the display device 200 interacts by sending and receiving information, and electronic program guide (EPG), receiving software program updates, or accessing a remotely stored digital media library. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. Other network service contents such as video-on-demand and advertising services are provided through the server 400 .

The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size and resolution are not limited. Those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.

The display device 200 may additionally provide a smart IPTV function that provides computer-supported functions, including but not limited to, IPTV, smart TV, Internet Protocol TV (IPTV), and the like, in addition to the broadcast receiving TV function.

FIG. 2 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to the exemplary embodiment.

In some embodiments, the display device 200 includes a controller 250, a tuner 210, a communicator 220, a detector 230, an input/output interface 255, a display 275, an audio output interface 285, a memory 260, a power supply 290, At least one of the user interface 265 and the external device interface 240 .

In some embodiments, the display 275, for receiving the image signal from the output of the first processor, performs components for displaying video content and images and a menu manipulation interface.

In some embodiments, the display 275 includes a display screen component for presenting pictures, and a driving component for driving image display.

In some embodiments, the video content displayed may be from broadcast television content or various broadcast signals that may be received via wired or wireless communication protocols. Alternatively, various image contents sent from the network server side can be displayed and received from the network communication protocol.

In some embodiments, display 275 is used to present a user-manipulated UI interface generated in display device 200 and used to control display device 200 .

In some embodiments, depending on the type of display 275, a driving component for driving the display is also included.

In some embodiments, display 275 is a projection display, and may also include a projection device and projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or external servers according to various communication protocol types. For example, the communicator may include at least one of a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip and other network communication protocol chips or a near field communication protocol chip, and an infrared receiver.

In some embodiments, the display device 200 may establish control signal and data signal transmission and reception between the communicator 220 and the external control device 100 or the content providing device.

In some embodiments, the user interface 265 may be used to receive infrared control signals from the control device 100 (eg, an infrared remote control, etc.).

In some embodiments, the detector 230 is a signal used by the display device 200 to collect the external environment or interact with the outside.

In some embodiments, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity, and can adaptively display parameter changes and the like by collecting ambient light.

In some embodiments, the detector 230 may also include an image collector, such as a camera, a camera, etc., which can be used to collect external environment scenes, and used to collect user attributes or interactive gestures with the user, and can adaptively change display parameters, User gestures can also be recognized to implement functions that interact with users.

In some embodiments, detector 230 may also include a temperature sensor or the like, such as by sensing ambient temperature.

In some embodiments, the display device 200 can adaptively adjust the display color temperature of the image. For example, when the temperature is relatively high, the display device 200 can be adjusted to display a relatively cool color temperature of the image, or when the temperature is relatively low, the display device 200 can be adjusted to display a warmer color of the image.

In some embodiments, the detector 230 may also be a sound collector or the like, such as a microphone, which may be used to receive the user's voice. Exemplarily, a voice signal including a control instruction of the user to control the display device 200, or collecting ambient sounds, is used to identify the type of the environment scene, so that the display device 200 can adaptively adapt to the ambient noise.

In some embodiments, as shown in FIG. 2 , the input/output interface 255 is configured to enable data transfer between the controller 250 and other external devices or other controllers 250 . Such as receiving video signal data and audio signal data of external equipment, or command instruction data, etc.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: any one or more of a high-definition multimedia interface HDMI interface, an analog or data high-definition component input interface, a composite video input interface, a USB input interface, an RGB port, etc. interface. It is also possible to form a composite input/output interface by a plurality of the above-mentioned interfaces.

In some embodiments, as shown in FIG. 2 , the tuner and demodulator 210 is configured to receive broadcast television signals through wired or wireless reception, and can perform modulation and demodulation processing such as amplification, frequency mixing, and resonance, from multiple wireless Or demodulate the audio and video signal from the cable broadcast TV signal, the audio and video signal may include the TV audio and video signal carried in the frequency of the TV channel selected by the user, and the EPG data signal.

In some embodiments, the frequency demodulated by the tuner-demodulator 210 is controlled by the controller 250, and the controller 250 can send a control signal according to the user's selection, so that the modem responds to the user-selected TV signal frequency and modulates and demodulates the frequency. The frequency carried by the television signal.

In some embodiments, broadcast television signals may be classified into terrestrial broadcast signals, cable broadcast signals, satellite broadcast signals, or Internet broadcast signals, etc. according to different broadcast formats of the television signals. Or according to different modulation types, it can be divided into digital modulation signal, analog modulation signal, etc. Or, it can be divided into digital signals, analog signals, etc. according to different types of signals.

In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box Wait. In this way, the set-top box outputs the modulated and demodulated television audio and video signals of the received broadcast television signals to the main device, and the main device receives the audio and video signals through the first input/output interface.

In some embodiments, the controller 250, through various software control programs stored on the memory, controls the operation of the display device and responds to user operations. The controller 250 may control the overall operation of the display apparatus 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any of the selectable objects, such as a hyperlink or an icon. Operations related to the selected object, such as displaying operations linked to hyperlinked pages, documents, images, etc., or executing operations corresponding to the icon. The user command for selecting the UI object may be an input command through various input devices (eg, a mouse, a keyboard, a touchpad, etc.) connected to the display device 200 or a voice command corresponding to a voice spoken by the user.

As shown in FIG. 2 , the controller 250 includes a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a video processor 270, an audio processor 280, and other processors 253 (For example: at least one of a graphics processing unit (Graphics Processing Unit, GPU), a central processing unit 254 (Central Processing Unit, CPU), a communication interface (Communication Interface), and a communication bus 256 (Bus). Wherein, the communication bus connects various parts.

In some embodiments, RAM 251 is used to store temporary data for the operating system or other running programs.

In some embodiments, ROM 252 is used to store various system startup instructions.

In some embodiments, ROM 252 is used to store a basic input output system, referred to as a Basic Input Output System (BIOS). It is used to complete the power-on self-check of the system, the initialization of each functional module in the system, the driver program of the basic input/output of the system, and the boot operating system.

In some embodiments, when a power-on signal is received, the power supply of the display device 200 starts, and the CPU executes the system start-up instruction in the ROM 252, and copies the temporary data of the operating system stored in the memory to the RAM 251, so as to facilitate the start-up or running operation system. After the startup of the operating system is completed, the CPU copies the temporary data of various application programs in the memory to the RAM 251, so as to facilitate the startup or operation of various application programs.

In some embodiments, processor 254 executes operating system and application program instructions stored in memory. And various application programs, data and content are executed according to various interactive instructions received from the external input, so as to finally display and play various audio and video content.

In some exemplary embodiments, processor 254 may include multiple processors. The plurality of processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or an operation of displaying a picture in the normal mode. One or more sub-processors for an operation in a state such as standby mode.

In some embodiments, the graphics processor 253 is used to generate various graphic objects, such as: icons, operation menus, and user input instructions to display graphics and the like. It includes an operator, which performs operations by receiving various interactive instructions input by the user, and displays various objects according to the display properties. and includes a renderer, which renders various objects obtained based on the operator, and the rendered objects are used for displaying on a display.

In some embodiments, the video processor 270 is configured to receive the external video signal and perform decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal. After video processing, a signal that can be directly displayed or played on the display device 200 can be obtained.

In some embodiments, the video processor 270 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.

Wherein, the demultiplexing module is used for demultiplexing the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module demultiplexes it into video signals and audio signals.

The video decoding module is used to process the demultiplexed video signal, including decoding and scaling.

The image synthesizing module, such as an image synthesizer, is used for superimposing and mixing the GUI signal generated by the graphics generator according to the user's input or itself, and the zoomed video image, so as to generate an image signal that can be displayed.

The frame rate conversion module is used to convert the input video frame rate, such as converting 60Hz frame rate to 120Hz frame rate or 240Hz frame rate. The usual format is implemented by means of frame insertion.

The display formatting module is used for converting the received frame rate into the video output signal, and changing the signal to conform to the display format signal, such as outputting the RGB data signal.

In some embodiments, the graphics processor 253 may be integrated with the video processor, or may be separately configured. When the integrated setting is performed, the processing of the graphics signal output to the display may be performed. When the separate setting is performed, different functions may be performed respectively. For example, GPU+FRC (Frame Rate Conversion)) architecture.

In some embodiments, the audio processor 280 is configured to receive an external audio signal, perform decompression and decoding, and noise reduction, digital-to-analog conversion, and amplification processing according to a standard codec protocol of the input signal to obtain a The sound signal played in the speaker.

In some embodiments, the video processor 270 may comprise one or more chips. The audio processor may also include one or more chips.

In some embodiments, the video processor 270 and the audio processor 280 may be separate chips, or may be integrated into one or more chips together with the controller.

In some embodiments, the audio output, under the control of the controller 250, receives the sound signal output by the audio processor 280, such as the speaker 286, and in addition to the speaker carried by the display device 200 itself, can be output to an external device. The external audio output terminal of the device, such as an external audio interface or an earphone interface, etc., may also include a short-range communication module in the communication interface, such as a Bluetooth module for outputting sound from a Bluetooth speaker.

The power supply 290, under the control of the controller 250, provides power supply support for the display device 200 with the power input from the external power supply. The power supply 290 may include a built-in power supply circuit installed inside the display device 200 , or may be an external power supply installed in the display device 200 to provide an external power supply interface in the display device 200 .

The user interface 265 is used for receiving user input signals, and then sending the received user input signals to the controller 250 . The user input signal may be a remote control signal received through an infrared receiver, and various user control signals may be received through a network communication module.

In some embodiments, the user inputs user commands through the control device 100 or the mobile terminal 300 , the user input interface is based on the user's input, and the display device 200 responds to the user's input through the controller 250 .

In some embodiments, the user may input user commands on a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

In some embodiments, a "user interface" is a medium interface for interaction and information exchange between an application program or an operating system and a user, which enables conversion between an internal form of information and a form acceptable to the user. A commonly used form of user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, wherein the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. visual interface elements.

The memory 260 includes storing various software modules for driving the display device 200 . For example, various software modules stored in the first memory include at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

The basic module is used for signal communication between various hardwares in the display device 200, and is a low-level software module that sends processing and control signals to the upper-layer module. The detection module is a management module used to collect various information from various sensors or user input interfaces, perform digital-to-analog conversion, and analyze and manage.

For example, the speech recognition module includes a speech parsing module and a speech instruction database module. The display control module is a module used to control the display to display image content, and can be used to play information such as multimedia image content and UI interface. Communication module, a module for control and data communication with external devices. The browser module is a module for performing data communication between browsing servers. Service modules are used to provide various services and modules including various applications. At the same time, the memory 260 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effect diagrams of focus objects, and the like.

FIG. 3 exemplarily shows a block diagram of the configuration of the control device 100 according to an exemplary embodiment. As shown in FIG. 3 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface, a memory, and a power supply.

The control device 100 is configured to control the display device 200 , and can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 . For example, the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operations.

In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may install various applications for controlling the display device 200 according to user requirements.

In some embodiments, as shown in FIG. 1 , the mobile terminal 300 or other intelligent electronic device can perform a similar function of controlling the device 100 after installing an application for operating the display device 200 . For example, the user can install the application, various function keys or virtual buttons of the graphical user interface provided on the mobile terminal 300 or other intelligent electronic devices, so as to realize the function of controlling the physical keys of the device 100 .

The controller 110 includes a processor 112 and RAM 113 and ROM 114, a communication interface 130, and a communication bus. The controller is used to control the operation and operation of the control device 100, as well as the communication and cooperation among the internal components and the external and internal data processing functions.

The communication interface 130 realizes the communication of control signals and data signals with the display device 200 under the control of the controller 110 . For example, the received user input signal is sent to the display device 200 . The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131 , a Bluetooth module 132 , and an NFC module 133 .

The user input/output interface 140, wherein the input interface includes at least one of other input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a key 144. For example, the user can implement the user command input function through actions such as voice, touch, gesture, pressing, etc. The input interface converts the received analog signal into a digital signal, and converts the digital signal into a corresponding command signal, and sends it to the display device 200.

The output interface includes an interface for transmitting received user instructions to the display device 200 . In some embodiments, it can be an infrared interface or a radio frequency interface. For example, when the infrared signal interface is used, the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol, and sent to the display device 200 through the infrared sending module. For another example, when a radio frequency signal interface is used, the user input command needs to be converted into a digital signal, and then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmission terminal.

In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input-output interface 140 . The control device 100 is configured with a communication interface 130, such as modules such as WiFi, Bluetooth, and NFC, and can send user input instructions to the display device 200 through WiFi protocol, Bluetooth protocol, or NFC protocol encoding.

The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 200 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

The power supply 180 is used to provide operating power support for each element of the control device 100 under the control of the controller. Can battery and related control circuit.

In some embodiments, a system may include a kernel (Kernel), a command parser (shell), a file system, and applications. Together, the kernel, shell, and file system make up the basic operating system structures that allow users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. An application is compiled into machine code after startup, forming a process.

Referring to FIG. 4 , in some embodiments, the system is divided into four layers, from top to bottom, the applications layer (referred to as “application layer”), the application framework layer (referred to as “framework layer”) "), the Android runtime (Android runtime) and the system library layer (referred to as "system runtime library layer"), and the kernel layer.

In some embodiments, at least one application program runs in the application program layer, and these application programs may be a Window program, a system setting program, a clock program, a camera application, etc. built into the operating system; they may also be developed by a third party The application programs developed by the author, such as the Hijian program, the K song program, the magic mirror program, etc. During specific implementation, the application package in the application layer is not limited to the above examples, and may actually include other application packages, which is not limited in this embodiment of the present application.

The framework layer provides an application programming interface (application programming interface, API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which decides to let the applications in the application layer take action. The application program can access the resources in the system and obtain the services of the system during execution through the API interface.

As shown in FIG. 4 , the application framework layer in the embodiment of the present application includes managers (Managers), content providers (Content Provider), etc., wherein the manager includes at least one of the following modules: the activity manager (ActivityManager) is used with Interact with all activities running in the system; Location Manager is used to provide system services or applications with access to system location services; Package Manager is used to retrieve files currently installed on the device. Various information related to the application package; Notification Manager (Notification Manager) is used to control the display and clearing of notification messages; Window Manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers and desktops on the user interface part.

In some embodiments, the activity manager is used to: manage the life cycle of each application and the usual navigation and fallback functions, such as controlling the exit of the application (including switching the user interface currently displayed in the display window to the system desktop), opening the , back (including switching the currently displayed user interface in the display window to the upper-level user interface of the currently displayed user interface), and the like.

In some embodiments, the window manager is used to manage all window programs, such as obtaining the size of the display screen, judging whether there is a status bar, locking the screen, taking screenshots, and controlling the change of the display window (for example, reducing the display window to display, shaking display, twisting deformation display, etc.), etc.

In some embodiments, the system runtime layer provides support for the upper layer, that is, the framework layer. When the framework layer is used, the Android operating system will run the C/C++ library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is the layer between hardware and software. As shown in Figure 4, the kernel layer at least includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch sensors, pressure sensors, etc.), etc.

In some embodiments, the kernel layer further includes a power driver module for power management.

In some embodiments, software programs and/or modules corresponding to the software architecture in FIG. 4 are stored in the first memory or the second memory shown in FIG. 2 or FIG. 3 .

In some embodiments, taking the magic mirror application (photography application) as an example, when the remote control receiving device receives the input operation of the remote control, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the input operation into the original input event (including the value of the input operation, the timestamp of the input operation and other information). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, identifies the control corresponding to the input event according to the current position of the focus, and regards the input operation as a confirmation operation, and the control corresponding to the confirmation operation is the control of the magic mirror application icon. The mirror application calls the interface of the application framework layer, starts the mirror application, and then starts the camera driver by calling the kernel layer to capture still images or videos through the camera.

In some embodiments, for a display device with a touch function, taking a split-screen operation as an example, the display device receives an input operation (such as a split-screen operation) performed by the user on the display screen, and the kernel layer can generate corresponding input operations according to the input operation. Enter an event and report the event to the application framework layer. The window mode (such as multi-window mode) and window position and size corresponding to the input operation are set by the activity manager of the application framework layer. The window management of the application framework layer draws the window according to the settings of the activity manager, and then sends the drawn window data to the display driver of the kernel layer, and the display driver displays the corresponding application interface in different display areas of the display screen.

In some embodiments, as shown in FIG. 5, the application layer contains at least one application that can display corresponding icon controls in the display, such as: live TV application icon control, video on demand application icon control, media center application Program icon controls, application center icon controls, game application icon controls, etc.

In some embodiments, the live TV application may provide live TV from different sources. For example, a live TV application may provide a TV signal using input from cable, over-the-air, satellite services, or other types of live TV services. And, the live TV application may display the video of the live TV signal on the display device 200 .

In some embodiments, a video-on-demand application may provide video from various storage sources. Unlike live TV applications, video-on-demand provides a display of video from certain storage sources. For example, video-on-demand can come from the server side of cloud storage, from local hard disk storage containing existing video programs.

In some embodiments, the media center application may provide various multimedia content playback applications. For example, a media center may provide services other than live TV or video-on-demand, where users can access various images or audio through a media center application.

In some embodiments, the application center may provide storage of various applications. An application can be a game, an application, or some other application that is related to a computer system or other device but can be run on a Smart TV. The application center can obtain these applications from various sources, store them in local storage, and then run them on the display device 200 .

In some embodiments, controller 250 controls the operation of display device 200 and responds to users associated with display 275 by running various software control programs (eg, operating systems and/or various application programs) stored on memory 260 operate. For example, the control presents a user interface on the display including several UI objects; in response to a received user command for the UI object on the user interface, the controller 250 may perform operations related to the object selected by the user command.

The software program stored on the memory 260 may include an image recognition program, and the controller 250 recognizes objects in a specific image by running the image recognition program. The objects in the image include but are not limited to characters (such as stars), objects, and the like. Exemplarily, the user opens the image recognition program, uploads the image to be recognized to the server through the image recognition program, and finally receives the recognition result returned by the server. Another example, when the user uses the video playback software to watch the video program, by operating the buttons provided in the playback interface, input an instruction for identifying the current playback screen, and the device responds to the instruction and performs a screenshot operation to capture the currently playing image. Played video images, and then identify the captured video images.

In an implementation scenario of recognizing a person in an image, the person information is usually matched by recognizing a person's face in the image. However, affected by the size of the face area and the size of the image resolution, if the size of the face area in the image is small or the image resolution is low, it is easy to fail to identify or to identify errors due to insufficient feature extraction points.

In order to solve the above problems, the present application provides an image recognition method. After obtaining the image to be processed, the method firstly judges whether the image to be processed satisfies the preset recognition conditions. After the to-be-processed image is processed into an image that satisfies the preset recognition conditions, the recognition is performed, thereby improving the accuracy of the person recognition in the image.

The image recognition methods provided in the embodiments of the present application include, but are not limited to, applications on display devices as shown in Figures 1-5, and can also be applied on other handheld devices that can provide voice and data connectivity and have wireless connectivity. , or other processing devices that can be connected to a wireless modem, such as mobile telephones (or "cellular" telephones) and computers with mobile terminals, and can also be portable, pocket-sized, hand-held, computer-built-in, or vehicle-mounted mobile devices , they exchange data with the radio access network.

When the method is applied to the display device, the execution body of the method may be the controller 250 . Therefore, the present application also provides a display device. Based on the display devices shown in FIGS. 1-5 , the controller 250 of the display device is configured to execute part or all of the steps of the image recognition method provided by the present application. . As shown in Figure 6, the method may include:

Step 601, acquiring an image to be processed.

The image to be processed can be an image pre-saved locally on the display device, or an image stored in an external device connected to the display device, such as a USB device, or it can be generated in response to identification instructions input by the user in some operation scenarios. The screenshot of the current playback screen of the display device. For example, in a scenario where a user uses video playback software to watch a video program, the display device performs a screenshot operation on the played video screen by pressing a key or inputting a recognition command to obtain an image to be processed.

Step 602, when the to-be-processed image and the first target image satisfy a preset recognition condition, acquire a recognition result based on the first target image.

In this embodiment of the present application, the first target image is an original image corresponding to the target area in the image to be processed. In a face recognition scene, the target area may be a face area. After detecting the target area in the image to be processed, the first target image can be obtained by trimming the image to be processed.

Step 603: When the to-be-processed image and the first target image do not meet the preset recognition conditions, obtain a recognition result based on the second target image.

In the embodiment of the present application, the second target image is an image obtained according to the first target image and satisfying the preset identification condition, and further, the second target image also corresponds to the target area in the image to be processed.

In some embodiments, if the resolution of the image to be processed is lower than the preset resolution or the size of the first target image is smaller than the preset size, the image to be processed and the first target image do not meet the preset recognition conditions; If the resolution of the image is not lower than the preset resolution and the size of the first target image is not smaller than the preset size, the to-be-processed image and the first target image satisfy the preset recognition conditions.

Exemplarily, first determine whether the resolution of the image to be processed is lower than the preset resolution, and if the resolution of the image to be processed is lower than the preset resolution, it is determined that the image to be processed and the first target image do not meet the preset recognition conditions , if the resolution of the image to be processed is not lower than the preset resolution, then determine whether the size of the first target image is lower than the preset size, and if the size of the first target image is lower than the preset size, determine whether the image to be processed and The first target image does not meet the preset identification condition, and if the size of the first target image is not less than the preset size, it is determined that the to-be-processed image and the first target image meet the preset identification condition.

Exemplarily, the preset resolution may be 480P, and the preset size may be 30 pixels×30 pixels.

In some embodiments, an image super-resolution (Super Resolution, SR) technology is used to process the first target image, and a second target image with a resolution not lower than a preset resolution can be obtained. It should be noted that the image super resolution (Super Resolution, SR) technology takes the blurred, noisy and spectral aliased low-resolution image (Low Resolution, LR) as input, and fuses a high-resolution image through signal processing technology. (High Resolution, HR) image technology. SR technology can overcome the resolution limitation of the image system, improve the quality of most images in image processing, and has the characteristics of low cost. If supplementary explanation is needed, the second target image can be obtained by using the image super-resolution algorithm and model in the prior art to process the first target image, including but not limited to reconstruction-based methods, learning-based methods, etc., such as Discriminative Generative Networks for super-resolution image generation.

FIG. 7 is a possible first target image and a second target image exemplarily shown in this application, wherein (a) shows the first target image, and (b) shows the first target image corresponding to (a). Two target images, it can be seen that the pixels of the first target image are extremely low, which does not meet the feature point extraction requirements required for face recognition, and the pixels of the second target image are higher, which meets the feature point extraction requirements required for face recognition. .

In some embodiments, after acquiring the to-be-processed image, the display device sends data corresponding to the to-be-processed image to the server, where the data corresponding to the to-be-processed image at least includes data for enabling the server to acquire the first target image and data for enabling the server to acquire the first target image. The server obtains the resolution data of the image to be processed. Among them, the data used to determine the resolution of the image to be processed may be the resolution parameter of the image to be processed, or an identifier indicating the sharpness mode configured when the display device plays the image to be processed, such as extreme speed mode, smooth mode , standard mode, high-definition mode, etc., so that the server determines the sharpness mode according to the identifier of the sharpness mode, and then determines the resolution of the image to be processed.

In these embodiments, after receiving the data corresponding to the to-be-processed image sent by the display device, the server detects the target area in the to-be-processed image, that is, the face area, and cuts the to-be-processed image according to the detected target area, Obtain the first target image. Then, the server determines whether the first target image satisfies the preset identification condition according to the resolution of the image to be processed and the size of the first target image. If the first target image satisfies the preset recognition conditions, directly perform feature extraction on the first target image, and match the extracted feature information with the preset feature information database to obtain the recognition result; if the first target image does not meet the preset recognition conditions Assuming the identification conditions, the image super-resolution (Super Resolution, SR) technology is used to process the first target image to obtain a second target image with a resolution not lower than the preset resolution, so as to perform feature extraction on the second target image , and match the extracted feature information with the preset feature information library to obtain the recognition result. Finally, the server returns the recognition result based on the first target image or the second target image to the display device.

In other embodiments, after acquiring the image to be processed, the display device controller detects the target area in the image to be processed by running an image recognition program to obtain the first target image; and determines the resolution of the image to be processed and the first target Whether the size of the image satisfies the preset identification conditions; and, if the resolution of the image to be processed and the size of the first target image meet the preset identification conditions, the display device controller will perform feature extraction on the first target image, and the extracted The feature information of the image is matched with the local preset feature information library to obtain the recognition result; if the image to be processed and the first target image do not meet the preset recognition conditions, the image super-resolution processing program is called to process the first target image, A second target image is obtained, feature extraction is performed on the second target image, and the extracted feature information is matched with a local preset feature information database to obtain a recognition result.

Exemplarily, the recognition result may be information matching the first target image or the second target image and other information in the preset feature information database, such as other pictures of the same person, person introduction, related web pages, and so on.

Step 604, displaying the recognition result in the user interface.

For example, in a scenario where a user uses video playback software to watch a video program, the recognition result is displayed in the playback interface.

According to the above embodiment, the present application further provides an image recognition method, which is applied to a server. As shown in FIG. 8 , the method may include the following steps:

Step 801: Receive data corresponding to the image to be processed sent by the display device.

Step 802: Determine the image to be processed and a first target image according to the data corresponding to the image to be processed, where the first target image is an original image corresponding to a target area in the image to be processed.

Step 803: Determine whether the to-be-processed image and the first target image satisfy a preset identification condition.

Wherein, if the resolution of the image to be processed is lower than the preset resolution or the size of the first target image is smaller than the preset size, the image to be processed and the first target image do not meet the preset recognition conditions; if If the resolution of the to-be-processed image is not lower than the preset resolution and the size of the first target image is not smaller than the preset size, the to-be-processed image and the first target image satisfy the preset recognition condition.

Step 804: When the to-be-processed image and the first target image satisfy a preset recognition condition, acquire a recognition result based on the first target image.

Step 805, when the to-be-processed image and the first target image do not meet the preset recognition conditions, obtain a recognition result based on the second target image, and the second target image is obtained according to the first target image and meets the requirements. images with the preset recognition conditions.

Step 806, returning the identification result to the display device.

It should be understood that a more specific implementation manner of the embodiment shown in FIG. 8 has been recorded in the relevant description of the embodiment shown in FIG. 6 , and thus will not be repeated here.

According to the image recognition method provided by the embodiment shown in FIG. 8 , the present application further provides a server, the server includes a processor and a memory, and the processor is configured to execute some or all of the steps in the embodiment shown in FIG. 8 .

In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium can store a program, and when the program is executed, it can include some or all of the steps in each embodiment of the image recognition method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, ROM for short) or a random access memory (English: random access memory, RAM for short).

Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products may be stored in a storage medium, such as ROM/RAM , magnetic disk, optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

It is sufficient to refer to each other for the same and similar parts among the various embodiments in this specification. In particular, for the method embodiments, since they are basically similar to the display device embodiments, the description is relatively simple, and for related parts, please refer to the descriptions in the method embodiments.

The embodiments of the present invention described above do not limit the protection scope of the present invention.

Claims (9)

1. A display device, comprising:

a display for displaying a user interface;

a controller configured to:

acquiring an image to be processed;

when the image to be processed and a first target image meet a preset identification condition, acquiring an identification result based on the first target image, wherein the first target image is an original image corresponding to a target area in the image to be processed;

when the image to be processed and the first target image do not meet a preset identification condition, acquiring an identification result based on a second target image, wherein the second target image is generated according to the first target image and meets the preset identification condition;

and displaying the recognition result in the user interface.

2. The display device of claim 1, wherein after the obtaining of the image to be processed, the controller is further configured to:

sending data corresponding to the image to be processed to a server, wherein the data corresponding to the image to be processed at least comprises data for enabling the server to obtain the first target image and the resolution of the image to be processed;

the acquiring of the recognition result based on the first target image comprises: receiving an identification result based on the first target image returned by the server after judging that the image to be processed and the first target image meet a preset identification condition;

the acquiring of the recognition result based on the second target image includes: and receiving a recognition result based on the second target image returned by the server after judging that the image to be processed and the first target image do not meet the preset recognition condition.

3. The display device of claim 1, wherein after the obtaining of the image to be processed, the controller is further configured to:

detecting a target area in the image to be processed to obtain a first target image;

judging whether the image to be processed and the first target image meet a preset identification condition or not;

and when the image to be processed and the first target image do not meet the preset identification condition, processing the first target image by using an image super-resolution processing technology to generate a second target image.

4. The display device according to claim 1, wherein: if the resolution of the image to be processed is lower than the preset resolution or the size of the first target image is smaller than the preset size, the image to be processed and the first target image do not meet the preset identification condition;

and if the resolution ratio of the image to be processed is not lower than the preset resolution ratio and the size of the first target image is not smaller than the preset size, the image to be processed and the first target image meet the preset identification condition.

5. The display device according to claim 1, wherein the target area is an area containing a human face target, and the image to be processed contains one or more target areas.

6. The display device according to claim 1, wherein the recognition result is a result of matching the first target image or the second target image with a preset feature information base.

7. An image recognition method is applied to a display device, and the method comprises the following steps:

acquiring an image to be processed;

when the image to be processed and a first target image meet a preset identification condition, acquiring an identification result based on the first target image, wherein the first target image is an original image corresponding to a target area in the image to be processed;

when the image to be processed and the first target image do not meet a preset identification condition, acquiring an identification result based on a second target image, wherein the second target image is generated according to the first target image and meets the preset identification condition;

and displaying the identification result.

8. A server comprising a processor and a memory, wherein the processor is configured to:

receiving data corresponding to an image to be processed sent by display equipment;

determining the image to be processed and a first target image according to the data corresponding to the image to be processed, wherein the first target image is an original image corresponding to a target area in the image to be processed;

judging whether the image to be processed and the first target image meet a preset identification condition or not;

when the image to be processed and the first target image meet a preset identification condition, acquiring an identification result based on the first target image;

when the image to be processed and the first target image do not meet a preset identification condition, acquiring an identification result based on a second target image, wherein the second target image is generated according to the first target image and meets the preset identification condition;

and returning the identification result to the display equipment.

9. An image recognition method is applied to a server, and the method comprises the following steps:

receiving data corresponding to an image to be processed sent by display equipment;

determining the image to be processed and a first target image according to the data corresponding to the image to be processed, wherein the first target image is an original image corresponding to a target area in the image to be processed;

judging whether the image to be processed and the first target image meet a preset identification condition or not;

when the image to be processed and the first target image meet a preset identification condition, acquiring an identification result based on the first target image;

when the image to be processed and the first target image do not meet a preset identification condition, acquiring an identification result based on a second target image, wherein the second target image is generated according to the first target image and meets the preset identification condition;

and returning the identification result to the display equipment.

Images