CN115206308A - Man-machine interaction method and electronic equipment - Google Patents

Abstract

The present application provides a method and electronic device for human-computer interaction. The method can be applied to electronic devices with voice recognition and voice interaction functions such as mobile phones and tablet computers. If the voice command issued by the user or the answer to the electronic device includes Wake-up word, this method can accurately determine the position of the wake-up word in the voice command, avoid the wake-up word in the voice command from interrupting the current interaction process, thereby avoiding interrupting the currently executing task, and ensuring the coherence of the man-machine dialogue In addition, for devices such as robots with sound source localization function and/or image acquisition function, this method can determine whether to deflect according to the sound source direction of the voice command, and estimate the user's interaction willingness according to the collected images, etc., and then More accurate voice interaction with users, improving user experience.

Description

A human-computer interaction method and electronic device

technical field

The present application relates to the field of electronic technology, and in particular, to a method and electronic device for human-computer interaction.

Background technique

With the development of technology, more and more electronic devices support "human-computer interaction", or "voice interaction", and human-computer interaction has gradually become a way for users to convey their intentions and control electronic devices. The user's voice command controls the electronic device, thereby liberating the user's hands and facilitating the user to control the electronic device.

Before the user interacts with the electronic device, the electronic device can generally be woken up through a "wake-up word". When the electronic device is woken up, it can provide the user with a successful wake-up response, start to collect the user's voice command and perform automatic speech recognition (ASR). During the voice recognition process after the electronic device is awakened, if the acquired voice command includes a wake-up word, the wake-up word may interrupt the current human-computer interaction process, and start to collect the user's voice command and perform voice recognition again. Interrupting the current human-computer interaction in this process may not be expected by the user, that is, the wake-up word directly interrupts the currently executing task, so that the electronic device restarts to collect the voice commands of the mobile phone user, which will lead to incoherent human-computer dialogue. It affects the user's use process and reduces the experience of human-computer interaction.

SUMMARY OF THE INVENTION

The present application provides a human-computer interaction method and electronic device. The electronic device may include mobile phones, robots, tablets, computers, and other devices with speech recognition functions. The method can provide users with a coherent immersive experience and improve the user's visual experience.

In a first aspect, a method for human-computer interaction is provided, the method includes: receiving a wake-up word issued by a user, and in response to the wake-up word, enabling a voice recognition function of an electronic device; acquiring a first voice command of the user, and detecting When the wake-up word is included in the first voice command, determine the first time period occupied by the wake-up word in the time period corresponding to the first voice command; The target voice command other than the wake-up word; in response to the target voice command, answer.

In a possible scenario, taking the user waking up the mobile phone through the wake-up word "Xiaoyi Xiaoyi" as an example, after the mobile phone is woken up, it enters the state of monitoring the voice command used for it, if the voice command issued by the user again includes the wake-up word " "Xiaoyi Xiaoyi", the wake-up word can interrupt the current human-computer interaction process and re-enter the next human-computer interaction process, which may not be what the user expects, that is, the wake-up word directly interrupts the currently executing task, making the phone It is necessary to restart the collection of the user's voice command, which will lead to incoherent human-computer dialogue, affect the user's use process, and reduce the human-computer interaction experience.

Through the above method, in the process of voice interaction between the user and the electronic device, after the user wakes up the electronic device through the wake-up word, if the voice command issued by the user again includes the wake-up word, the method can avoid the wake-up word in the voice command from interrupting the current Therefore, the process of collecting the user's voice command is restarted, so as to avoid directly interrupting the task being performed by the current electronic device, which ensures the continuity of the human-machine dialogue and improves the user experience.

It should be understood that the automatic speech recognition (ASR) module of the mobile phone is not always on and working. When the user issues a voice command, the ASR module of the mobile phone is turned off; or, when the mobile phone is answering the user. , the ASR module is closed to avoid collecting the phone's own voice and interfering with the collection and recognition of the user's voice commands. Through the wake-up word, the mobile phone can first detect whether the ASR module is turned on after being woken up. If the ASR is in a dormant or inactive state, the ASR module can be triggered to turn on, that is, the voice recognition function of the electronic device can be turned on.

Optionally, when the mobile phone acquires and recognizes the wake-up word "Xiaoyi Xiaoyi" for the first time, if it is determined that the mobile phone is currently in the state where the ASR module is turned on, the wake-up can be ignored and the current dialogue process can be continued.

With reference to the first aspect, in some implementations of the first aspect, the first period is an end period, a middle period, or a start period of the period corresponding to the first voice instruction.

After the mobile phone is woken up, it monitors the user's first voice command, and when it is detected that the first voice command includes the wake-up word "Xiaoyi Xiaoyi" again, it can be judged that the wake-up word "Xiaoyi Xiaoyi" is in the first voice command. A position in a voice command, the position may mainly include being located at the first position of the first voice command, the middle of the first voice command, and the end of the first voice command. Exemplarily, when the first voice command issued by the user includes a wake-up word, it may be "Imitate the call of a cow, Xiaoyi Xiaoyi" (the wake-up word is at the end of the first voice command), "Imitate the sound of an animal. , Xiaoyi Xiaoyi, imitating the call of a cow (the wake-up word is in the middle of the first voice command)" or "Xiaoyi Xiaoyi, imitating the call of a cow" (the wake-up word is at the top of the first voice command).

In combination with the first aspect and the above-mentioned implementation manner, in some implementation manners of the first aspect, when the first time period is the end period of the time period corresponding to the first voice instruction, the method further includes: detecting that in the first voice instruction The voice command closest to the wake-up word, the time interval to the wake-up word; when the time interval is greater than or equal to the first preset value, suspend the current dialogue flow and restart the electronic device in response to the wake-up word. The voice recognition function enables the electronic device to obtain the second voice command.

It should be understood that the first preset value can be used to determine whether the current user wishes to interrupt the dialog flow. Exemplarily, when the first voice instruction issued by the user is: "Imitate the call of a cow, Xiaoyi Xiaoyi", the wake-up word is located at the end of the voice instruction. The closest voice command to the wake-up word "Xiaoyi Xiaoyi" is "Imitate the call of a cow". The mobile phone can judge the user to issue the wake-up word according to the time interval between "Imitate the call of a cow" and "Xiaoyi Xiaoyi". The mother of "Xiaoyi Xiaoyi". When the time interval between the "sound" of "imitation of the cow's cry" and the first "small" of "Xiaoyi Xiaoyi" is less than the first preset value, it can be judged that the user may only put the wake-up word "Xiaoyi" As part of the mantra, Xiaoyi hopes to continue the current dialogue flow without switching to the next new dialogue flow.

Optionally, the mobile phone can record the time information of the wake-up word "Xiaoyi Xiaoyi" in the first voice instruction according to the first voice instruction. The embodiment of the present application does not limit the rules for recording and marking time information. Exemplarily, if the initial wake-up word wakes up the mobile phone as the starting time, the time period during which the wake-up word reappears in the first voice instruction is t ₁ -t ₂ If the initial wake-up word wakes up the mobile phone as the starting time, the wake-up word reappears in the first voice command in a period of time T ₁ -T ₂ , and the position of the wake-up word in the first voice command can be determined according to the time information.

In a second aspect, a method for human-computer interaction is provided, the method comprising: acquiring a first voice command of a user, detecting a sound source direction of the first voice command according to the first voice command; The first angle between the sound source direction and the first line of sight direction the electronic device is currently facing; when the first angle is greater than or equal to the first preset angle, determine the sound source direction of the first voice command and the second voice command The second angle of the sound source direction, the second voice command is the voice command issued by the user before the first voice command and is closest to the first voice command; when the second angle is less than or equal to the second preset When the angle is turned, the electronic device responds to the first voice command and responds.

In another possible scenario, some electronic devices may have the capability of sound source localization, or have the function of camera image acquisition, such as robots. When the robot is awakened by the wake-up word, it can determine the direction of the user according to the sound source localization function, and turn the camera with the image acquisition function to directly turn to the direction or position of the user according to the sound source localization. During this process, there may be a large judgment error in the direction of the user due to problems such as sound being reflected by the wall. When such a large error occurs, the device will not face the person after rotating.

Through the above method, the wake-up process of the robot is more in line with human expectations. When the angle θ between the sound source direction of the user's voice command and the current line of sight of the robot is greater than or equal to the first preset angle and the user's willingness to interact When it is strong, the robot can determine to automatically turn to the user; when the angle θ between the sound source direction of the user's voice command and the direction of sight the robot is currently facing is greater than or equal to the first preset angle and the user's willingness to interact is low, the robot It can also be turned back, and the interaction process between the user and the robot will not be interrupted in the process, bringing users a better human-computer interaction experience.

It should be understood that when the angle θ between the sound source direction of the first voice command and the line of sight direction is greater than or equal to the first preset angle, it can be considered that the user and the robot issuing the voice command are not in a face-to-face positional relationship, In other words, the user who issued the voice command is not within the range of the central area of the image captured by the robot, and the embodiment of the present application does not limit the range corresponding to the central area.

It should also be understood that the "previous voice command" here is the closest voice command before the first voice command. Optionally, the "previous voice command" may be the user's wake-up word command, for example: Xiaoyi Xiaoyi. Or the "previous voice command" is another voice command after the wake-up word, for example: please imitate the call of a cow. This embodiment of the present application does not limit this.

With reference to the second aspect, in some implementations of the second aspect, the method further includes: detecting a time interval between the first voice command and the second voice command; when the time interval is less than or equal to a second preset value , call the steering execution function, and turn the electronic device to face or infinitely approach the sound source direction of the first voice command.

Optionally, "the angle between the sound source direction of the first voice command and the sound source direction of the previous voice command is greater than or equal to the second preset angle" and "the time interval between the two voice commands is greater than or equal to the second The display conditions of "preset value" can satisfy any one, or both, and call the steering execution function to change the direction of the robot. This embodiment of the present application does not limit this.

In combination with the second aspect and the foregoing implementations, in some implementations of the second aspect, the method further includes: collecting a first image of the electronic device in the first line of sight direction; when the first image includes the user and When the third angle between the user's sight direction and the sound source direction of the first voice command is less than or equal to the third preset angle, the steering execution function is called, and the electronic device is rotated to face or infinitely close to the first voice command. sound source direction.

Optionally, the robot may also collect images through a camera, and detect the direction in which the user's eyes are looking in the collected images to estimate the user's willingness to interact.

In combination with the second aspect and the above implementations, in some implementations of the second aspect, the method further includes: collecting, by the electronic device, a second image facing or infinitely close to the sound source direction of the first voice instruction; When the user is not included in the second image or the fourth angle between the user's gaze direction and the current second gaze direction of the electronic device is greater than a fourth preset angle, the electronic device is rotated to return to the first gaze direction.

To sum up, in the process of voice interaction between the user and the electronic device, after the user wakes up the electronic device through the wake-up word, if the voice command issued by the user or the answer to the electronic device again includes the wake-up word, this method can avoid the voice command. The wake-up word in the device interrupts the current interaction process, so as to avoid directly interrupting the task being performed by the current electronic device, and restart the process of collecting the user's voice command, which ensures the continuity of the man-machine dialogue and improves the user experience.

In addition, for electronic devices such as robots that have the capability of sound source localization, the method provided by the embodiment of the present application can determine whether to deflect according to the direction of the sound source of the voice command, and estimate the user's willingness to interact according to the collected images, etc., and furthermore Precise voice interaction with users. Specifically, when the angle θ between the sound source direction of the user's voice command and the direction of sight that the robot is currently facing is greater than or equal to the first preset angle and the user's willingness to interact is strong, the robot can determine to automatically turn to the user; when the user When the angle θ between the sound source direction of the voice command and the line-of-sight direction currently facing the robot is greater than or equal to the first preset angle and the user's willingness to interact is low, the robot can also turn back, and it will not turn back during the process. Interrupt the interaction process between the user and the robot, and bring the user a better human-computer interaction experience.

In a third aspect, an electronic device is provided, comprising: one or more processors; one or more memories; a module installed with a plurality of application programs; the memory stores one or more programs, when the one or more When a program is executed by the processor, the electronic device is executed and the electronic device performs the following steps: receiving a wake-up word issued by the user, in response to the wake-up word, enabling the voice recognition function of the electronic device; acquiring the user's first voice instruction, when it is detected that the wake-up word is included in the first voice command, determine the first time period occupied by the wake-up word in the time period corresponding to the first voice command; remove the wake-up word in the first time period, and identify the first time period The target voice command except the wake-up word in the voice command; in response to the target voice command, answer.

With reference to the third aspect, in some implementations of the third aspect, the first period is an end period, a middle period, or a start period of the period corresponding to the first voice instruction.

In combination with the third aspect and the above implementation manner, in some implementation manners of the third aspect, when the first period is the end period of the period corresponding to the first voice command, the electronic device may further perform the following steps: detecting the first time period The time interval from the voice command closest to the wake-up word in a voice command to the wake-up word; when the time interval is greater than or equal to the first preset value, the current dialogue flow is suspended and restarted in response to the wake-up word The voice recognition function of the electronic device enables the electronic device to acquire the second voice command.

In a fourth aspect, an electronic device is provided, comprising: a camera; one or more processors; one or more memories; a module in which multiple application programs are installed; Or when multiple programs are executed by the processor, the electronic device is made to perform the following steps: acquiring the user's first voice command, detecting the sound source direction of the first voice command according to the first voice command; determining The first angle between the sound source direction of the first voice command and the first line of sight direction the electronic device is currently facing; when the first angle is greater than or equal to the first preset angle, determine the sound source of the first voice command The second angle between the direction and the sound source direction of the second voice command, the second voice command is the voice command issued by the user before the first voice command and is closest to the first voice command; when the second angle is less than When the angle is equal to or equal to the second preset angle, the electronic device responds to the first voice command and responds.

In conjunction with the fourth aspect, in some implementations of the fourth aspect, when the one or more programs are executed by the processor, causing the electronic device to execute causes the electronic device to perform the following steps: detecting the first voice instruction and The time interval of the second voice command; when the time interval is less than or equal to the second preset value, the steering execution function is called to rotate the electronic device to face or infinitely approach the sound source direction of the first voice command.

In combination with the fourth aspect and the above-mentioned implementation manners, in some implementation manners of the fourth aspect, when the one or more programs are executed by the processor, the electronic device is caused to execute so that the electronic device performs the following steps: collecting the electronic The first image of the device in the first sight direction; when the first image includes the user and the third angle between the user's sight direction and the sound source direction of the first voice command is less than or equal to a third preset When the angle is changed, the steering execution function is called, and the electronic device is turned to face or infinitely approach the sound source direction of the first voice command.

In combination with the fourth aspect and the above-mentioned implementation manners, in some implementation manners of the fourth aspect, when the one or more programs are executed by the processor, the electronic device is caused to execute, causing the electronic device to perform the following steps: collecting oriented or The second image is infinitely close to the sound source direction of the first voice instruction; when the user is not included in the second image or the fourth angle between the user's gaze direction and the current second gaze direction of the electronic device is greater than At the fourth preset angle, rotate the electronic device to return to the first line of sight.

In a fifth aspect, the present application provides an apparatus, the apparatus is included in an electronic device, and the apparatus has a function of implementing the behavior of the electronic device in the above-mentioned aspect and possible implementation manners of the above-mentioned aspect. The functions can be implemented by hardware, or by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a display module or unit, a detection module or unit, a processing module or unit, and the like.

In a sixth aspect, the present application provides an electronic device, comprising: a touch display screen, wherein the touch display screen includes a touch-sensitive surface and a display; one or more audio devices; a camera; one or more processors; a memory; an application program; and one or more computer programs. Wherein, one or more computer programs are stored in the memory, the one or more computer programs comprising instructions. When the instructions are executed by the electronic device, the electronic device is caused to perform the method of human-computer interaction in any of the possible implementations of any of the above aspects.

In a seventh aspect, the present application provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors for storing computer program code, the computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform A method for human-computer interaction in any possible implementation of any of the above aspects.

In an eighth aspect, the present application provides a computer-readable storage medium, including computer instructions, when the computer instructions are executed on an electronic device, the electronic device performs any of the possible human-computer interaction methods in any of the foregoing aspects.

In a ninth aspect, the present application provides a computer program product that, when the computer program product runs on an electronic device, enables the electronic device to perform any of the possible human-computer interaction methods in any of the foregoing aspects.

Description of drawings

FIG. 1 is a schematic structural diagram of an example of an electronic device provided by an embodiment of the present application.

FIG. 2 is a block diagram of a software structure of an electronic device according to an embodiment of the present application.

FIG. 3 is a schematic diagram of a graphical user interface of an example of a human-computer interaction process.

FIG. 4 is a schematic flowchart of an example of a method for human-computer interaction provided by an embodiment of the present application.

FIG. 5 is a schematic diagram of an example of a human-computer interaction scenario provided by an embodiment of the present application.

FIG. 6 is a schematic flowchart of an example of a method for human-computer interaction provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, unless otherwise stated, “/” means or means, for example, A/B can mean A or B; “and/or” in this document is only a description of the associated object The association relationship of , indicates that there can be three kinds of relationships, for example, A and/or B, can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may expressly or implicitly include one or more of that feature.

The human-computer interaction method provided by the embodiments of the present application can be applied to mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR) devices, laptop computers, super mobile devices On electronic devices such as a personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), the embodiments of the present application do not impose any restrictions on the specific type of the electronic device.

Exemplarily, FIG. 1 is a schematic structural diagram of an example of an electronic device provided by an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and user A subscriber identification module (SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor ( image signal processor, ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and/or neural-network processing unit (NPU), etc. . Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver (universal asynchronous receiver) interface /transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and/or Universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate with each other through the I2C bus interface, so as to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may contain multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through a Bluetooth headset.

The PCM interface can also be used for audio communications, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 110 with the wireless communication module 160 . For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . The MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (displayserial interface, DSI), and the like. In some embodiments, the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 charges the battery 142 , it can also supply power to the electronic device through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellite systems applied on the electronic device 100 . (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , and/or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou satellite navigation system (BDS), a quasi-zenith satellite system (quasi- zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). , AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), MiniLED, MicroLED, Micro-OLED, quantum dot light emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.

The electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .

The camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example, moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the voice can be answered by placing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The earphone jack 170D is used to connect wired earphones. The earphone port 170D may be the USB port 130 or a 3.5mm open mobile terminal platform (OMTP) standard port, a cellular telecommunications industry association of the USA (CTIA) standard port.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. The capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 . In some embodiments, the angular velocity of electronic device 100 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse motion to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenarios.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

Distance sensor 180F for measuring distance. The electronic device 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light to the outside through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, and realize the function of heart rate detection.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 employs an eSIM, ie: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

系统为例，示例性说明电子设备100的软件结构。The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application use a layered architecture

Taking the system as an example, the software structure of the electronic device 100 is exemplarily described.

系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(

runtime)和系统库，以及内核层。应用程序层可以包括一系列应用程序包。FIG. 2 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the

The system is divided into four layers, from top to bottom are the application layer, the application framework layer, the Android runtime (

runtime) and system libraries, as well as the kernel layer. The application layer can include a series of application packages.

As shown in Figure 2, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

A window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether the screen has a status bar, or participate in performing operations such as locking the screen and taking screenshots.

Content providers are used to store and retrieve data and make these data accessible to applications. The stored data may include video data, image data, audio data, etc., and may also include dialed and answered call record data, user browsing history, bookmarks, and other data, which will not be repeated here.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of the call state (including the connection and hang-up of the phone, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files, and so on.

The notification manager enables applications to display notification information in the status bar of the screen, which can be used to convey messages to the user, and the notification information can disappear automatically after a short stay in the status bar, without the need for user interaction such as closing operations. For example, the notification manager can notify the user of messages such as download completion. The notification manager can also be a notification that appears in the status bar at the top of the system in the form of a graph or scroll bar text, such as a notification of an application running in the background; or, the notification manager can also be a notification that appears on the screen in the form of a dialog window, For example, text information is prompted in the status bar; alternatively, the notification manager can also control the electronic device to emit a prompt sound, the electronic device to vibrate, the electronic device's indicator light to flash, etc., which will not be repeated here.

runtime包括核心库和虚拟机。

runtime负责安卓系统的调度和管理。

The runtime includes core libraries and virtual machines.

The runtime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: a surface manager (surface manager), a media library (media libraries), a three-dimensional (3D) graphics processing library (eg: OpenGL ES), a two-dimensional (two dimensional, 2D) graphics engine, and the like.

Surface Manager is used to manage the display subsystem of an electronic device and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

A 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

For ease of understanding, the following embodiments of the present application will take the electronic device having the structure shown in FIG. 1 and FIG. 2 as an example, and combine the drawings and application scenarios to specifically describe the human-computer interaction method provided by the embodiments of the present application.

First, before introducing the human-computer interaction method provided by the embodiments of the present application, several possible application scenarios are listed first.

In a possible scenario, the human-computer interaction method provided by the embodiments of the present application may be applied to a scenario including a separate electronic device. Exemplarily, the electronic device may be different electronic devices, such as a mobile phone, a tablet, and a smart screen, as described above in conjunction with the structure shown in FIG. 1 , which is not limited in this embodiment of the present application. The following will take a mobile phone as an example to describe in detail the method for displaying a prompt of a human-computer interaction instruction provided by the present application.

FIG. 3 is a schematic diagram of a graphical user interface (GUI) of an example of a human-computer interaction process, wherein (a) in FIG. 3 shows that in the unlocking mode of the mobile phone, the screen display system of the mobile phone displays the current output The interface content 301 is the main interface of the mobile phone. The interface content 301 displays a variety of applications (application, App), such as mail, calculator, settings, and music. It should be understood that the interface content 301 may also include other more application programs, which is not limited in this application.

In a possible implementation manner, during the use of the voice assistant, the user can enable the smart voice function of the mobile phone through a setting application. Exemplarily, as shown in (a) of FIG. 3 , the user can click on the icon of the setting application, and in response to the user's click operation, the mobile phone displays the main interface 302 of the setting application as shown in (b) of FIG. 3 . . The main interface 302 of the setting application may include multiple menus, such as menus of WLAN, Bluetooth, desktop and wallpaper, display and brightness, sound, and smart assistant. The user can click on the smart assistant menu on the interface 302, and in response to the user's click operation, the mobile phone displays the smart assistant interface 303 as shown in (c) in FIG. Screen recognition, situational intelligence, smart search and other options, in addition, the smart assistant interface 303 also displays the wake-up word "Xiaoyi Xiaoyi", the wake-up word "Xiaoyi Xiaoyi" can be used by the user to wake up the mobile phone, so that the mobile phone Enter the state of monitoring and collecting the user's voice command.

As shown in (c) in Figure 3, the user clicks the smart voice option on the smart assistant interface 303, and in response to the user's click operation, the mobile phone displays the smart voice interface 304 as shown in (d) in Figure 3 . The intelligent voice interface 304 may include a voice wake-up switch, a power key change switch, an artificial intelligence (AI) letter switch, a driving scene switch, and the like. In the embodiment of the present application, the user can click the voice wake-up switch to enable the voice interaction function of the mobile phone. In other words, after the voice wake-up switch is turned on, the mobile phone can be woken up by the wake-up word "Xiaoyi Xiaoyi", start to collect the user's voice command and enter the voice recognition stage.

When the user turns on the voice interaction function of the mobile phone, if the user sends the wake-up word "Xiaoyi Xiaoyi", a floating window can be displayed on the screen of the mobile phone to remind the user to start collecting the user's voice command and start the voice interaction process. Exemplarily, as shown in (e) in Figure 3, after the user sends out the wake-up word "Xiaoyi Xiaoyi", the mobile phone is woken up by the wake-up word, and a floating window 10 can be displayed on the screen, and the floating window 10 It includes the conversation content between the user and the mobile phone (for example: Hi, I'm listening...), and the monitoring icon 10-1 when the mobile phone is monitoring the user's voice command. The monitoring icon 10-1 can be dynamically flashed to indicate that it is currently The voice command of the user is monitored, which is not limited in this embodiment of the present application.

As shown in (e) of Figure 3, the mobile phone monitors the user's voice command: imitating the sound of a cow, Xiaoyi Xiaoyi. The mobile phone can recognize the content of the voice command, and display the recognized voice command in the floating window 10. In the existing solution, the mobile phone can respond according to the voice command, such as imitating the sound of a cow.

However, when the voice command includes the wake-up word "Xiaoyi Xiaoyi" again, the mobile phone may be interrupted by the wake-up word "Xiaoyi Xiaoyi" in the voice command, thereby interrupting the current human-computer interaction process , and restart to collect the user's voice command and perform voice recognition. As shown in (f) in Figure 3, after the mobile phone recognizes that the user's voice command includes the wake-up word "Xiaoyi Xiaoyi", the mobile phone will respond to the voice command including the wake-up word "Xiaoyi Xiaoyi". Re-enter the next human-computer interaction process, and respond in the floating window 10: Hey, I'm listening..., and the dynamic blinking monitor icon 10-1 that displays the phone's monitoring of the user's voice command to indicate that the current phone starts monitoring again The user's voice command.

In the above scenario, if the voice command issued by the user includes the wake-up word "Xiaoyi Xiaoyi", the wake-up word can interrupt the current human-computer interaction process and re-enter the next human-computer interaction process, which may not be expected by the user. That is, the wake-up word directly interrupts the currently executing task, so that the mobile phone needs to start collecting the user's voice commands again, which will lead to incoherent human-computer dialogue, affect the user's use process, and reduce the human-computer interaction experience. .

The embodiment of the present application provides a human-computer interaction method, which can avoid interruption of the human-computer interaction process by the wake-up word in the voice command, so as to bring a better human-computer interaction experience to the user.

FIG. 4 is a schematic flowchart of an example of a method for human-computer interaction provided by an embodiment of the present application. It should be understood that the method 400 can be applied to electronic devices such as mobile phones, PCs, vehicle-mounted devices, etc. having the structures shown in FIGS. 1 and 2 . . As shown in FIG. 4, method 400 includes:

401. Acquire a first voice instruction of a user, and detect that the first voice instruction includes a wake-up word.

Exemplarily, with reference to the scenario shown in (e) of FIG. 3 , if the user currently expects a conversation with the mobile phone to be the following:

User: Xiaoyi Xiaoyi.

Phone: Hi, I'm listening...

User: Imitate the sound of a cow, Xiaoyi Xiaoyi.

Cell Phone: Moo Moo...

When the mobile phone detects that the user speaks the wake-up word "Xiaoyi Xiaoyi" for the first time, the mobile phone wakes up, and the mobile phone enters the state of monitoring the user's voice command.

402. Determine whether the ASR module is in an on state.

It should be understood that the ASR module of the mobile phone is not always on and working. When the user issues a voice command, the mobile phone turns off the automatic speech recognition (ASR) function, that is, the ASR module is turned off; When answering the user, the ASR module is also closed to avoid collecting the phone's own voice and interfering with the collection and recognition of the user's voice command. Through step 402, the mobile phone first detects whether the ASR module is in an on state, and if the ASR is in a dormant or inactive state, the ASR module can be triggered to be turned on.

Optionally, when the mobile phone acquires and recognizes the wake-up word "Xiaoyi Xiaoyi" for the first time, if it is determined that the mobile phone is currently in the state where the ASR module is turned on, the wake-up can be ignored and the current dialogue process can be continued.

403. When the mobile phone determines that the ASR module is in an on state, determine the position of the wake-up word in the first voice instruction.

In a possible implementation, the mobile phone monitors the user's first voice command after being woken up, and when it is detected that the first voice command includes the wake-up word "Xiaoyi Xiaoyi" again, the wake-up word "Xiaoyi Xiaoyi" can be judged first. The position of "Xiaoyi Xiaoyi" in the first voice command, the position may mainly include being located at the first position of the first voice command, the middle of the first voice command, and the end of the first voice command. Exemplarily, when the first voice command issued by the user includes a wake-up word, it may be "Imitate the call of a cow, Xiaoyi Xiaoyi" (the wake-up word is at the end of the first voice command), "Imitate the sound of an animal. , Xiaoyi Xiaoyi, imitating the call of a cow (the wake-up word is in the middle of the first voice command)" or "Xiaoyi Xiaoyi, imitating the call of a cow" (the wake-up word is at the top of the first voice command).

404-1, when the position of the wake-up word in the first voice command is at the end, step 405 is executed to determine whether the duration of the voice command whose wake-up word distance is closest is less than the first preset value.

406 , when the time interval between the wake-up word and the voice command closest to the wake-up word is less than the first preset value, record time information corresponding to the wake-up word.

It should be understood that the first preset value can be used to determine whether the current user wishes to interrupt the dialog flow. Exemplarily, when the first voice instruction issued by the user is: "Imitate the call of a cow, Xiaoyi Xiaoyi", the wake-up word is located at the end of the voice instruction. According to step 406, the closest voice command to the wake-up word "Xiaoyi Xiaoyi" is "Imitate the sound of a cow", and the mobile phone can judge the user according to the time interval between "Imitate the sound of a cow" and "Xiaoyi Xiaoyi" The mother who issued the wake-up word "Xiaoyi Xiaoyi". When the time interval between the "sound" of "imitation of the cow's cry" and the first "small" of "Xiaoyi Xiaoyi" is less than the first preset value, it can be judged that the user may only put the wake-up word "Xiaoyi" As part of the mantra, Xiaoyi hopes to continue the current dialogue flow without switching to the next new dialogue flow.

Optionally, the mobile phone can record the time information of the wake-up word "Xiaoyi Xiaoyi" in the first voice instruction according to the first voice instruction. The embodiments of the present application do not limit the rules for recording and marking time information. Exemplarily, if the initial wake-up word wakes up the mobile phone as the starting time, the time period during which the wake-up word reappears in the first voice command is t ₁ -t ₂ If the initial wake-up word wakes up the mobile phone as the starting time, the wake-up word reappears in the first voice command in a period of time T ₁ -T ₂ , and the position of the wake-up word in the first voice command can be determined according to the time information.

407. According to the time information corresponding to the wake-up word, ignore the wake-up word, and recognize the first voice instruction.

408, normal response. Optionally, the normal response here may include feedback from the mobile phone according to the user's question, and continuous dialogue with the user, or may also include voice responses such as "um" and "okay", which are not limited in this embodiment of the present application.

409. When the duration of the voice command with the closest wake-up word distance is greater than or equal to the first preset value, suspend the current dialog box and open a next new dialog box.

410, turn on ASR, identify the user's second voice command in the new dialog box, and respond normally according to the user's second voice command again, or return to step 401, check again whether the wake-up word is included in the second voice command, repeat For the sake of simplicity, the above process is not repeated here.

Exemplarily, when the first voice instruction issued by the user is: "Imitate the sound of a cow, Xiaoyi Xiaoyi", the wake-up word is located at the end of the first voice instruction. When the time interval between the "sound" of "Imitating the Cow's Cow" and the first "small" of "Xiaoyi Xiaoyi" is greater than or equal to the first preset value, it can be judged that the user may wish to interrupt the current conversation flow to enter the next new dialogue flow. In other words, the mobile phone can use the wake-up word "Xiaoyi Xiaoyi" included in the first voice command as the wake-up word for the next dialogue process, and the mobile phone is awakened again, interrupting the previous dialogue process of "imitation of a cow's cry" . Optionally, at this time, the mobile phone may reply "Hi, I'm listening...", which is not limited in this embodiment of the present application.

Optionally, the first preset value may be 1 second, 2 seconds, etc., which is not limited in this embodiment of the present application.

411 , when the mobile phone determines that the ASR module is not turned on, turn on the ASR module and start the monitoring function. And after the ASR monitoring function is enabled, the process of obtaining the user's voice command in step 401 is continued, which will not be repeated here.

For step 403, when it is determined that the wake-up word is at the first position in the first voice command or in the middle of the first voice command, that is, 404-3, when the wake-up word is at the first position in the first voice command, or 404- 2. When the wake-up word is in the middle of the first voice command, perform steps 406-408, record the time information corresponding to the wake-up word, ignore the wake-up word according to the time information corresponding to the wake-up word, and perform the first voice command. Identify and perform a normal response. For simplicity, details are not repeated here.

In a possible scenario, if speech recognition is turned on again shortly after speech recognition, the mobile phone can determine whether the user continues to speak. If the user does not continue to speak, the mobile phone can continue to talk to the user using the previous speech recognition results. .

Through the above method, in the process of voice interaction between the user and the electronic device, after the user wakes up the electronic device through the wake-up word, if the voice command issued by the user again includes the wake-up word, the method can avoid the wake-up word in the voice command from interrupting the current Therefore, the process of collecting the user's voice command is restarted, so as to avoid directly interrupting the task being performed by the current electronic device, which ensures the continuity of the human-machine dialogue and improves the user experience.

In addition, in another possible scenario, some electronic devices may have the ability of sound source localization, or have the function of image acquisition of cameras, such as robots. When the robot is awakened by the wake-up word, it can determine the direction of the user according to the sound source localization function, and turn the camera with the image acquisition function to directly turn to the direction or position of the user according to the sound source localization. During this process, there may be a large judgment error in the direction of the user due to problems such as sound being reflected by the wall. When such a large error occurs, the device will not face the person after rotating.

It should be understood that the robot may have some or all of the structures shown in FIG. 1 , or have the software architecture shown in FIG. 2 , which is not limited in this embodiment of the present application.

Exemplarily, FIG. 5 is a schematic diagram of an example of human-computer interaction provided by an embodiment of the present application. As shown in Figure 5, assuming that the robot has the capability of sound source localization and image acquisition, the robot can determine the direction of the sound source according to the user's voice command, and can determine its own gaze estimation direction according to the image collected by the camera. Among them, the angle between the line of sight direction and the sound source direction is denoted as θ.

Optionally, in the process of determining its own gaze estimation direction according to the image collected by the camera, the robot can establish a camera coordinate system, and based on the public parameters of the camera, pass the gaze target and the coordinates of the user's eye position through six three-dimensional keys. For transforming points and other algorithms into camera coordinates, specific reference may be made to the calculation process in the prior art, which will not be repeated here.

The embodiments of the present application also provide a human-computer interaction method for electronic devices such as robots capable of sound source localization, which can prevent the human-computer interaction process from being interrupted by the wake-up word in the voice command, so as to provide users with better human-computer interaction experience.

FIG. 6 is a schematic flowchart of an example of a method for human-computer interaction provided by an embodiment of the present application. It should be understood that the method 600 may be applied to an electronic device having the capability of sound source localization, such as a robot. As shown in FIG. 6, method 600 includes:

601. The robot obtains the first voice instruction of the user.

602. The robot detects the sound source direction of the first voice command according to the first voice command.

603: The robot determines whether the included angle θ between the sound source direction of the first voice command and the current line of sight of the robot is greater than or equal to a first preset angle.

604. When the angle θ between the sound source direction of the first voice command and the line of sight direction is greater than or equal to the first preset angle, the robot determines whether the user's willingness to interact is less than the preset value.

It should be understood that when the angle θ between the sound source direction of the first voice command and the line of sight direction is greater than or equal to the first preset angle, it can be considered that the user and the robot issuing the voice command are not in a face-to-face positional relationship, In other words, the user who issued the voice command is not within the range of the central area of the image captured by the robot, and the embodiment of the present application does not limit the range corresponding to the central area.

Optionally, in step 604, the robot may collect an image through a camera, and detect the direction in which the user's eyes are looking in the collected image to estimate the user's willingness to interact. For example, Table 1 lists an example of a range of possible user interaction intentions.

Table 1

The angle range between the user's gaze direction and the robot's gaze direction Estimated range of willingness to interact 0°-30° 0.8-1.0 30°-60° 0.5-0.8 60°-90° 0.1-0.5

As shown in Table 1, when the estimated range of interaction willingness is determined to be 0.8-1.0 according to the angle between the user's gaze direction and the robot's gaze direction, the robot can judge that the user's current willingness to interact is strong; When the estimated range of interaction willingness is determined by the angle range of the gaze direction, the robot can judge that the user's current interaction willingness is normal; when the estimated range of interaction willingness is determined according to the angle range between the user's gaze direction and the robot's gaze direction, the estimated range of interaction willingness is 0.1 When -0.5, the robot can determine that the user's current willingness to interact is low, which is not limited in this embodiment of the present application.

Optionally, the preset value may be set to 0.5. When the estimated user's current interaction intention is greater than or equal to the preset value, the following step 605 is continued.

605. The robot determines whether the angle between the sound source direction of the first voice command and the sound source direction of the previous voice command is less than a second preset angle, and whether the time interval between two voice commands is less than a second preset value .

606, when the angle between the sound source direction of the first voice command and the sound source direction of the previous voice command is less than the second preset angle, and the time interval between the two voice commands is less than the second preset value, the robot performs Respond normally.

It should be understood that the "previous voice command" here is the closest voice command before the first voice command. Optionally, the "previous voice command" may be the user's wake-up word command, for example: Xiaoyi Xiaoyi. Or the "previous voice command" is another voice command after the wake-up word, for example: please imitate the call of a cow. This embodiment of the present application does not limit this.

It should also be understood that the normal response here can be understood as the robot recognizing the first voice command of the user and making corresponding feedback according to the first voice command, which will not be repeated here.

607, when the angle between the sound source direction of the first voice command and the sound source direction of the previous voice command is greater than or equal to the second preset angle, and the time interval between the two voice commands is greater than or equal to the second preset angle When the value is set, the robot calls the steering execution function to convert the robot direction.

Optionally, "the angle between the sound source direction of the first voice command and the sound source direction of the previous voice command is greater than or equal to the second preset angle" and "the time interval between the two voice commands is greater than or equal to the second The display conditions of "preset value" can satisfy any one, or both, and call the steering execution function to change the direction of the robot. This embodiment of the present application does not limit this.

608. In response to the steering execution function, after the robot changes direction, determine the user interaction intention after the direction is changed. Optionally, the process of determining the user's willingness to interact can be determined by collecting an image and judging the direction of the user's gaze in the image. For details, please refer to the relevant introduction in the foregoing step 604, which will not be repeated here.

In a possible implementation manner, in step 608, after the robot changes the direction in response to the steering execution function, it is determined that the user's willingness to interact after the direction is changed is relatively low, and the robot can turn back to the line of sight direction, and step 606 is performed at the same time. The voice command will give corresponding feedback and respond normally.

In another possible scenario, if the user's first voice instruction may contain a wake-up word, the method shown in FIG. 4 can be combined to collect images and estimate the user's willingness to interact according to the direction of the user's gaze in the image. When the robot does not detect a person in the image, it can be determined that the user's willingness to interact is very low, or the current human-computer interaction process is interrupted. In other words, this scenario can be considered as the robot being awakened by mistake.

In another possible scenario, if the first voice command is the wake-up word, and the angle θ between the sound source direction of the wake-up word and the current line of sight of the robot is greater than or equal to the first preset angle, the robot can Whether there is a user in the collected images and estimating whether the user's willingness to interact is strong to determine whether it is necessary to respond to this wake-up.

Exemplarily, if the angle θ between the direction of the sound source of the wake-up word and the direction of sight that the robot is currently facing is greater than or equal to the first preset angle, and the current user's willingness to interact is strong, it can be set that the robot needs to The wake-up word in the same sound source direction can wake up the robot, that is, the robot will respond to the user's wake-up word.

Or, if the angle θ between the sound source direction of the wake-up word and the current line-of-sight direction of the robot is greater than or equal to the first preset angle, after the robot turns to the sound source direction of the wake-up word, it does not detect the user, and can repeat Turn back to the pre-wake angle and continue the voice interaction with the person before the wake-up.

Through the above method, the wake-up process of the robot is more in line with human expectations. When the angle θ between the sound source direction of the user's voice command and the current line of sight of the robot is greater than or equal to the first preset angle and the user's willingness to interact When it is strong, the robot can determine to automatically turn to the user; when the angle θ between the sound source direction of the user's voice command and the direction of sight the robot is currently facing is greater than or equal to the first preset angle and the user's willingness to interact is low, the robot It can also be turned back, and the interaction process between the user and the robot will not be interrupted in the process, bringing users a better human-computer interaction experience.

To sum up, in the process of voice interaction between the user and the electronic device, after the user wakes up the electronic device through the wake-up word, if the voice command issued by the user or the answer to the electronic device again includes the wake-up word, this method can avoid the voice command. The wake-up word in the device interrupts the current interaction process, so as to avoid directly interrupting the task being performed by the current electronic device, and restart the process of collecting the user's voice command, which ensures the continuity of the man-machine dialogue and improves the user experience.

In addition, for electronic devices such as robots that have the capability of sound source localization, the method provided by the embodiment of the present application can determine whether to deflect according to the direction of the sound source of the voice command, and estimate the user's willingness to interact according to the collected images, etc., and furthermore Precise voice interaction with users. Specifically, when the angle θ between the sound source direction of the user's voice command and the direction of sight that the robot is currently facing is greater than or equal to the first preset angle and the user's willingness to interact is strong, the robot can determine to automatically turn to the user; when the user When the angle θ between the sound source direction of the voice command and the line-of-sight direction currently facing the robot is greater than or equal to the first preset angle and the user's willingness to interact is low, the robot can also turn back, and it will not turn back during the process. Interrupt the interaction process between the user and the robot, and bring the user a better human-computer interaction experience.

It can be understood that, in order to realize the above-mentioned functions, the electronic device includes corresponding hardware and/or software modules for executing each function. The present application can be implemented in hardware or in the form of a combination of hardware and computer software in conjunction with the algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functionality for each particular application in conjunction with the embodiments, but such implementations should not be considered beyond the scope of this application.

In this embodiment, the electronic device can be divided into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that, the division of modules in this embodiment is schematic, and is only a logical function division, and there may be other division manners in actual implementation.

In the case where each functional module is divided according to each function, the electronic devices such as robots and mobile phones involved in the above embodiments may include: a collection unit, a detection unit, and a processing unit.

Wherein, the acquisition unit, the detection unit and the processing unit cooperate with each other, and can be used to support electronic devices such as robots and mobile phones to perform the above steps, etc., and/or be used for other processes of the technology described herein.

It should be noted that, all relevant contents of the steps involved in the above method embodiments can be cited in the functional description of the corresponding functional module, which will not be repeated here.

The electronic device provided in this embodiment is used to execute the above-mentioned video playback method, and thus can achieve the same effect as the above-mentioned implementation method.

Where an integrated unit is employed, the electronic device may include a processing module, a memory module and a communication module. The processing module may be used to control and manage the actions of the electronic device, for example, may be used to support the electronic device to perform the steps performed by the above acquisition unit, detection unit and processing unit. The storage module may be used to support the electronic device to execute stored program codes and data, and the like. The communication module can be used to support the communication between the electronic device and other devices.

The processing module may be a processor or a controller. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that implements computing functions, such as a combination comprising one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and the like. The storage module may be a memory. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, and a Wi-Fi chip.

In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a device having the structure shown in FIG. 1 .

This embodiment also provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on the electronic device, the electronic device executes the above-mentioned related method steps to realize the above-mentioned embodiments. methods of human-computer interaction.

This embodiment also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute the above-mentioned relevant steps, so as to realize the method for human-computer interaction in the above-mentioned embodiment.

In addition, the embodiments of the present application also provide an apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a connected processor and a memory; wherein, the memory is used for storing computer execution instructions, and when the apparatus is running, The processor can execute the computer-executed instructions stored in the memory, so that the chip executes the method for human-computer interaction in the foregoing method embodiments.

Wherein, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved, reference may be made to the above-provided method. The beneficial effects in the corresponding method will not be repeated here.

From the description of the above embodiments, those skilled in the art can understand that for the convenience and brevity of the description, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions can be allocated by different The function module is completed, that is, the internal structure of the device is divided into different function modules, so as to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or May be integrated into another device, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

Units described as separate components may or may not be physically separated, and components shown as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, which are stored in a storage medium , including several instructions to make a device (which may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a removable hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk and other media that can store program codes.

The above content is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Covered within the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method for human-computer interaction, wherein the method comprises: Receive a wake-up word issued by the user, and in response to the wake-up word, enable the voice recognition function of the electronic device; Acquire the first voice command of the user, and when detecting that the wake-up word is included in the first voice command, determine the first time period occupied by the wake-up word in the time period corresponding to the first voice command; removing the wake-up word in the first time period, and identifying the target voice command in the first voice command except the wake-up word; Responding to the target voice command. 2 . The method according to claim 1 , wherein the first period is an end period, a middle period or a start period of a period corresponding to the first voice instruction. 3 . 3. The method according to claim 1 or 2, wherein when the first period is the end period of the period corresponding to the first voice instruction, the method further comprises: Detecting the voice command closest to the wake-up word in the first voice command, and the time interval from the wake-up word; When the time interval is greater than or equal to the first preset value, the current dialogue flow is suspended and the voice recognition function of the electronic device is restarted in response to the wake-up word, so that the electronic device acquires the second voice command. 4. A method for human-computer interaction, wherein the method comprises: Obtaining the user's first voice command, and detecting the sound source direction of the first voice command according to the first voice command; determining a first angle between the sound source direction of the first voice instruction and the first line of sight direction currently facing the electronic device; When the first angle is greater than or equal to the first preset angle, a second angle between the sound source direction of the first voice command and the sound source direction of the second voice command is determined, and the second voice command is The voice command issued before the first voice command and closest to the first voice command; When the second angle is less than or equal to a second preset angle, the electronic device responds to the first voice command and responds. 5. The method according to claim 4, wherein the method further comprises: detecting the time interval between the first voice command and the second voice command; When the time interval is less than or equal to the second preset value, the steering execution function is called, and the electronic device is rotated to face or infinitely approach the sound source direction of the first voice instruction. 6. The method according to claim 4 or 5, wherein the method further comprises: collecting a first image of the electronic device in the first line of sight direction; When the first image includes the user and the third angle between the user's gaze direction and the sound source direction of the first voice command is less than or equal to a third preset angle, the steering execution function is called, Rotate the electronic device to face or infinitely approach the direction of the sound source of the first voice command. 7. The method according to claim 6, wherein the method further comprises: The electronic device collects a second image facing or infinitely close to the sound source direction of the first voice instruction; When the second image does not include the user or the fourth angle between the user's gaze direction and the current second gaze direction of the electronic device is greater than a fourth preset angle, rotating the electronic device restores to the first line of sight. 8. An electronic device, characterized in that, comprising: one or more processors; one or more memories; Modules with multiple applications installed; The memory stores one or more programs that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 7 . 9. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores computer instructions, which, when the computer instructions are executed on an electronic device, cause the electronic device to perform the execution of claims 1 to 7 The method of any of the above. 10. A computer program product, characterized in that, when the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 7.

Images