CN114093391B

CN114093391B - A method and device for filtering abnormal signals

Info

Publication number: CN114093391B
Application number: CN202010744779.3A
Authority: CN
Inventors: 任兵飞; 毛哲文; 肖艳光
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-07-29
Filing date: 2020-07-29
Publication date: 2025-02-25
Anticipated expiration: 2040-07-29
Also published as: CN114093391A

Abstract

The application provides a method and a device for filtering abnormal signals. The method comprises the steps that the electronic equipment detects the acquired sound signals according to a detection model trained in advance. If the electronic device detects the presence of an abnormal high frequency signal in the sound signal, a warning may be issued to the user or the abnormal high frequency signal in the sound signal may be filtered out. This has the advantage that the privacy of the user can be prevented from being compromised without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, and the risk of privacy information disclosure is reduced.

Description

Abnormal signal filtering method and device

Technical Field

The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for filtering an abnormal signal.

Background

With the rapid development of networks and informatization, information is being released through the internet, network trade is being conducted through the internet, and the like, and meanwhile, various confidential information, including national security information, private information (such as personal account numbers) and the like, needs to be transmitted through the network, so that in order to protect the security of various confidential information, the hidden communication technology can effectively prevent and control the interception and acquisition of the confidential information by a third party.

Hidden communication technology based on sound waves has been widely used. The method mainly comprises the working procedures that the synthesizing equipment modulates information to be transmitted into carrier signals with specific frequencies and synthesizes the carrier signals into normal sound signals, the sending end sends out the normal sound signals, and the receiving end receives and demodulates the information to be transmitted in the normal sound signals to complete hidden communication based on sound waves.

Hidden communication based on sound waves also carries the risk of personal information leakage to the user. An attacker modulates special information into a high-frequency carrier signal by a synthesizing device and synthesizes the high-frequency carrier signal into a normal sound signal, the high-frequency carrier signal being imperceptible to a user. A playback device (e.g., audio) plays back the synthesized sound signal. When the user approaches the playback device, the user's electronic device (e.g., a cell phone) may receive the synthesized sound signal. If an application program developed by an attacker is installed in an electronic device (such as a mobile phone) of a user, and the application program can detect a high-frequency carrier signal modulated by the attacker, the application program can collect user or device information, such as an identification of the electronic device (such as the mobile phone) or a current position of the electronic device (such as the mobile phone), and send the information to the attacker. At this time, the attacker obtains the information of the user without the knowledge of the user.

Disclosure of Invention

The application provides a filtering method of abnormal signals, which is characterized in that electronic equipment detects collected sound signals, an abnormal signal detection module detects abnormal high-frequency signals, namely when the high-frequency energy value of the sound signals is higher than a threshold value, the abnormal situation is informed to a user, the user can select to filter the abnormal high-frequency signals so as to intercept potential hidden communication threats, and meanwhile, the sound signals with the abnormal high-frequency signals filtered are returned to a first application. Thus, hidden communication can be identified and intercepted, and the safety of user privacy is improved.

In a first aspect, the present application provides a method for detecting and filtering an abnormally high frequency signal. The method comprises the steps that the electronic equipment receives a sound signal acquisition request, responds to the sound signal acquisition request, acquires a sound signal, and determines a high-frequency energy value of the sound signal according to the sound signal. If the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold value, the electronic equipment sends prompt information to the user, and the prompt information is used for prompting the user that an abnormal high-frequency signal exists in the sound signal.

The electronic device detects the acquired sound signal according to a detection model trained in advance. If the electronic device detects the presence of an abnormal high frequency signal in the sound signal, a warning may be issued to the user or the abnormal high frequency signal in the sound signal may be filtered out. This has the advantage that the privacy of the user can be prevented from being compromised without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, and the risk of privacy information disclosure is reduced.

With reference to the first aspect, in one possible implementation manner of the first aspect, after the electronic device sends the prompt information to the user, the electronic device further receives a filtering operation of the user, and in response to the filtering operation, the electronic device filters out an abnormal high-frequency signal existing in the sound signal. In this way, the user can sense that an abnormal high-frequency signal exists in the sound signal acquired by the electronic equipment, and when the user passes through the area next time, the abnormal high-frequency signal existing in the area can be blocked, or the user reports the situation that the abnormal high-frequency signal exists in the area to the server, and the server gathers the information reported by the user and takes measures to prevent the privacy information of the user from being leaked. In some embodiments, when the electronic device determines that the high frequency energy value of the sound signal is greater than the environmental sound energy threshold, the electronic device does not need to send a prompt message to the user, and the electronic device directly removes the abnormal high frequency signal in the sound signal. Thus, when the user makes a call with friends through the electronic equipment or the electronic equipment is displaying a navigation route, the electronic equipment does not send prompt information to the user, and the current operation of the user is not affected.

With reference to the first aspect, in a possible implementation manner of the first aspect, the electronic device further sends a sound signal from which the abnormal high frequency signal is filtered out to the receiving device. When a user calls a friend or plays a game with a friend group through the electronic equipment, the electronic equipment sends a sound signal for filtering abnormal high-frequency signals to the receiving equipment through the communication module, and the receiving equipment is equipment which is used by the friend of the user, so that the current operation of the user is not influenced.

With reference to the first aspect, in a possible implementation manner of the first aspect, the electronic device obtains first environmental information, where the first environmental information includes location information and/or environmental sound, and determines an environmental sound energy threshold according to the first environmental information. Here, the first environmental information is used for determining a current environment (for example, a mall), the electronic device determines an environmental sound energy threshold corresponding to the current environment according to the current environment, the environmental sound energy threshold is used for comparing with a high-frequency energy value of the sound signal, and if the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold corresponding to the current environment, the electronic device determines that an abnormal high-frequency signal exists in the sound signal.

With reference to the first aspect, in one possible implementation manner of the first aspect, the electronic device determines a high-frequency energy value of the sound signal according to the sound signal, and specifically includes the steps that firstly, the electronic device acquires environmental sound before acquiring the sound signal, secondly, the electronic device determines an environmental noise energy value according to the environmental sound, and finally, the electronic device determines the high-frequency energy value of the sound signal according to an actual high-frequency energy value of the sound signal and the environmental noise energy value. Here, the electronic device determines the high frequency energy value of the sound signal based on the actual high frequency energy value of the sound signal and the environmental noise energy value in order to eliminate the influence of the environmental noise, and removes the environmental noise energy value from the actual high frequency energy value of the sound signal. In this way, the accuracy of the high-frequency energy value of the confirmation sound signal can be improved.

With reference to the first aspect, in one possible implementation manner of the first aspect, the electronic device determines a high-frequency energy value of the sound signal according to an actual high-frequency energy value of the sound signal and the environmental noise energy value, and specifically includes the steps that the electronic device acquires the high-frequency signal in the sound signal, divides the high-frequency signal into high-frequency signal segments of n time windows, n is a positive integer, calculates the actual high-frequency energy value of the high-frequency signal segments in the n time windows, calculates the full-decibel high-frequency energy value of the high-frequency signal segments in the n time windows according to the actual high-frequency energy value of the high-frequency signal segments in the n time windows, and determines the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy value of the high-frequency signal segments in the n time windows and the environmental noise energy value. Here, since the frequency range of the abnormal high-frequency signal is in the high frequency range, only the high-frequency signal of the sound signal is acquired, and the high-frequency energy value of the sound signal is calculated according to the high-frequency signal of the sound signal, so that the operation of the electronic device can be reduced, and the consumption can be saved.

With reference to the first aspect, in one possible implementation manner of the first aspect, according to the actual high-frequency energy values of the high-frequency signal segments in the n time windows, the electronic device may determine full db high-frequency energy values of the high-frequency signal segments in the n time windows by using the following formula:

pb _i is the full-dB high-frequency energy value of the high-frequency signal section in the ith time window, sb _i is the actual high-frequency energy value of the medium-high-frequency signal section in the ith time window, sj is the number of bits required by the electronic equipment for storing the high-frequency signal section in the jth time window, j is a positive integer greater than or equal to 1 and less than or equal to n, and i is a positive integer less than or equal to n.

With reference to the first aspect, in one possible implementation manner of the first aspect, the electronic device may determine, according to the full db high frequency energy value and the environmental noise energy value of the high frequency signal segment in the n time windows, the high frequency energy value of the sound signal by the following formula:

Pb _i is the full-dB high-frequency energy value of the high-frequency signal section in the ith time window, pci is the difference between the full-dB high-frequency energy value of the high-frequency signal section in the ith time window and the environmental noise energy value, pm is the environmental noise energy value, i is a positive integer less than or equal to n, and P0 is the high-frequency energy value of the sound signal.

With reference to the first aspect, in a possible implementation manner of the first aspect, a frequency band range of the abnormal high frequency signal includes 18KHz-20KHz.

The application provides electronic equipment, which comprises a sound collecting module and an abnormal signal detecting frame, wherein the sound collecting module is used for obtaining sound signals, the abnormal signal detecting frame is used for determining high-frequency energy values of the sound signals, and the abnormal signal detecting frame is also used for judging that if the high-frequency energy values of the sound signals are larger than an environment sound energy threshold value, prompt information is sent to a user, and the prompt information is used for prompting that the abnormal high-frequency signals exist in the sound signals.

The electronic device detects the acquired sound signal according to the abnormal signal detection frame. If the abnormal signal detection frame detects that an abnormal high-frequency signal exists in the sound signal, a warning can be sent to a user or the abnormal high-frequency signal in the sound signal can be filtered out. This has the advantage that the privacy of the user can be prevented from being compromised without the user's knowledge. Thus, potential hidden communication threats can be identified and intercepted, and the risk of privacy information disclosure is reduced.

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to receive a filtering operation of a user, and respond to the filtering operation of the user to filter an abnormal high-frequency signal existing in the sound signal. In this way, the user can sense that an abnormal high-frequency signal exists in the sound signal acquired by the electronic equipment, and when the user passes through the area next time, the abnormal high-frequency signal existing in the area can be blocked, or the user reports the situation that the abnormal high-frequency signal exists in the area to the server, and the server gathers the information reported by the user and takes measures to prevent the privacy information of the user from being leaked. In some embodiments, when the electronic device determines that the high frequency energy value of the sound signal is greater than the environmental sound energy threshold, the electronic device does not need to send a prompt message to the user, and the electronic device directly removes the abnormal high frequency signal in the sound signal. Thus, when the user makes a call with friends through the electronic equipment or the electronic equipment is displaying a navigation route, the electronic equipment does not send prompt information to the user, and the current operation of the user is not affected.

With reference to the second aspect, in a possible implementation manner of the second aspect, the sound collecting module is further configured to send a sound signal from which the abnormal high frequency signal is removed to the receiving device. When a user calls a friend through the electronic equipment or plays a game with the friend, the abnormal signal detection framework calls the sound collection module, the sound collection module sends sound signals with abnormal high-frequency signals filtered to the receiving equipment through the communication module, and the receiving equipment is equipment which is used by the friend of the user, so that the current operation of the user cannot be influenced. In some embodiments, the abnormal signal detection framework does not need to call the sound collection module, and directly sends the sound signals with the abnormal high-frequency signals filtered out to the receiving device through the communication module.

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to obtain first environmental information, where the first environmental information includes location information and/or environmental sound, and determine an environmental sound energy threshold according to the first environmental information. Here, the first environmental information is used for determining a current environment (for example, a mall), the abnormal signal detection frame determines an environmental sound energy threshold corresponding to the current environment according to the current environment, the environmental sound energy threshold is used for comparing with a high-frequency energy value of the sound signal, and if the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold corresponding to the current environment, the abnormal signal detection frame determines that an abnormal high-frequency signal exists in the sound signal.

With reference to the second aspect, in a possible implementation manner of the second aspect, the sound collecting module is further configured to obtain an ambient sound before obtaining the sound signal, the abnormal signal detecting frame is further configured to determine an ambient noise energy value according to the ambient sound, and determine a high frequency energy value of the sound signal according to an actual high frequency energy value of the sound signal and the ambient noise energy value. Here, the electronic device determines the high frequency energy value of the sound signal in order to eliminate the influence of the environmental noise based on the actual high frequency energy value of the sound signal and the environmental noise energy value, and removes the environmental noise energy value from the actual high frequency energy value of the sound signal, so that the accuracy of confirming the high frequency energy value of the sound signal can be improved.

With reference to the second aspect, in one possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to obtain a high-frequency signal in the sound signal, divide the high-frequency signal into high-frequency signal segments of n time windows, where n is a positive integer, calculate an actual high-frequency energy value of the high-frequency signal segments in the n time windows, calculate a full db high-frequency energy value of the high-frequency signal segments in the n time windows according to the actual high-frequency energy value of the high-frequency signal segments in the n time windows, and determine a high-frequency energy value of the sound signal according to the full db high-frequency energy value of the high-frequency signal segments in the n time windows and an ambient noise energy value. Here, since the frequency range of the abnormal high frequency signal is in the high frequency range, the abnormal signal detection frame only acquires the high frequency signal of the sound signal, and calculates the high frequency energy value of the sound signal according to the high frequency signal of the sound signal, so that the operation can be reduced, and the consumption can be saved.

With reference to the second aspect, in one possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to determine, according to actual high frequency energy values of the high frequency signal segments in the n time windows, full db high frequency energy values of the high frequency signal segments in the n time windows by using the following formula:

With reference to the second aspect, in one possible implementation manner of the second aspect, the abnormal signal detection framework is specifically configured to determine, according to a full db high frequency energy value and an environmental noise energy value of a high frequency signal segment in n time windows, a high frequency energy value of a sound signal according to the following formula:

Pc_i＝Pb_i–Pm;

With reference to the second aspect, in a possible implementation manner of the second aspect, a frequency band range of the high-frequency signal includes 18KHz-20KHz.

In a third aspect, the application provides an abnormal signal filtering device comprising one or more processors, one or more memories and a sound collector, wherein the one or more memories and the sound collector are coupled with the one or more processors, the one or more memories are used for storing computer program codes, the computer program codes comprise computer instructions, and the one or more processors call the computer instructions to enable the device to execute the abnormal signal filtering method in any possible implementation manner of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer storage medium, including computer instructions, which when executed on an electronic device, cause a communication apparatus to perform a method for filtering an anomaly signal in any one of the possible implementations of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer program product, which when run on a computer causes the computer to perform a method of filtering an anomaly signal in any one of the possible implementations of the first aspect.

The application provides a method and a device for filtering abnormal signals. The electronic device detects the acquired sound signal according to a detection model trained in advance. If the electronic device detects the presence of an abnormal high frequency signal in the sound signal, a warning may be issued to the user or the abnormal high frequency signal in the sound signal may be filtered out. This has the advantage that the privacy of the user can be prevented from being compromised without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, and the risk of privacy information disclosure is reduced.

Drawings

FIG. 1 is a flowchart of a method for obtaining user privacy information by a hidden communication technology according to an embodiment of the present application;

fig. 2 is a schematic architecture diagram of an electronic device 100 according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a system 10 according to an embodiment of the present application;

FIG. 4 is a functional schematic of each module according to an embodiment of the present application;

FIG. 5 is a flowchart of a method for filtering abnormal signals according to an embodiment of the present application;

FIG. 6 is a flowchart illustrating another method for filtering abnormal signals according to an embodiment of the present application;

FIGS. 7A-7F are a set of UI diagrams provided by an embodiment of the application;

Fig. 8A to fig. 8D are schematic flow diagrams of an abnormal high-frequency signal detection and filtering method according to an embodiment of the present application;

fig. 9 is a schematic diagram of an apparatus according to an embodiment of the present application.

Detailed Description

The following description will be given in detail of the technical solutions in the embodiments of the present application with reference to the accompanying drawings. In the description of the embodiment of the present application, unless otherwise indicated, "/" means or, for example, a/B may represent a or B, and "and/or" in the text is merely an association relationship describing an association object, which means that three relationships may exist, for example, a and/or B, and that three cases of a alone, a and B together, and B alone exist, and further, in the description of the embodiment of the present application, "a plurality" means two or more.

The terms "first," "second," and the like, are used below for descriptive purposes only and are not to be construed as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature, and in the description of embodiments of the application, unless otherwise indicated, the meaning of "a plurality" is two or more.

Currently, the hidden communication technology based on sound waves is mainly implemented through high-frequency signals and application programs capable of identifying the high-frequency signals. That is, the application program, upon recognizing a specific high frequency signal, may perform an operation set in advance by the application program developer, for example, acquiring privacy information of the user. The frequency of the high frequency signal is typically greater than 18KHz, and most users are unable to perceive the presence of the high frequency signal in the surrounding environment because most users can only hear sounds at frequencies below 16KHz due to the limitations of the human ear hearing system (human auditory system, HAS). Meanwhile, most of the hardware configurations of the existing electronic devices can support the reception and demodulation of signals above 18KHz, but lack control measures for high-frequency signals. In other words, it is difficult for most users to know whether or not there is a high frequency signal at present, and whether or not there is a hidden communication at present. If the application program acquires the privacy information (such as the electronic device identifier and the position of the electronic device) of the user after identifying the specific high-frequency signal and transmits the privacy information to the specific server or the electronic device, the privacy information of the user, such as the user identity information, the position information, the operation habit and the like, can be acquired under the condition that the user is not felt.

Fig. 1 shows a flowchart of a method for obtaining user privacy information through a hidden communication technology according to an embodiment of the present application. As shown in FIG. 1, the sound at the entrance of the store is now playing a promotional advertisement containing an abnormally high frequency signal synthesized by a third party. The electronic device carried by the user may capture the sound of the promotional advertisement as the user passes through the store during running. When an application program capable of identifying the high-frequency signal contained in the promotion advertisement exists in the user electronic equipment, the application program may acquire the identification of the user electronic equipment and the position of the electronic equipment after identifying the abnormal high-frequency signal and send the identification and the position of the electronic equipment to a third party. The application may be a normally functioning application. For example, the electronic device receives the voice assistant which is opened by the user operation and collects the voice signals, for example, the user plays a game with a friend team through a game application program, for convenience in communication, the user opens a microphone control of a game interface, the voice collection module of the user electronic device collects the voice signals, for example, the user performs video call with friends in a mall through a social application program, and the voice collection module of the user electronic device collects the voice. If the information of hidden communication exists in the position of the user (for example, a high-frequency signal exists in music played in a mall), the user electronic equipment collects the sound signal and sends the sound signal to an application program, and the application program in the user electronic equipment (for example, a voice assistant, a game application program and a social application program) can identify the high-frequency signal in the sound signal, and the high-frequency signal triggers the application program to acquire the identification of the user electronic equipment and the position of the electronic equipment and send the identification and the position of the electronic equipment to a third party.

It should be noted that the above application scenario is only for explaining the present application, and may include more or fewer application scenarios, and should not be construed as limiting.

Therefore, when the electronic device with the voice input function acquires the external voice signal, the first application (for example, the navigation application) installed in the electronic device has the risk of acquiring the user privacy information by the hidden communication and causing the personal privacy disclosure.

In order to identify the high frequency signal, there are two ways to detect the sound signal.

In one mode, the abnormal sound detection and abnormal sound classification technology is mainly used for detecting abnormal sounds in environmental sounds, such as squeaking sounds, gunshot sounds, broken sounds and the like, which represent abnormal events. The abnormal sound detection technology respectively models various abnormal sound signals by extracting characteristics such as zero-crossing rate, amplitude difference, power spectral density, spectral entropy, mel frequency cepstrum coefficient (mel-frequency cepstral coefficients, MFCC) and the like from a sample sound set, and respectively trains a classifier for the various abnormal sound signals. And extracting various characteristics such as zero-crossing rate, amplitude difference, power spectral density, spectral entropy, mel frequency cepstrum coefficient and the like from the sound to be detected for anomaly detection, so that the sound to be detected is classified into a specific sound class, and if the signal to be detected is not classified into the specific sound class, judging that the sound to be detected is an unknown sound class. The method one is mainly used for detecting abnormal sounds which can be perceived by human beings in environmental sounds, and is large in calculated amount, bad in real-time performance and unfavorable for being realized in electronic equipment.

The second mode is that the ultrasonic voice hiding attack detection method is to obtain the center frequency of the ultrasonic signal by carrying out frequency spectrum analysis on the ultrasonic signal collected by the electronic equipment, judge whether the center frequency is in the set attack frequency range, if so, carry out band-pass filtering on the ultrasonic signal, filter the frequency components of the upper limit and the lower limit of the voice frequency, so as to obtain the baseband signal, and detect whether the baseband signal contains voice. The second mode can only detect the attack mode of the ultrasonic signal carrying the voice signal, and can not identify the hidden attack behavior generated by transmitting the non-voice signal by the ultrasonic signal.

The application provides a method for detecting and filtering an abnormal high-frequency signal. The electronic device detects the acquired sound signal according to a detection model trained in advance. If the electronic device detects the presence of an abnormal high frequency signal in the sound signal, a warning may be issued to the user or the abnormal high frequency signal in the sound signal may be filtered out. This has the advantage that the privacy of the user can be prevented from being compromised without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, the risk of privacy information disclosure is reduced, and user experience is ensured.

For example, when a user passes through a mall, a sound placed in the mall is playing a promotional advertisement or music to the outside. The promotion advertisement or music is synthesized with an abnormal high-frequency signal, and the user electronic equipment (such as a mobile phone) can detect the abnormal high-frequency signal and send an alarm prompt to the user so as to prevent a third party from acquiring the privacy of the user and push the advertisement to the user.

For another example, when the user watches the television program at home, an abnormal high-frequency signal is synthesized in the audio played in the television program (the special information carried by the abnormal high-frequency signal can be used for marking the video content currently played, such as a computer promotion, a clothes advertisement and the like), the user electronic equipment (such as a mobile phone) can detect the abnormal high-frequency signal and send an alarm prompt to the user, so that a third party is prevented from grasping the habit of watching the program by the user, and the recommended program is pushed to the user.

For another example, a user accesses an anonymous network through an electronic device (e.g., a desktop computer), and a third party server distributes audio and video content combined with an abnormal high frequency signal in the anonymous network, and the audio and video content combined with the abnormal high frequency signal can attract the anonymous user to watch. The user electronic device (e.g., a cell phone) may detect the abnormal high frequency signal and send an alert prompt to the user to prevent a third party from obtaining real information of the user accessing the anonymous network.

It should be noted that, the hidden communication based on the sound wave is widely used to protect the security of the secret information, for example, the information that the sending end needs to transmit in a secret way is modulated into the high-frequency carrier wave, then the high-frequency carrier wave is hidden in the common information, the secret information is published through the common information, and the attacker can hardly perceive the existing secret information from the common information, so that the secret information has greater concealment and security. The receiving end receives the common information and demodulates the secret information in the common information, at this time, the receiving end will display the corresponding prompt information, and the prompt information may be the prompt information containing the device identifier of the sending end, for example, the prompt information may be "receive secret information sent from the HUAWEI P30", then the user of the receiving end judges that the information is useful information instead of junk information according to the prompt information, and the user may choose to not filter the signal, so as to receive the secret information sent by the sending end. The method can ensure that useful information is not mistakenly regarded as junk information to be filtered.

The above examples are only for explaining the present application and should not be construed as limiting.

Fig. 2 shows a schematic structural diagram of the electronic device 100.

The embodiment will be specifically described below taking the electronic device 100 as an example. The device types of the electronic device 100 may include cell phones, televisions, tablet computers, speakers, watches, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, and personal digital assistants (personal digitalassistant, PDA), augmented reality (augmentedreality, AR)/virtual reality (virtualreality, VR) devices, and the like. The embodiment of the present application does not particularly limit the device type of the electronic device 100.

It should be understood that the electronic device 100 shown in fig. 2 is only one example, and that the electronic device 100 may have more or fewer components than shown in fig. 2, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It should be understood that the illustrated structure of the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), a controller, a memory, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller may be a neural hub and a command center of the electronic device 100, among others. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-INTEGRATED CIRCUIT, I2C) interface, an integrated circuit built-in audio (inter-INTEGRATED CIRCUIT SOUND, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.

The I2C interface is a bi-directional synchronous serial bus comprising a serial data line (SERIAL DATA LINE, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., respectively, through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through an I2C interface, such that the processor 110 communicates with the touch sensor 180K through an I2C bus interface, to implement a touch function of the electronic device 100.

The I2S interface may be used for audio communication. In some embodiments, the processor 110 may contain multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may transmit an audio signal to the wireless communication module 160 through the I2S interface, to implement a function of answering a call through the bluetooth headset.

PCM interfaces may also be used for audio communication to sample, quantize and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface to implement a function of answering a call through the bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus for asynchronous communications. The bus may be a bi-directional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 110 with the wireless communication module 160. For example, the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement bluetooth functions. In some embodiments, the audio module 170 may transmit an audio signal to the wireless communication module 160 through a UART interface, to implement a function of playing music through a bluetooth headset.

The MIPI interface may be used to connect the processor 110 to peripheral devices such as a display 194, a camera 193, and the like. The MIPI interfaces include camera serial interfaces (CAMERA SERIAL INTERFACE, CSI), display serial interfaces (DISPLAY SERIAL INTERFACE, DSI), and the like. In some embodiments, processor 110 and camera 193 communicate through a CSI interface to implement the photographing functions of electronic device 100. The processor 110 and the display 194 communicate via a DSI interface to implement the display functionality of the electronic device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or as a data signal. In some embodiments, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, an MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transfer data between the electronic device 100 and a peripheral device. And can also be used for connecting with a headset, and playing audio through the headset. The interface may also be used to connect other electronic devices, such as AR devices, etc.

It should be understood that the interfacing relationship between the modules illustrated in the embodiments of the present application is only illustrative, and is not meant to limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also employ different interfacing manners in the above embodiments, or a combination of multiple interfacing manners.

The charge management module 140 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charge management module 140 may receive a charging input of a wired charger through the USB interface 130. In some wireless charging embodiments, the charge management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 may also supply power to the electronic device through the power management module 141 while charging the battery 142.

The power management module 141 is used for connecting the battery 142, and the charge management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be configured to monitor battery capacity, battery cycle number, battery health (leakage, impedance) and other parameters. In other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charge management module 140 may be disposed in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can amplify the signal modulated by the modem processor, and convert the signal into electromagnetic waves through the antenna 1 to radiate. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be provided in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or video through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional module, independent of the processor 110.

The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (WIRELESS FIDELITY, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation SATELLITE SYSTEM, GNSS), frequency modulation (frequency modulation, FM), near field communication (NEAR FIELD communication, NFC), infrared (IR), etc., applied to the electronic device 100. The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.

In some embodiments, antenna 1 and mobile communication module 150 of electronic device 100 are coupled, and antenna 2 and wireless communication module 160 are coupled, such that electronic device 100 may communicate with a network and other devices through wireless communication techniques. The wireless communication techniques can include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (GENERAL PACKET radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation SATELLITE SYSTEM, GLONASS), a beidou satellite navigation system (beidou navigation SATELLITE SYSTEM, BDS), a quasi zenith satellite system (quasi-zenith SATELLITE SYSTEM, QZSS) and/or a satellite based augmentation system (SATELLITE BASED AUGMENTATION SYSTEMS, SBAS).

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. The display panel may employ a Liquid Crystal Display (LCD) CRYSTAL DISPLAY, an organic light-emitting diode (OLED), an active-matrix organic LIGHT EMITTING diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microLed, micro-oLed, a quantum dot LIGHT EMITTING diode (QLED), or the like. In some embodiments, the electronic device 100 may include 1 or N display screens 194, N being a positive integer greater than 1.

The electronic device 100 may implement photographing functions through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process data fed back by the camera 193. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in the camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to fourier transform the frequency bin energy, or the like.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record video in a variety of encoding formats, such as moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor, and can rapidly process input information by referencing a biological neural network structure, for example, referencing a transmission mode between human brain neurons, and can also continuously perform self-learning. Applications such as intelligent recognition of the electronic device 100, for example, image recognition, face recognition, voice recognition, text understanding, etc., can be realized through the NPU.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to enable expansion of the memory capabilities of the electronic device 100. The external memory card communicates with the processor 110 through an external memory interface 120 to implement data storage functions. For example, files such as music, video, etc. are stored in an external memory card.

The internal memory 121 may be used to store computer executable program code including instructions. The processor 110 executes various functional applications of the electronic device 100 and data processing by executing instructions stored in the internal memory 121. The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data created during use of the electronic device 100 (e.g., audio data, phonebook, etc.), and so on. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or a portion of the functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also referred to as a "horn," is used to convert audio electrical signals into sound signals. The electronic device 100 may listen to music, or to hands-free conversations, through the speaker 170A.

A receiver 170B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal. When electronic device 100 is answering a telephone call or voice message, voice may be received by placing receiver 170B in close proximity to the human ear.

Microphone 170C, also referred to as a "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can sound near the microphone 170C through the mouth, inputting a sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, and may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may also be provided with three, four, or more microphones 170C to enable collection of sound signals, noise reduction, identification of sound sources, directional recording functions, etc.

The earphone interface 170D is used to connect a wired earphone. The headset interface 170D may be a USB interface 130 or a 3.5mm open mobile electronic device platform (open mobile terminal platform, OMTP) standard interface, a american cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used to sense a pressure signal, and may convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A is of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a capacitive pressure sensor comprising at least two parallel plates with conductive material. The capacitance between the electrodes changes when a force is applied to the pressure sensor 180A. The electronic device 100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display screen 194, the electronic apparatus 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 may also calculate the location of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch location, but at different touch operation strengths, may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity smaller than a first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. And executing an instruction for newly creating the short message when the touch operation with the touch operation intensity being greater than or equal to the first pressure threshold acts on the short message application icon.

The gyro sensor 180B may be used to determine a motion gesture of the electronic device 100. In some embodiments, the angular velocity of electronic device 100 about three axes (i.e., x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. For example, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the electronic device 100, calculates the distance to be compensated by the lens module according to the angle, and makes the lens counteract the shake of the electronic device 100 through the reverse motion, so as to realize anti-shake. The gyro sensor 180B may also be used for navigating, somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, electronic device 100 calculates altitude from barometric pressure values measured by barometric pressure sensor 180C, aiding in positioning and navigation.

The magnetic sensor 180D includes a hall sensor. The electronic device 100 may detect the opening and closing of the flip cover using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 may detect the opening and closing of the flip according to the magnetic sensor 180D. And then according to the detected opening and closing state of the leather sheath or the opening and closing state of the flip, the characteristics of automatic unlocking of the flip and the like are set.

The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity may be detected when the electronic device 100 is stationary. The electronic equipment gesture recognition method can also be used for recognizing the gesture of the electronic equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.

A distance sensor 180F for measuring a distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, the electronic device 100 may range using the distance sensor 180F to achieve quick focus.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it may be determined that there is an object in the vicinity of the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there is no object in the vicinity of the electronic device 100. The electronic device 100 can detect that the user holds the electronic device 100 close to the ear by using the proximity light sensor 180G, so as to automatically extinguish the screen for the purpose of saving power. The proximity light sensor 180G may also be used in holster mode, pocket mode to automatically unlock and lock the screen.

The ambient light sensor 180L is used to sense ambient light level. The electronic device 100 may adaptively adjust the brightness of the display 194 based on the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust white balance when taking a photograph. Ambient light sensor 180L may also cooperate with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent false touches.

The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 may utilize the collected fingerprint feature to unlock the fingerprint, access the application lock, photograph the fingerprint, answer the incoming call, etc.

The temperature sensor 180J is for detecting temperature. In some embodiments, the electronic device 100 performs a temperature processing strategy using the temperature detected by the temperature sensor 180J. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs a reduction in the performance of a processor located in the vicinity of temperature sensor 180J in order to reduce power consumption to implement thermal protection. In other embodiments, when the temperature is below another threshold, the electronic device 100 heats the battery 142 to avoid the low temperature causing the electronic device 100 to be abnormally shut down. In other embodiments, when the temperature is below a further threshold, the electronic device 100 performs boosting of the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperatures.

The touch sensor 180K, also referred to as a "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is for detecting a touch operation acting thereon or thereabout. The touch sensor may communicate the detected touch operation to the application processor to determine the touch event type. Visual output related to touch operations may be provided through the display 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a different location than the display 194.

The bone conduction sensor 180M may acquire a vibration signal. In some embodiments, bone conduction sensor 180M may acquire a vibration signal of a human vocal tract vibrating bone pieces. The bone conduction sensor 180M may also contact the pulse of the human body to receive the blood pressure pulsation signal. In some embodiments, bone conduction sensor 180M may also be provided in a headset, in combination with an osteoinductive headset. The audio module 170 may analyze the voice signal based on the vibration signal of the sound portion vibration bone block obtained by the bone conduction sensor 180M, so as to implement a voice function. The application processor may analyze the heart rate information based on the blood pressure beat signal acquired by the bone conduction sensor 180M, so as to implement a heart rate detection function.

The keys 190 include a power-on key, a volume key, etc. The keys 190 may be mechanical keys. Or may be a touch key. The electronic device 100 may receive key inputs, generating key signal inputs related to user settings and function controls of the electronic device 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration alerting as well as for touch vibration feedback. For example, touch operations acting on different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects by touching different areas of the display screen 194. Different application scenarios (such as time reminding, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

The indicator 192 may be an indicator light, may be used to indicate a state of charge, a change in charge, a message indicating a missed call, a notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card may be inserted into the SIM card interface 195, or removed from the SIM card interface 195 to enable contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 195 may support Nano SIM cards, micro SIM cards, and the like. The same SIM card interface 195 may be used to insert multiple cards simultaneously. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as communication and data communication. In some embodiments, the electronic device 100 employs an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

In order to facilitate understanding of the embodiments of the present application, the system architecture of the embodiments of the present application is described below.

Fig. 3 schematically illustrates an architecture of a system 10 provided by an embodiment of the present application. As shown in fig. 3, the system 10 includes an electronic device 100 and a server 200.

The electronic device 100 comprises a sound collection module 101, an abnormal signal detection framework 102 and an application 103. The sound collection module 101 may be configured to collect sound and transmit a sound signal to the application 103. The abnormal signal detection frame 102 may be configured to detect the sound signal received by the application 103, and determine whether an abnormal high-frequency signal exists in the sound signal. If it is determined that an abnormal high-frequency signal exists in the sound signal, the abnormal signal detection frame 102 may send an alarm prompt to the user and filter the abnormal high-frequency signal in the sound signal. The application 103 may be configured to send an acquisition request to the sound collection module 101, where the acquisition request is used to request the sound collection module 101 to collect a sound signal, and the application 103 is further configured to receive the sound signal sent by the sound collection module 101.

The server 200 includes a detection module 104. The detection module 104 may be configured to train each ambient sound energy threshold from the sample sound dataset using a detection model.

The detection model can comprise each environment identifier and the environment sound energy threshold corresponding to each environment identifier, or the detection model comprehensively considers the environment sound energy threshold corresponding to each environment identifier to obtain the average environment sound energy threshold.

In some embodiments, after the server 200 trains the detection model, the detection model is sent to the electronic device 100. The electronic device 100 receives the detection model transmitted by the server 200.

Or the server 200 periodically updates the detection model according to the sample sound data set, and the server 200 periodically transmits the detection model to the electronic device 100. The electronic device 100 receives the detection model transmitted from the server 200 and periodically updates the detection model.

In other embodiments, the manufacturer of the electronic device 100 presets the detection model in the electronic device 100. That is, when the user starts up and starts using after purchasing the electronic device 100, the detection model already exists in the electronic device 100, and the electronic device 100 does not need to obtain the detection model from the server 200.

In other embodiments, the application developer of the first application presets the detection model in the first application, and when the electronic device 100 receives a user operation to download the first application from the application store, the detection model is downloaded to the electronic device 100 simultaneously with the first application. Thus, the electronic device 100 does not need to obtain the detection model from the server 200.

In other embodiments, the electronic device 100 may train the detection model by collecting sound samples, or by obtaining sound samples from a network.

In the embodiment in which the electronic device 100 does not need to obtain the detection model from the server 200, the detection model in the electronic device 100 is already trained, that is, the detection model includes each environment identifier and the environmental sound energy threshold corresponding to each environment identifier, or the detection model comprehensively considers the environmental sound energy thresholds corresponding to each environment identifier and only includes the average environmental sound energy threshold. It will be appreciated that embodiments of the application

The above-described embodiments are only for explaining the present application, and should not be construed as limiting.

The detection model is used for detecting whether an abnormal high-frequency signal exists in the sound signal. After the electronic device 100 acquires the detection model, it may be determined whether an abnormal high-frequency signal exists in the acquired sound signal according to the detection model.

In some embodiments, the detection model comprehensively considers the environmental sound energy thresholds corresponding to the environmental identifiers to obtain an average environmental sound energy threshold. When the electronic device 100 needs to detect the collected sound signal, the electronic device 100 first determines the high-frequency energy value of the sound signal, the electronic device 100 inputs the high-frequency energy value of the sound signal into a detection model, the detection model determines that if the high-frequency energy value of the sound signal is greater than the average environmental sound energy threshold, the detection model outputs a detection result, and the detection result is that an abnormal high-frequency signal exists in the sound signal.

In other embodiments, the detection model may include each environment identifier and an environmental sound energy threshold corresponding to each environment identifier, when the electronic device 100 needs to detect the collected sound signal, the electronic device 100 determines a high-frequency energy value of the sound signal, the electronic device 100 determines the current environment identifier (for example, a mall) according to the sound signal, and the electronic device 100 inputs the high-frequency energy value of the sound signal into the detection model and determines the environmental sound energy threshold corresponding to the current environment identifier from the detection model according to the current environment identifier. The detection model judges that if the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold corresponding to the current environmental identifier, the detection model outputs a detection result, and the detection result is that an abnormal high-frequency signal exists in the sound signal.

The abnormal signal detection framework 102 includes an abnormal signal detection module 1021, a model management module 1022, an alarm module 1023, an alarm application 1024, and a filtering module 1025.

The abnormal signal detection module 1021 may be configured to detect whether an abnormal high-frequency signal exists in the sound signal acquired by the sound acquisition module 101. The abnormal signal detection module 1021 receives the detection model sent by the model management module 1022, and the abnormal signal detection module 1021 may calculate the high frequency energy value of the sound signal first, then input the high frequency energy value into the detection model, and if the detection model determines that the high frequency energy value of the sound signal is greater than the environmental sound energy threshold (the environmental sound energy threshold may be an average environmental sound energy threshold or an environmental sound energy threshold corresponding to the current environmental identifier), the detection model determines that the abnormal high frequency signal exists in the sound signal.

The model management module 1022 is responsible for downloading and updating the detection model, and sends the detection model to the abnormal signal detection module 1021.

The alert module 1023 is operable to send an alert instruction to the alert application 1024, the alert instruction being operable to instruct the alert application 1024 to present alert information to a user.

An alert application 1024 may be used to present alert prompts to the user, provide a user operation interface, and obtain user operations (e.g., choose to ignore alerts or choose to confirm alerts).

The filtering module 1025 may be configured to filter out the abnormal high frequency signal in the sound signal after receiving the user confirmation operation.

In one possible implementation, after the electronic device 100 receives the user canceling the filtering operation, that is, the user selects to not filter out the abnormal high-frequency signal in the sound signal, the alert application 1024 sends a first alert message to the alert module 1023, where the first alert message is used to alert the alert module 1023 to call the sound collection module 101, and the sound collection module 101 returns the sound signal without filtering out the abnormal high-frequency signal to the application 103.

In another possible implementation manner, after the electronic device 100 receives the filtering operation confirmed by the user, that is, the user selects to filter the abnormal high-frequency signals in the sound signals, the alert application 1024 sends a second alert message to the alert module 1023, where the second alert message is used to alert the alert module 1023 to send a filtering instruction to the filtering module 1025, the filtering module 1025 filters the abnormal high-frequency signals in the sound signals, and at the same time, the filtering module 1025 invokes the sound collection module 101, and the sound collection module 101 returns the sound signals with the abnormal high-frequency signals filtered back to the application 103.

The above-described embodiments are only for explaining the present application, and should not be construed as limiting. In some embodiments, the detection model is located in the server 200. That is, after the detection module 104 trains the detection model, the server 200 does not need to send the detection model to the electronic device 100. At this time, the electronic device may send the sound signal to the server after acquiring the sound signal. After receiving the sound signal, the server can determine whether the sound signal has an abnormal high-frequency signal according to the detection model, and return the determination result to the electronic equipment.

Illustratively, after the electronic device 100 acquires the sound signal, the acquired sound signal is sent to the server 200. The abnormal signal detection framework 102 in the server 200 confirms the high-frequency energy value of the sound signal, and the server 200 confirms the current environment identifier according to the sound signal, and matches the environment sound energy threshold corresponding to the current environment identifier from the detection model. If the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold, the detection model outputs a detection result, and the detection result is that an abnormal high-frequency signal exists in the sound signal, the server 200 sends a warning instruction to the electronic device 100, the electronic device 100 receives and responds to the warning instruction, and the electronic device 100 sends a warning prompt to a user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, the electronic device 100 transmits the filtering instruction to the server 200, and the abnormality signal detection frame 102 in the server 200 filters out the abnormality high frequency signal in the sound signal. The server 200 returns the sound signal from which the abnormal high frequency signal is filtered to the electronic device 100, and the electronic device 100 receives the sound signal from which the abnormal high frequency signal is filtered and transmits to the application.

It is understood that the detection model and the abnormal signal detection framework 102 shown in fig. 3 may be located in the server 200. At this time, the abnormal signal detection module 1021, the model management module 1022, and the filtering module 1025 in the abnormal signal detection framework 102 are located in the server 200, and the alarm application 1024 and the alarm module 1023 in the abnormal signal detection framework 102 are located in the electronic device 100. There is no limitation as to whether the respective modules are located in the server 200 or in the electronic device 100. Another possible implementation is that the abnormal signal detection framework 102 may be located in the electronic device 100 and the detection model in the server 200. The electronic device 100 may confirm the high frequency energy value of the sound signal after the electronic device 100 collects the sound signal, and the electronic device 100 confirms the current environment identification according to the sound signal, and the electronic device 100 transmits the current environment identification to the server 200. The server 200 matches the environmental sound energy threshold corresponding to the current environmental identifier from the detection model according to the current environmental identifier. The server 200 sends the ambient sound energy threshold to the electronic device 100. If the electronic device 100 determines that the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold, an abnormal high-frequency signal exists in the sound signal, the server 200 sends a warning prompt to the electronic device 100, and the electronic device 100 displays warning information to the user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, the electronic device 100 transmits the filtering instruction to the server 200, the abnormality signal detection frame 102 in the server 200 filters out the abnormality high frequency signal from the sound signal, the server 200 returns the sound signal from which the abnormality high frequency signal was filtered out to the electronic device 100, and the electronic device 100 receives the sound signal from which the abnormality high frequency signal was removed and transmits to the application 103.

In other embodiments, the detection model is located in the electronic device 100. That is, after the detection module 104 trains the detection model, the server 200 sends the detection model to the electronic device 100 or the detection model already exists in the electronic device 100 in advance, and the electronic device 100 does not need to acquire the detection model from the server 200.

One possible implementation is that the detection model, the abnormal signal detection framework 102, may all be located in the electronic device 100. After the electronic device 100 acquires the sound signal, the abnormal signal detection framework 102 in the electronic device 100 confirms the high-frequency energy value of the sound signal, and the electronic device 100 confirms the current environment identifier according to the sound signal, and matches the environment sound energy threshold corresponding to the current environment identifier from the detection model. If the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold, the detection model outputs a detection result, and the detection result is that an abnormal high-frequency signal exists in the sound signal, and the electronic device 100 sends a warning prompt to the user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, and the abnormality signal detection frame 102 in the electronic device 100 filters out the abnormality high frequency signal from the sound signal and transmits the sound signal from which the abnormality high frequency signal is filtered out to the application.

Another possible implementation is that the detection model is located in the electronic device 100 and the abnormal signal detection framework 102 may be located in the server 200.

At this time, the abnormal signal detection module 1021, the model management module 1022, and the filtering module 1025 in the abnormal signal detection framework 102 are located in the server 200, and the alarm application 1024 and the alarm module 1023 in the abnormal signal detection framework 102 are located in the electronic device 100. There is no limitation as to whether the respective modules are located in the server 200 or in the electronic device 100.

After the electronic device 100 collects the sound signal, the electronic device 100 may send the sound signal to the server 200, the abnormal signal detection frame 102 in the server 200 confirms the high-frequency energy value of the sound signal, and the electronic device 100 confirms the current environment identifier according to the sound signal, and matches the environmental sound energy threshold corresponding to the current environment identifier. The electronic device 100 transmits the environmental sound energy threshold to the server 200, and the server 200 determines that the abnormal high-frequency signal exists in the sound signal if the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold. The server 200 sends a warning alert to the electronic device 100, and the electronic device 100 displays warning information to the user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, the electronic device 100 transmits the filtering instruction to the server 200, the abnormal signal detection frame 102 in the server 200 filters out the abnormal high frequency signal in the sound signal, the server 200 returns the sound signal from which the abnormal high frequency signal was filtered out to the electronic device 100, and the electronic device 100 receives the sound signal from which the abnormal high frequency signal was removed and transmits to the application.

As shown in fig. 4, the functions of the respective modules included in the sound collection module 101, the detection module 104, and the abnormal signal detection frame 102 are described.

The sound collection module 101 may be composed of a microphone for collecting sound signals, a processor for converting the collected sound signals into analog signals, and the like.

For example, the electronic device 100 does not need to collect the sound signal by the sound collection module 101, the sound signal obtained by the electronic device 100 may be obtained by an intelligent wearable device (such as a bluetooth watch, a bluetooth headset, etc.), the intelligent wearable device (such as a bluetooth watch, a bluetooth headset, etc.) establishes a communication connection with the electronic device 100, the intelligent wearable device (such as a bluetooth watch, a bluetooth headset, etc.) obtains the sound signal and sends the sound signal to the electronic device 100, and the electronic device 100 obtains the sound signal.

The model management module 1022 receives the detection model and sends to the anomaly signal detection module 1021. In some embodiments, the detection model may be transmitted by the detection module 104 shown in FIG. 3. In other embodiments, the detection model may also be trained by the electronic device itself.

The sound collection module 101 collects sound, and the sound collection module 101 sends a sound signal to the abnormal signal detection module 1021.

The abnormal signal detection module 1021 detects whether an abnormal high frequency signal exists in the sound signal, and when detecting that the abnormal high frequency signal exists in the sound signal, the abnormal signal detection module 1021 sends an abnormal command for prompting the alarm module 1013 that the abnormal high frequency signal exists in the sound signal.

Meanwhile, the abnormal signal detection module 1021 sends the sound signal to the filtering module 1025.

Specifically, the abnormal signal detection module 1021 performs detection on whether an abnormal high-frequency signal exists in the sound signal or not, which includes the abnormal signal detection module 1021 performing spectrum analysis on the sound signal to obtain a high-frequency energy value of the sound signal, inputting the high-frequency energy value of the sound signal into a detection model, and if the detection model judges that the high-frequency energy value of the sound signal is greater than an environmental sound energy threshold, the detection model outputs a detection result, and the detection result is that the abnormal high-frequency signal exists in the sound signal. The abnormal signal detection module 1021 determines that an abnormal high frequency signal exists in the sound signal.

The abnormal high-frequency signal refers to a high-frequency signal in a specified frequency band (for example, 18KHz-20 KHz) in the sound signal when the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold.

The alert module 1023 receives and responds to the abnormality instruction sent by the abnormality signal detection module 1021, and in response to the abnormality instruction, the alert module 1023 is configured to send an alert instruction to the alert application 1024, where the alert instruction is configured to instruct the alert application 1024 to send alert information to the user. The presentation of the alert information is in a wide variety of forms, and the details are described below.

The electronic device 100 receives the user operation, and generates a filtering instruction according to the user operation, and the electronic device 100 sends the filtering instruction to the filtering module 1025.

The filtering module 1025 receives and responds to the filtering instruction, and filters out the abnormal high-frequency signals in the sound signal.

The filtering module 1025 invokes the sound collection module 101, and the sound collection module 101 sends the sound signal with the abnormal high frequency signal filtered to the application 1013.

In some embodiments, the filtering module 1025 does not need to invoke the sound collection module 101, and the filtering module 1025 directly sends the sound signal with the abnormal high frequency signal filtered out to the application 1013.

Referring to fig. 5, fig. 5 is a flowchart of a method for filtering abnormal signals according to an embodiment of the present application, which can be applied to the system 10 shown in fig. 3, wherein the system 10 may include an electronic device 100 and a server 200. The electronic device 100 may include a sound collection module 101, an abnormal signal detection framework 102, and an application 103. For a specific description of the system 10, reference may be made to the embodiment shown in fig. 3 and will not be repeated here. Wherein the method may comprise:

S501, the server 200 trains to obtain sound energy thresholds in different environments by utilizing sound signal data sets in different environments, obtains average environment sound energy thresholds, and updates a detection model.

Specifically, 1, the server 200 calculates an environmental noise high frequency energy value P _z according to the environmental noise signal.

The server 200 acquires the ambient noise signal and reads the ambient noise signal for n time windows according to the ambient noise signal. The server 200 calculates the high frequency energy value under the full decibel scale (decibels full scale, DBFS) of the environmental noise signal of each time window according to the actual high frequency energy value of the environmental noise signal of each time window, and stores the high frequency energy value in the queue 1, wherein DBFS refers to the decibel value based on the full scale in the digital sound, and the calculation formula of the high frequency energy value under the full decibel scale of each time window can be as follows:

As shown in formula (1), pi represents the full db high frequency energy value of the high frequency ambient noise signal segment in the ith time window, S _i represents the actual high frequency (e.g., 18KHz-20 KHz) energy value of the high frequency ambient noise signal segment in the ith time window, and the actual high frequency energy value of the ambient noise signal can be obtained by means of short-time fourier transform or the like. Sj represents the maximum value expressed by the number of bits required for storing the high-frequency environmental noise signal segment in the jth time window, namely S _j＝2^j -1, j is a positive integer greater than or equal to 1 and less than or equal to n, and i is a positive integer less than or equal to n.

For example, if S _i =7841 is set and a 16-bit signed number is used to store each sound signal value, the maximum value that S _j can express is 2 ¹⁵ -1, and the full db high frequency energy value of the ambient noise signal in the current time window is-12.4 according to formula (1).

The server 200 may calculate full db high frequency energy values of the high frequency ambient noise signal segments for n time windows, respectively, through equation (1).

It is to be understood that equation (1) is merely used to explain the present application and should not be construed as limiting.

The server 200 obtains the high-frequency energy values of the first group of environmental noise signals according to the average high-frequency energy of the full db high-frequency energy values of the high-frequency environmental noise signal segments of the n time windows, and stores the average energy in the queue 2, where the calculation formula of the average high-frequency energy may be:

as shown in formula (2), RMS represents the average high frequency energy of the ambient noise signal, P _i represents the full db high frequency energy value of the high frequency ambient noise signal segment of the ith time window, n is a positive integer, and i is a positive integer less than or equal to n.

It is to be understood that equation (2) is merely illustrative of the present application and should not be construed as limiting.

The server 200 continues to acquire a new noise signal from the environment and reads the sound signals of n time windows according to the new noise signal, discards the existing sound signals in the n time windows, calculates the average high frequency energy of the new noise signal according to the formula (1) and the formula (2), and adds the new noise signal to the queue 2.

And so on, the average high frequency energy of the m sets of ambient noise signals is obtained in the manner described above and is present in the queue 2.

The server 200 calculates an arithmetic average of the average high frequency energy of the m sets of environmental noise signals in the queue 2 as the current environmental noise high frequency energy value P _z.

2. The server 200 calculates an ambient sound energy threshold Pt from the ambient sound signal sample dataset.

The server 200 first extracts a first set of sound signals from the current sample data set of environmental sound signals, reads sound signals of n time windows according to the first set of sound signals, and calculates full db high frequency energy values of n time windows according to the above-mentioned equation (1).

In order to eliminate the high-frequency signal interference of the environmental noise signal caused by the frequency shift phenomenon, the application subtracts the high-frequency energy value PZ of the current environmental noise signal from the full-decibel high-frequency energy value of the first group of sound signals calculated according to the formula (1), and the calculation formula can be:

As shown in the formula (3), pwi represents a high-frequency energy value difference between the full db high-frequency energy value of the high-frequency sound signal segment in the ith time window and the high-frequency energy value Pz of the current environmental noise signal, where Pz is the high-frequency energy value of the current environmental noise signal.

Next, the server 200 may calculate the average high frequency energy value of the first set of sound signals according to formula (2).

Then, the server 200 continues to extract m groups of sound signals from the current ambient sound signal dataset, and calculates the average high-frequency energy value of the m groups of sound signals by the method described in the second step.

Finally, the server 200 calculates an arithmetic average value of the average high frequency energy of the m groups of sound signals to obtain an environmental sound energy threshold Pt, where the calculation formula may be:

pt= (pw1+pw2..pwm)/m formula (4)

As shown in formula (4), pwm is the average high frequency energy value of the m-th group of sound signals, m representing sound signals in a total of m groups of different environments.

It is to be understood that equation (4) is merely illustrative of the present application and should not be construed as limiting.

It is to be understood that the server 200 comprehensively considers the high-frequency energy values of the sound signals in the respective environments, and the ambient sound energy threshold Pt represents an average ambient sound energy threshold.

In some embodiments, the server 200 may obtain each environment identifier and the corresponding environmental sound energy threshold for each environment identifier, respectively. At this time, the environmental identifiers and the environmental sound energy thresholds corresponding to the environmental identifiers can be obtained only through the formula (1), the formula (2) and the formula (3), and the detailed description of the embodiments is omitted herein.

In some embodiments, the detection model comprehensively considers the environmental sound energy thresholds corresponding to the environmental identifiers to obtain an average environmental sound energy threshold, and the server 200 updates the detection model, where the detection model includes the average environmental sound energy threshold.

In other embodiments, the server 200 updates the detection model, where each environmental identifier and the environmental sound energy threshold value corresponding to each environmental identifier are included in the detection model.

In some embodiments, after the server 200 updates the detection model, the server 200 sends the detection model to the electronic device 100, and the electronic device 100 receives the detection model.

Here, the electronic device 100 may refer to the embodiment described in fig. 3 for confirming whether the abnormal high-frequency signal exists in the sound signal according to the detection model, and the present application is not described herein.

In other embodiments, after the server 200 updates the detection model, the server 200 need not send the detection model to the electronic device 100.

The detection model may include each environment identifier and an environmental sound energy threshold corresponding to each environment identifier. The electronic device 100 confirms the high frequency energy value of the sound signal, and the electronic device 100 confirms the current environment identifier (e.g. a mall) according to the sound signal, or the electronic device 100 acquires the first environment information (e.g. the location information and/or the environment sound) and confirms the current environment identifier (e.g. the mall) according to the first environment information. The electronic device 100 sends the current environment identifier to the server 200, the detection model confirms the environmental sound energy threshold corresponding to the current environment identifier according to the current environment identifier, and the server 200 sends the environmental sound energy threshold corresponding to the current environment identifier to the electronic device 100. The electronic device 100 determines whether the high frequency energy value of the sound signal is greater than an ambient sound energy threshold corresponding to the current environment. If the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold corresponding to the current environment, an abnormal high-frequency signal exists in the sound signal.

Or the detection model comprehensively considers the environmental sound energy thresholds corresponding to the environmental identifiers to obtain an average environmental sound energy threshold. When the electronic device 100 needs to detect the collected sound signal, the electronic device 100 sends the sound signal to the server 200, the server 200 confirms the high-frequency energy value of the sound signal, and inputs the high-frequency energy value of the sound signal into the detection model, and the detection model outputs a detection result, wherein the detection result may be that an abnormal high-frequency signal exists in the sound signal or that an abnormal high-frequency signal does not exist in the sound signal. If the detection result is that an abnormal high-frequency signal exists in the sound signal, the server 200 transmits the sound signal to the electronic device 100 when the abnormal high-frequency signal exists in the sound signal.

S502, the application 103 sends a request for acquiring a sound signal to the sound collection module 101.

In some embodiments, the application 103 may send a request to acquire a sound signal to the sound collection module 101 in response to a user operation. By way of example, the application 103 may be a voice assistant and the user operation may be a voice instruction. For example, the voice command may be the wake word "small E", which the voice assistant receives and responds to the wake word "small E", which triggers the voice assistant to turn on, and in response to the voice assistant turning on, the voice assistant sends a request to the sound collection module 101 to acquire a sound signal. The application 103 may also be a game application. The user interface displayed by the application 103 includes a microphone control, and the microphone control receives and responds to a clicking operation of a user, and a game application program sends a request for acquiring a sound signal to the sound acquisition module 101, and the sound acquisition module 101 acquires the sound signal and sends the sound signal to the receiving device through the communication module, so that a user can conveniently exchange game experience with a game friend of the receiving device. The application 103 may also be a social application, where the user interface displayed by the application 103 includes a video call or audio call control, and the video call or audio call control receives and responds to a clicking operation of a user, and the social application sends a request for acquiring a sound signal to the sound acquisition module 101, where the sound acquisition module 101 acquires the sound signal and sends the sound signal to the receiving device through the communication module. In this way, the user may engage in a voice call or a video call with friends through the social application.

In other embodiments, the application 103 may send the request for acquiring the sound signal to the sound collection module 101 without the user agreeing. The embodiment of the application does not limit the triggering condition for sending the request for acquiring the sound signal.

S503, the sound collection module 101 responds to the request for obtaining the sound signal, and the sound collection module 101 collects the sound.

In some embodiments, the sound collection module may begin collecting sound after receiving a request to collect sound signals. For example, in a video telephony application scenario, it is assumed that the application 103 is a video telephony application. When the user sends a request for establishing a video connection to the friend through the application 103, the friend receives the request for establishing a video connection, and at this time, the application 103 sends a request for acquiring a sound signal to the sound acquisition module 101, where the request is used for acquiring an external sound signal. When the sound collection module receives the request for obtaining the sound signal, the sound collection module starts to collect the sound signal.

In other embodiments, the application 103 need not send a get sound signal request to the sound collection module 101. The electronic device 100 may acquire the sound signal by using an intelligent wearable device (such as a bluetooth watch, a bluetooth headset, etc.), where the intelligent wearable device (such as a bluetooth watch, a bluetooth headset, etc.) establishes a communication connection with the electronic device 100, acquires the sound signal by using the intelligent wearable device (such as a bluetooth watch, a bluetooth headset, etc.) and sends the sound signal to the electronic device 100, and the electronic device 100 acquires the sound signal.

S504, the sound collection module 101 sends the collected sound signal to the abnormal signal detection frame 102.

After acquiring the sound signal, the sound acquisition module 101 sends the sound signal to the abnormal signal detection frame 102. For example, in the video call application scenario, before the sound collection module 101 sends the collected sound signal to the abnormal signal detection frame 102, the abnormal signal detection frame 102 detects the sound signal, and determines whether there is an abnormal high-frequency signal in the sound signal, so as to avoid leakage of user privacy.

S505, the abnormal signal detection framework 102 transmits an acquisition request to the server 200.

The acquisition request is used to request acquisition of a detection model from the server 200, the detection model comprising an average ambient sound energy threshold. Or the detection model comprises each environment identifier and an environment sound energy threshold corresponding to each environment identifier.

The electronic device may perform this step before step S502, or may perform this step after step S504. The execution sequence of the steps is not limited in the embodiment of the present application.

It will be appreciated that this step is an optional step. In some embodiments, the detection model is generated by the electronic device. At this time, after the abnormal signal detection frame acquires the sound signal, whether the sound signal has an abnormal high-frequency signal may be determined according to a detection model acquired in advance or a detection model generated by the electronic device 100.

S506, the server 200 sends a detection model to the abnormal signal detection frame 102, where the detection model includes an average environmental sound energy threshold.

The abnormal signal detection frame 102 receives the detection model.

In some embodiments, the detection model includes environment identifiers and environment sound energy thresholds corresponding to the environment identifiers, and the electronic device 100 may determine a current environment identifier (e.g., a mall) according to the collected sound signal, and determine the environment sound energy threshold from the detection model according to the current environment identifier.

In some embodiments, the electronic device 100 may also obtain first environmental information, which may be location information and/or environmental sounds.

The electronic device 100 transmits the first environment information to the server 200. Based on the current location technology, the server 200 can accurately confirm the current environment identifier of the user according to the current location information of the user, and the current environment identifier can be a mall, a coffee shop, a library, or the like.

It should be noted that, the first environmental information including the location information and/or the environmental sound is only used to explain the present application, and should not be construed as limiting, and in a specific implementation, the first environmental information may include other contents.

In other embodiments, the electronic device 100 may determine a detection model or a corresponding ambient sound energy threshold value to be acquired according to the first ambient information, and then request acquisition of the determined detection model or ambient sound energy threshold value from the server 200 through an acquisition request.

S507, the abnormal signal detection frame 102 performs spectrum analysis on the sound signal to obtain a high-frequency energy value P0 of the sound signal.

1. The abnormal signal detection frame 102 calculates an ambient noise energy value Pm from the ambient noise.

The sound collection module 101 collects a sound signal as ambient noise a period of time before the application 103 responds to the sound acquisition request.

For example, in the first two seconds before the application 103 responds to the acquisition of the sound request, the sound acquisition module 101 uses the acquired sound signal as the environmental noise, and the abnormal signal detection frame 102 calculates the environmental noise high-frequency energy value Pm from the environmental sound.

For example, in a video call application scenario, when a user sends a request for establishing a video connection to a friend through the application 103, the application 103 does not start to collect a sound signal before the friend receives the request for establishing the video connection, at this time, the sound collection module 101 will collect 2 seconds of ambient sound, and calculate an ambient noise energy value from the 2 seconds of ambient sound. After the buddy receives the request to establish a video connection, the application 103 sends the acquired sound signal to the buddy.

Specific:

1. The abnormal signal detection frame 102 acquires a high-frequency band (for example, 18 KHz to 20 KHz) signal of the environmental sound, reads the high-frequency environmental sound signals of n time windows, and calculates full-db high-frequency energy values of the high-frequency environmental sound signals of n time windows respectively and exists in the queue 1, where the calculation formula may be:

As shown in formula (5), pa _i is a full-db high-frequency energy value of the high-frequency environmental sound signal segment in the ith time window, sa _i is an actual high-frequency energy value of the high-frequency environmental sound signal segment in the ith time window, and the high-actual-frequency (e.g., 18KHz-20 KHz) energy value of the environmental sound signal can be obtained by means of short-time fourier transform or the like. Sj represents the maximum value expressed by the number of bits required for storing the high-frequency environmental noise signal segment in the jth time window, namely S _j＝2^j -1, j is a positive integer greater than or equal to 1 and less than or equal to n, and i is a positive integer less than or equal to n.

The abnormal signal detection frame 102 may calculate full-db high-frequency energy values of the high-frequency environmental sound signal segments of the n time windows through the formula (5), respectively.

2. The abnormal signal detection frame 102 obtains the high frequency energy values of the first group of environmental sound signals according to the average high frequency energy of the full db high frequency energy values of the high frequency environmental sound signal segments of the n time windows, and the average energy is stored in the queue 2, and the calculation formula of the average high frequency energy may be:

As shown in formula (6), pu represents the average high-frequency energy of the ambient sound signal, P _ai represents the full-db high-frequency energy value of the high-frequency ambient sound signal segment of the ith time window, and i is a positive integer less than or equal to n.

It is to be understood that equation (6) is merely used to explain the present application and should not be construed as limiting.

The abnormal signal detection frame 102 continues to acquire new environmental noise from the environment, reads sound signals of n time windows according to the new environmental noise, discards existing sound signals in the n time windows, calculates average high frequency energy of the new environmental sound according to the formula (5) and the formula (6), and adds the new environmental sound to the queue 2.

And so on, the average high frequency energy of the w sets of ambient sounds is obtained in the above manner and exists in the queue 2.

The abnormal signal detection frame 102 calculates an arithmetic average value of the average high-frequency energy of the w sets of environmental sounds in the queue 2 as an environmental noise energy value Pm.

3. The abnormal signal detection frame 102 calculates a high frequency energy value P0 under the full decibel scale of the sound signal according to the actual high frequency energy value of the sound signal.

After the application 103 responds to the request for acquiring the sound, the sound acquisition module 101 acquires the sound, the sound acquisition module 101 sends the sound signal to the abnormal signal detection frame 102, and the abnormal signal detection frame 102 calculates a high-frequency energy value P0 under the full decibel scale of the sound signal according to the sound signal, specifically:

Firstly, reading sound signals of n time windows, and calculating high-frequency energy values under full decibel scales of the sound signals of each time window according to actual high-frequency energy values of the sound signals of each time window, wherein the high-frequency energy values exist in a queue 1, and the calculation formula is as follows:

As shown in formula (7), pb _i is the full db high-frequency energy value of the high-frequency signal segment in the ith time window, sb _i is the actual high-frequency energy value of the medium-high-frequency signal segment in the ith time window, and the high-frequency energy value can be obtained by short-time fourier transform or the like. Sj is the number of bits required for the electronic device to store the high frequency signal segment in the ith time window, i.e. S _j＝2^j -1, and the explanation of S _j may refer to the foregoing embodiment, which is not repeated herein.

Secondly, in order to eliminate the high-frequency signal interference of the environmental noise signal caused by the frequency shift phenomenon, the application subtracts the environmental noise energy value Pm from the full db high-frequency energy value of the sound signal calculated according to the formula (7), and the calculation formula can be:

pc _i＝Pb_i -Pm formula (8)

As shown in formula (8), pb _i is the full db high-frequency energy value of the high-frequency signal segment in the ith time window, pp is the difference between the full db high-frequency energy value of the high-frequency signal segment in the ith time window and the ambient noise energy value, and Pm is the ambient noise energy value.

Finally, the average energy value of the difference between the full db high frequency energy value of the sound signal and the high frequency energy value Pm of the ambient noise signal in n time windows is used as the high frequency energy value P0 of the sound signal. The calculation formula can be:

as shown in formula (9), pci is the difference between the full dB high frequency energy value of the high frequency signal section in the ith time window and the environmental noise energy value, i is a positive integer less than or equal to n, and P0 represents the high frequency energy value of the sound signal.

S508, the abnormal signal detection frame 102 determines whether the high-frequency energy value P0 of the sound signal is greater than the average environmental sound energy threshold Pt.

The abnormal signal detection frame 102 compares the high frequency energy value P0 of the sound signal with the average environmental sound energy threshold, and if the high frequency energy value P0 of the sound signal is smaller than the average environmental sound energy threshold, the electronic device 100 continues to acquire a new sound signal from the current environment and calculates the high frequency energy value P0 of the new sound signal. Until the high frequency energy value P0 of the new sound signal is greater than the average ambient sound energy threshold.

If the abnormal signal detection frame 102 does not detect that the high frequency energy value P0 of the new sound signal is greater than the average ambient sound energy threshold, it indicates that no abnormal high frequency signal exists in the current environment of the user.

S509, when the high-frequency energy value P0 of the sound signal is greater than the average environmental sound energy threshold, the abnormal signal detection frame 102 sends an alert prompt to the user.

The abnormal signal detection frame 102 judges that the high frequency energy value P0 of the sound signal is greater than the average environmental sound energy threshold, an abnormal high frequency signal exists in the current environment. The abnormal signal detection frame 102 outputs an alarm prompt to the user.

The alert prompt may be a prompt box displayed on a user interface of the electronic device 100 for prompting a user that an abnormal high frequency signal exists in the current environment.

The alarm prompt may also be a prompt bar displayed on the edge of the display screen of the electronic device 100, where the prompt box may be static or flash, and is used to prompt the user that an abnormal high-frequency signal exists in the current environment, where the prompt bar may receive a click operation from the user, and respond to the click operation from the user, and the user interface of the electronic device 100 displays the prompt box. The effect of displaying the reminder bar is that the current operation of the user, such as the user being engaged in a video call, may not be affected, and the reminder bar does not affect the call quality of the user (e.g., the occurrence of a click).

The alert prompt may also be a voice prompt, such as the electronic device 100 playing an alert sound through a speaker, etc.

The alert prompt may also be a vibration of the electronic device 100.

The alert prompt may also be a status bar display prompt message for the electronic device 100.

The alert prompt may also be a flashlight light flashing of the electronic device 100.

It is to be understood that the alert prompt may be a combination of two or more alert prompt notification modes, which is not limited herein.

In some possible embodiments, when the abnormal signal detection frame 102 detects an abnormal high frequency signal, the electronic device 100 outputs an alarm prompt to the user only once, and when the abnormal signal detection frame 102 detects the abnormal high frequency signal again, the abnormal signal detection frame 102 directly filters out the abnormal high frequency signal in the sound signal, and the electronic device 100 no longer outputs the alarm prompt to the user, so that the current operation of the user is not affected.

S510, the abnormal signal detection frame 102 receives the filtering instruction.

S511, in response to the filtering instruction, the abnormal signal detection frame 102 filters out the abnormal high frequency signal in the sound signal.

Wherein the abnormal high frequency signal refers to a high frequency signal in a specified frequency band (e.g., 18KHz-20 KHz) in the sound signal when the high frequency energy value of the sound signal is greater than the ambient sound energy threshold.

After the user selects to perform filtering of the abnormal high frequency signals in the sound signals, the electronic device 100 generates a filtering instruction according to the operation of the user, the electronic device 100 sends the filtering instruction to the abnormal signal detection frame 102, the abnormal signal detection frame 102 receives and responds to the filtering instruction, and the abnormal signal detection frame 102 filters out the abnormal high frequency signals in the sound signals.

Specifically, after the user selects to perform the filtering operation, the filtering module 1025 in the abnormal signal detecting frame 102 may use a low-pass filter or a band-stop filter to process the sound signal, i.e. filter out the abnormal high-frequency signal in the sound signal.

Here, a band reject filter is described as an example, and the filter has the form:

y_IRR(m)＝b(0)x(m)+b(1)x(m-1)+…+b(p)x(m-p)-a(1)y_IRR(m-1)-a(2)y_IRR(m-2)-…--a(q)y_IRR(m-q) Formula (10)

As shown in formula (10), x (m) represents a sound signal collected at m time, m represents a certain time of the sound signal, p and q represent a certain time in the middle of 0 to m time, a (q) represents a correlation coefficient of a q time filter, and b (p) represents a correlation coefficient of a p time filter.

S512, the abnormal signal detection frame 102 calls the sound collection module 101.

S513, the sound collection module 101 sends the sound signal with the abnormal high frequency signal filtered to the application 103.

The abnormal signal detection frame 102 invokes the sound collection module 101, and the sound collection module 101 transmits the sound signal from which the abnormal high-frequency signal is filtered to the application 103, so that the operation of the user may not be affected.

In the video call application scenario described above, the sound collection module 101 sends the sound signal from which the abnormal high-frequency signal is filtered to the communication module in the electronic device 100, and the communication module is configured to send the sound signal from which the abnormal high-frequency signal is filtered to the receiving device.

In some embodiments, the abnormal signal detection framework 102 does not need to invoke the sound collection module 101, and the abnormal signal detection framework 102 directly sends the sound signal from which the abnormal high frequency signal is filtered to the application 103.

For example, in the video call application scenario described above, the abnormal signal detection framework 102 does not need to call the sound collection module 101, and the abnormal signal detection framework 102 directly sends the sound signal from which the abnormal high frequency signal is filtered to the communication module in the electronic device 100, where the communication module is configured to send the sound signal from which the abnormal high frequency signal is filtered to the receiving device.

Fig. 6 is a flow chart of another filtering method for abnormal signals according to an embodiment of the present application.

S601, the electronic device 100 acquires a detection model.

In some embodiments, the electronic device 100 may obtain the detection model from the server 200.

In other embodiments, the detection model may also be preset in the electronic device 100. Please refer to this as such, and are not described in detail herein.

The electronic device 100 may acquire the detection model before the abnormal high-frequency signal detection function is turned on, or may acquire the detection model after the abnormal high-frequency signal detection function is turned on, which is not limited herein.

In some embodiments, alert application 1024 resides in electronic device 100 in the form of a system application.

The mode of turning on the abnormal high frequency signal detection function in the electronic device 100 may be that the electronic device 100 receives the user operation to turn on the abnormal high frequency signal detection function, or may be that the electronic device 100 is automatically turned on, which is not limited herein.

As shown in fig. 7A to 7C, fig. 7A to 7C are UI diagrams in which the warning application 1024 receives a user operation to turn on the abnormal high frequency signal detection function.

Fig. 7A is a UI diagram of a setup user interface in the electronic device 100 including an alert application 1024.

As shown in fig. 7A, fig. 7A includes a settings user interface 700 of the electronic device 100.

The setup user interface 700 includes one or more functionality switch controls, such as, for example, an airplane mode control 7001, a wireless local area network control 7002, a bluetooth control 7003, an alert application control 7004, a notification control 7005.

Wherein the flight mode control 7001 displays "off", the wireless local area network control 7002 displays "on", the bluetooth control 7003 displays "on", the alert application control 7004 displays "off", and the notification control 7005 displays "on".

It should be noted that the setting user interface 700 may include more or fewer setting controls, and the present application is not limited herein.

The alert application control 7004 may receive a user operation (e.g., a click) and, in response to the user operation (e.g., a click), the electronic device 100 displays an alert application user interface 710 as shown in fig. 7B.

Fig. 7B is a UI diagram of an alert application setting user interface in the electronic device 100.

The alarm application setting user interface may receive a user operation to turn on or off the abnormal high frequency signal detection function, and may also receive a time when the user operation sets the abnormal high frequency signal detection function.

The alert application user interface 710 includes an alert application control 7004, the alert application control 7004 displays "off", when the alert application control 7004 displays "off", the abnormal high frequency signal detection function is turned off, a detection time control 7006 displays "07:00-22:00", that is, the electronic device 100 detects whether the sound signal includes the abnormal high frequency signal between the time periods "07:00-22:00", the rest time period users rest at home, the abnormal high frequency signal detection function is turned off, and the electronic device 100 does not detect, so that consumption can be saved.

Fig. 7C is a UI diagram of the alert application in the electronic device 100 that receives a user operation to turn on the abnormal high frequency signal detection function.

When the alert application control 7004 displays "off", the alert application control 7004 may receive a user operation (e.g., a click) to turn on the abnormal high frequency signal detection function.

When the alert application control 7004 displays "on", the alert application control 7004 may receive a user operation (e.g., a click) to turn off the abnormal high frequency signal detection function.

Fig. 7D to 7F are UI diagrams of the electronic device 100 receiving a user operation setting of the abnormal high frequency signal detection function on time.

As shown in fig. 7D, the detection time control 7006 may receive a user operation (e.g., a click) to set the on time of the abnormal high frequency signal detection function.

In response to the user clicking on the detect time control 7006, the electronic device 100 displays a set detect time user interface 720 as shown in fig. 7E, the user interface 720 including a time selection control 7007, a save control 7008.

The time selection control 7007 may receive a user single-finger up-slide or down-slide operation to select a time to be set by the user, for example, the user sets the detection time to "06:00-21:00", and then the abnormal high-frequency signal detection function is turned on in a time period of "06:00-21:00", that is, the electronic device 100 performs whether the sound signal includes abnormal high-frequency signal detection in a time period of "06:30-21:30".

As shown in FIG. 7F, the time selection control 7007 in the user interface 720 is displayed as "06:30-21:30". Save control 7008 may receive a user operation (e.g., a single click) and, in response to the user (e.g., a single click), electronic device 100 sets the on period of the abnormal high frequency signal detection function to "06:30-21:30".

The electronic device 100 may also automatically turn on the abnormality signal detection function.

When the abnormal signal detection function in the electronic device 100 is turned on, the electronic device 100 detects the collected sound signal, and detects whether an abnormal high-frequency signal exists in the sound signal.

In some embodiments, when the electronic device 100 is in a power-on state, the electronic device 100 turns on the abnormal signal detection function.

In some embodiments, when the electronic device 100 is off screen, the electronic device 100 turns off the anomaly signal detection function.

In some embodiments, the electronic device 100 is in a stationary location (e.g., library) for a long period of time. When the electronic device 100 carried by the user just reaches the place, after the electronic device 100 starts to turn on the abnormal signal detection function for a period of time, the electronic device 100 does not detect an abnormal high-frequency signal in the collected sound signal, and because the electronic device 100 is located at the place (for example, a library) for a long time, the surrounding environment of the place is single, so that the abnormal signal detection function can be turned off after the electronic device 100 is turned on for a period of time, and consumption is saved.

In some embodiments, the electronic device 100 may selectively turn on the anomaly signal detection function based on historical location information of the electronic device 100.

For example, between time periods "8:00-22:00", the electronic device 100 turns on the abnormality signal detection function, because between time periods "22:00-8:00", the user is at rest at home, and the electronic device 100 detects that the historical location information (e.g., the location of the user's home) of the electronic device 100 is substantially the same between time periods "22:00-8:00", so the abnormality signal detection function may be turned off, reducing consumption.

For example, if the electronic device 100 detects that the historical location information (for example, the company location) of the electronic device 100 is substantially the same between the time periods "9:00-17:00", and the environment of the company is relatively single, the electronic device 100 may turn off the abnormal signal detection function.

For example, between the time periods "7:00-9:00" and between the time periods "17:00-19:00" of monday to friday, the electronic device 100 turns on the abnormality signal detection function. The electronic device 100 determines, according to the historical position information, that the position information of the electronic device 100 arrives at the second location from the first location in the time period "7:00-9:00", and that the position information of the electronic device 100 arrives at the first location from the second location in the time period "17:00-19:00", so that it can be determined that the user is on the way to and from work between the two time periods. Because the surrounding environment is complex during the work and the work, there may be a risk of acquiring the privacy of the user due to the hidden communication technology, the electronic device 100 starts the abnormal signal detection function, and prevents the disclosure of the privacy information of the user.

In some embodiments, the electronic device 100 may selectively turn on the anomaly signal detection function according to user behavior habits collected by the electronic device 100.

For example, the electronic device 100 may be a high risk website that may obtain user terminal information and user personal information based on user surfing habits, such as the electronic device 100 detecting that a website that the user is frequently logged in to. When a user accesses such a website through the electronic device 100, the electronic device 100 turns on the abnormal signal detection function, avoiding such a website from acquiring privacy information of the user.

In some embodiments, the server may send an abnormal signal detection function on notification to the electronic device 100, in response to which the electronic device 100 turns on the abnormal signal detection function.

The server may collect information reported by a plurality of electronic devices, and when one or more electronic devices of the plurality of electronic devices detect special information in a specific area (for example, a mall or a supermarket), the one or more electronic devices report that a special signal exists in the specific area to the server, and the server gathers the information reported by the one or more electronic devices.

Before the user reaches this specific area, the server may send an abnormality signal detection function activation notification to the electronic device 100 carried by the user, which may be used to notify the electronic device 100 to activate the abnormality signal detection function when a certain distance (e.g., 100 meters) from this specific area.

Or when the user reaches the specific area, the server sends an abnormal signal detection function starting notification to the electronic device 100 carried by the user, and the electronic device 100 carried by the user responds to the notification, so that the electronic device 100 starts the abnormal signal detection function.

In other embodiments, the server may send an advertisement from an advertisement server in a specific area (e.g., a mall supermarket), and when the server 200 detects that special information exists in the advertisement, the server 200 sends an abnormality signal detection function on notification to the user's electronic device, where the notification may be used to notify the electronic device that the abnormality signal detection function is on at a distance (e.g., 100 meters) from the specific area or when the user arrives at the specific area.

In this way, a third party can be prevented from acquiring the user's private information without the user's knowledge or authorization.

S602, the electronic device 100 acquires a sound signal.

The electronic device 100 may receive a user operation to acquire a sound signal through the sound collection module.

The user operation may be a voice instruction. For example, the voice command may be the wake word "small E", which the electronic device 100 receives and responds to the wake word "small E", which triggers the voice assistant to turn on, and in response to the voice assistant turning on, the electronic device 100 begins to collect the sound signal.

The user operation may also be an operation that the user triggers the social application to talk with the friend, and the electronic device 100 responds to the user operation, and the electronic device 100 obtains the sound signal through the sound collecting module and sends the sound signal to the electronic device on the friend side.

In some embodiments, the electronic device 100 may also acquire the sound signal by an intelligent wearable device (e.g., a bluetooth watch, a bluetooth headset, etc.), the intelligent wearable device (e.g., a bluetooth watch, a bluetooth headset, etc.) establishes a communication connection with the electronic device 100, the intelligent wearable device (e.g., a bluetooth watch, a bluetooth headset, etc.) acquires the sound signal and sends the sound signal to the electronic device 100, and the electronic device 100 acquires the sound signal.

S603, the electronic device 100 determines whether an abnormal high-frequency signal exists in the sound signal according to the detection model.

After the electronic device 100 confirms the high-frequency energy value P0 of the sound signal according to the acquired sound signal, the electronic device 100 inputs the high-frequency energy value P0 of the sound signal into the detection model, and if the electronic device 100 determines that the high-frequency energy value P0 of the sound signal is greater than the average environmental sound energy threshold, an abnormal high-frequency signal exists in the sound signal.

Or after the electronic device 100 confirms the high-frequency energy value P0 of the sound signal according to the obtained sound signal, the electronic device 100 inputs the high-frequency energy value of the sound signal into the detection model, the electronic device 100 confirms the environmental sound energy threshold corresponding to the current environmental identifier from the detection model according to the sound signal, and if the electronic device 100 judges that the high-frequency energy value P0 of the sound signal is greater than the environmental sound energy threshold corresponding to the current environmental identifier, an abnormal high-frequency signal exists in the sound signal.

Here, the electronic device 100 may refer to the embodiment described in S507 in fig. 5 for identifying the high-frequency energy value P0 of the sound signal according to the acquired sound signal, and the present application will not be repeated.

The electronic device 100 may refer to the embodiment described in S501 in fig. 5 for determining whether the abnormal high-frequency signal exists in the sound signal according to the detection model, and the present application is not repeated here.

If the electronic device 100 determines that the abnormal high-frequency signal exists in the sound signal, the electronic device 100 may send an alarm prompt to the user.

Here, the alert prompt may refer to the embodiment described in S509 in the embodiment of fig. 5, and is not described herein.

For example, when a user is in a mall, music is being played in the mall, the user performs a video call with a friend, the electronic device 100 collects a sound signal through the sound collection module, and if hidden communication information (such as an abnormal high-frequency signal) exists in the music played in the mall, the hidden communication information (such as the abnormal high-frequency signal) can steal privacy information of the user existing in the electronic device 100. Hidden communication information (e.g., abnormally high frequency signals) in the sound signal may be detected and filtered out in the electronic device 100.

Fig. 8A-8C are UI diagrams of the electronic device 100 displaying alert prompts.

Fig. 8A includes a UI diagram of a user interface 801 for a user to engage in a video call with a buddy.

The electronic device 100 collects the user image and the sound signal and sends the user image and the sound signal to the user friend terminal device, and meanwhile, the electronic device 100 receives and displays the user friend image and the sound signal collected by the user friend terminal device.

The user interface 801 includes a first screen 8001, a second screen 8002, a shooting direction control before and after transition 8003, a call ending control 8004, and a mute control 8005.

The first screen 8001 is a user image collected by the electronic device 100.

The second screen 8002 is an image of the user's friends received and displayed by the electronic device 100.

The flip shooting direction control 8003 may receive a user operation (e.g., a click), the electronic device 100 converting the shooting direction to the rear if the current shooting direction of the electronic device 100 is the front (e.g., the self-shooting direction), and the electronic device 100 converting the shooting direction to the front (e.g., the self-shooting direction) if the current shooting direction of the electronic device 100 is the rear.

The end call control 8004 may receive a user operation (e.g., a single click) and the electronic device 100 ends the current video call.

Mute control 8005 may receive a user action (e.g., a click) that electronic device 100 is no longer sending sound signals collected by electronic device 100 to the user's friends.

In one possible implementation, when the electronic device 100 detects that there is hidden communication information (e.g., an abnormally high frequency signal) in the current environment, as shown in fig. 8B, a user interface 801 of the electronic device 100 displays a prompt box 8006, where the prompt box 8006 is used to prompt the user that there is hidden communication information (e.g., an abnormally high frequency signal) in the current environment, and the prompt box 8006 includes a prompt message of "detect abnormally high frequency signal, is to be filtered out.

Wherein the "yes" control 8007 may receive a user operation (e.g., a single click), in response to which the anomaly signal detection framework 102 will filter out hidden communication information (e.g., an anomaly high frequency signal) in the sound signal.

The no control 8008 may receive a user operation (e.g., a click), and in response to the user operation (e.g., a click), the abnormal signal detection frame 102 does not filter out hidden communication information (e.g., abnormal high frequency signals) in the sound signal.

In another possible implementation, as shown in FIG. 8C, the user interface 801 displays a reminder bar 8009, the reminder bar 8009 being used to remind the user that hidden communication information (e.g., an abnormally high frequency signal) is present in the current environment. This may not affect the current operation of the user.

Wherein the reminder bar 8009 may receive a user operation (e.g., a single click), in response to which the user interface 801 displays a reminder box 8006 as shown in FIG. 8B.

S604, the electronic device 100 filters out abnormal high-frequency signals in the sound signals according to the user confirmation filtering operation.

In some embodiments, after the electronic device 100 detects the presence of an abnormal high frequency signal in the sound signal, the user interface of the electronic device 100 may display an alert prompt that may receive a user operation (e.g., a click), and in response to the user operation, the electronic device 100 filters out the abnormal high frequency signal in the sound signal.

In some embodiments, when the electronic device 100 detects an abnormally high frequency signal, the electronic device 100 directly filters out the abnormally high frequency signal in the sound signal, and the electronic device 100 does not output an alert prompt to the user.

Fig. 8D is a UI diagram of the electronic device 100 displaying the prompt information of "abnormal high frequency signal has been filtered out".

In response to the user clicking the "yes" control 8007 shown in fig. 8B, as shown in fig. 8D, the user interface 801 of the electronic device 100 displays a prompt box 8010, where the prompt box 8010 is used to prompt the user that the electronic device 100 has filtered out the abnormal high frequency signals in the sound signals, and the prompt box 8010 includes a prompt message of "the abnormal high frequency signals have been filtered out".

In other possible embodiments, the user is driving, the user interface of the electronic device 100 is displaying a navigation route, the bluetooth headset has established a communication connection with the electronic device 100, and when the electronic device 100 detects that an abnormal high frequency signal exists nearby, the user interface of the electronic device 100 does not display an alarm prompt, so that the navigation route of the user is not affected, but a voice prompt message is played through the bluetooth headset to prompt the user that an abnormal high frequency signal exists nearby, and the user can inform the electronic device 100 to filter the abnormal high frequency signal in a voice input mode.

In other possible embodiments, the bluetooth watch has established a communication connection with the electronic device 100, and when the electronic device 100 detects that an abnormal high frequency signal exists nearby, the electronic device 100 does not display a warning message, and prompts the abnormal high frequency signal through the bluetooth watch, where the prompting mode may be a mode of watch vibration, voice prompt, etc., and the user informs the electronic device 100 to filter out the abnormal high frequency through a voice input mode. In other possible embodiments, after the user filters out the abnormal high frequency signal detected by the electronic device 100, the user may report the abnormal high frequency signal to the server 200, and the server receives the abnormal high frequency signal information uploaded by the electronic device 100. The server 200 counts the abnormal high frequency signal information uploaded by each user in each region, counts the probability of the abnormal high frequency signal information existing in each region, and informs map manufacturers of regions with high probability of the abnormal high frequency signal information, when a user plans a route by using the map, the navigation route displayed by the map can avoid the user passing through the regions with high probability of the abnormal high frequency signal information, or the map prompts the user to ' exist an abnormal high frequency signal nearby ' through voice when the user approaches the regions with high probability of the abnormal high frequency signal information, personal information leakage is easy to cause, and the user is required to bypass ', so that personal privacy information of the user can be prevented from being leaked.

It should be noted that the above embodiments may be combined with the embodiments shown in fig. 3 to 5.

Referring to fig. 9, fig. 9 is a schematic diagram of an apparatus according to an embodiment of the present application. The apparatus 910 includes a sound collector 9001, a processor 9002. Wherein:

The sound collector 9001 may be used to collect sound signals and transmit the sound signals to the application of the electronic device 100.

The processor 9002 is configured to process the sound signal, and specifically includes:

1. the processor 9002 identifies an ambient sound energy threshold from the sound signal.

2. The processor 9002 determines a high frequency energy value of the sound signal from the sound signal.

3. The processor 9002 determines a high frequency energy value of the sound signal and an ambient sound energy threshold, and if the high frequency energy value of the sound signal is greater than the current ambient sound energy threshold, determines that an abnormal high frequency signal exists in the sound signal.

4. The processor 9002 sends alert information to the user.

5. The processor 9002 filters out an abnormal high-frequency signal in the sound signal according to the filtering operation by the user.

6. The processor 9002 invokes the sound collector 9001, and the sound collector 9001 transmits a sound signal from which an abnormal high-frequency signal is filtered out to the application of the electronic apparatus 100.

The sound collector 9001 may perform the steps performed by the sound collecting module, and specific reference may be made to the foregoing embodiments, which are not described herein.

The processor 9002 may perform the steps performed by the abnormal signal detection framework, and specific reference may be made to the foregoing embodiments, which are not described herein.

While the application has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that the foregoing embodiments may be modified or equivalents may be substituted for some of the features thereof, and that the modifications or substitutions do not depart from the spirit of the embodiments.

Claims

1. A method for filtering an anomaly signal, comprising:

The electronic equipment receives a sound signal acquisition request;

The electronic equipment acquires a sound signal in response to the sound signal acquisition request;

the electronic equipment acquires environmental sound;

The electronic equipment determines an environmental noise energy value according to the environmental sound;

The electronic equipment determines the high-frequency energy value of the sound signal according to the actual high-frequency energy value of the sound signal and the environmental noise energy value;

If the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold value, the electronic equipment sends prompt information to a user, and the prompt information is used for prompting that the sound signal has abnormal high-frequency signals.

2. The method of claim 1, wherein after the electronic device issues the alert message to the user, the method further comprises:

The electronic equipment receives filtering operation of a user;

and responding to the filtering operation, and filtering the abnormal high-frequency signals by the electronic equipment.

3. The method according to claim 2, wherein the method further comprises:

And the electronic equipment transmits the sound signal with the abnormal high-frequency signal filtered to the receiving equipment.

4. A method according to any one of claims 1 to 3, further comprising:

the electronic equipment acquires first environment information, wherein the first environment information comprises position information and/or environment sound;

the electronic device determines the ambient sound energy threshold from the first ambient information.

5. The method according to claim 1, wherein the electronic device determines the high frequency energy value of the sound signal based on the actual high frequency energy value of the sound signal and the ambient noise energy value, in particular comprising:

The electronic equipment acquires a high-frequency signal in the sound signal;

The electronic equipment divides the high-frequency signal into high-frequency signal segments of n time windows, wherein n is a positive integer;

the electronic equipment calculates actual high-frequency energy values of high-frequency signal segments in the n time windows;

The electronic equipment calculates full-decibel high-frequency energy values of the high-frequency signal segments in the n time windows according to the actual high-frequency energy values of the high-frequency signal segments in the n time windows;

And the electronic equipment determines the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy value of the high-frequency signal section in the n time windows and the environmental noise energy value.

6. The method according to claim 5, wherein the electronic device calculates the full db high frequency energy value of the high frequency signal segment in the n time windows according to the actual high frequency energy value of the high frequency signal segment in the n time windows, specifically comprising:

The electronic equipment determines the full-decibel high-frequency energy value of the high-frequency signal section in the n time windows through the following formula:

Wherein Pb _i is a full db high-frequency energy value of the high-frequency signal segment in the ith time window, sb _i is an actual high-frequency energy value of the high-frequency signal segment in the ith time window, sj is a number of bits required for storing the high-frequency signal segment in the jth time window by the electronic device, j is a positive integer greater than or equal to 1 and less than or equal to n, and i is a positive integer less than or equal to n.

7. The method according to claim 5, wherein the electronic device determines the high frequency energy value of the sound signal according to the full db high frequency energy value of the high frequency signal segment in the n time windows and the ambient noise energy value, specifically comprising:

the electronic device determines the high frequency energy value of the sound signal by the following formula:

Pc_i＝Pb_i–Pm;

8. A method according to any one of claims 1 to 3, wherein the frequency range of the abnormally high frequency signal includes 18KHz-20KHz.

9. An anomaly signal filtering apparatus, characterized by one or more processors, one or more memories, a sound collector, the one or more memories, the sound collector being coupled to the one or more processors, the one or more memories for storing computer program code comprising computer instructions that one or more processors invoke to cause the apparatus to perform the method of any of claims 1-8.

10. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 8.