CN113782058B - Dynamic audio perception tracking system and method, storage medium, and earphone - Google Patents
Dynamic audio perception tracking system and method, storage medium, and earphone Download PDFInfo
- Publication number
- CN113782058B CN113782058B CN202111082292.4A CN202111082292A CN113782058B CN 113782058 B CN113782058 B CN 113782058B CN 202111082292 A CN202111082292 A CN 202111082292A CN 113782058 B CN113782058 B CN 113782058B
- Authority
- CN
- China
- Prior art keywords
- audio
- module
- audio signal
- perception
- dynamic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000008447 perception Effects 0.000 title claims abstract description 43
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000005236 sound signal Effects 0.000 claims abstract description 120
- 210000004556 brain Anatomy 0.000 claims abstract description 64
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 210000005069 ears Anatomy 0.000 claims abstract description 12
- 238000001914 filtration Methods 0.000 claims abstract description 9
- 230000003139 buffering effect Effects 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 5
- 230000000875 corresponding effect Effects 0.000 description 22
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 208000016354 hearing loss disease Diseases 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/377—Electroencephalography [EEG] using evoked responses
- A61B5/38—Acoustic or auditory stimuli
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/6802—Sensor mounted on worn items
- A61B5/6803—Head-worn items, e.g. helmets, masks, headphones or goggles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1091—Details not provided for in groups H04R1/1008 - H04R1/1083
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
- H04R2201/10—Details of earpieces, attachments therefor, earphones or monophonic headphones covered by H04R1/10 but not provided for in any of its subgroups
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Acoustics & Sound (AREA)
- Radar, Positioning & Navigation (AREA)
- Signal Processing (AREA)
- Remote Sensing (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Surgery (AREA)
- Psychology (AREA)
- Psychiatry (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a dynamic audio perception tracking system and method, a storage medium and an earphone, wherein the dynamic audio perception tracking system comprises an audio perception module, a brain wave detection module, an audio analysis module and an audio tracking module; the audio perception module is used for acquiring first audio signals corresponding to different sound sources in the environment, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears; the brain wave detection module is used for detecting brain wave changes of the human brain for each first audio signal so as to acquire a second audio signal of interest of the human brain; the audio analysis module is used for slicing and extracting the characteristics of the waveforms of the second audio signals, marking the waveforms, and judging the priority level of each second audio signal; the audio tracking module is used for respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level. According to the dynamic audio perception tracking system provided by the invention, the interesting sound can be positioned and tracked.
Description
Technical Field
The invention relates to the technical field of sound source localization, in particular to a dynamic audio perception tracking system and method, a storage medium and an earphone.
Background
When a user wears the earphone, whether the user is listening to music or not, the earphone can shield some environmental sound sources; therefore, when the user encounters an emergency or is interested in a certain sound source, he or she has to pick up the earphone to carefully listen and judge, which causes the situation that he or she carelessly drops the earphone to the ground or loses the earphone. In addition, due to uncertainty of sound source direction, sound source type, sound source distance, perception priority level of sound source, etc., the judgment capability of users wearing headphones on emergency situations can be affected, especially for users with hearing impairment or weaker hearing, when the headphones are used, information useful for the users in the environment sound source can be easily ignored, stations may be missed, important instructions may be missed, alarm sounds may be ignored, etc.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, the invention provides a dynamic audio perception tracking system which can detect sounds in the external environment in real time and accurately judge the distance, direction, type, importance degree and the like of the sounds under the condition that a user wears a headset.
The invention also provides a dynamic audio perception tracking method.
The invention also proposes a computer readable storage medium.
The invention also provides an earphone.
In a first aspect, a dynamic audio perception tracking system according to an embodiment of the present invention includes an audio perception module, a brain wave detection module, an audio analysis module, and an audio tracking module; the audio perception module is used for collecting first audio signals corresponding to different sound sources in the environment respectively, filtering and amplifying the first audio signals and then transmitting the first audio signals to ears; the brain wave detection module is used for detecting brain wave changes of the human brain for each first audio signal so as to acquire a second audio signal of interest of the human brain; the audio analysis module is used for slicing and extracting the characteristics of the waveforms of the second audio signals, marking the waveforms, and judging the priority level of each second audio signal; the audio tracking module is used for respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.
The dynamic audio perception tracking system according to the embodiment of the invention has at least the following beneficial effects: the method can help users wearing headphones to listen to music, detect sounds in the external environment in real time, accurately judge the distance, direction, type, importance degree and the like of the sounds, thereby helping the users to dynamically locate and track sound sources of interest, and helping the users to accurately judge the content of the sounds and take corresponding actions, and avoiding missing important information.
According to some embodiments of the invention, the audio buffer module is configured to store the first audio signal and transmit the first audio signal with the frequency of occurrence higher than a threshold value to the audio analysis module.
According to some embodiments of the invention, the audio sensing module comprises a first audio signal filter, an audio signal collector and an audio signal amplifier electrically connected in sequence.
According to some embodiments of the invention, the brain wave detection module comprises a detection electrode, a current timer, an brain wave amplifier, an brain wave frequency counter, a controller and a second audio signal filter which are electrically connected in sequence.
According to some embodiments of the invention, the audio tracking module comprises a six-axis inertial sensor.
In a second aspect, a dynamic audio perception tracking method according to an embodiment of the present invention includes the steps of: collecting first audio signals corresponding to different sound sources in the environment, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears; detecting a change in brain waves of the human brain for each of the first audio signals to obtain a second audio signal of interest to the human brain; slicing and feature extracting are carried out on the second audio signals, marking is carried out, and the priority level of each second audio signal is judged; and respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.
The dynamic audio perception tracking method provided by the embodiment of the invention has at least the following beneficial effects: the method can help users wearing headphones to listen to music, monitor sounds in the external environment in real time, accurately judge the distance, direction, type and importance degree of the sounds, thereby helping the users to dynamically locate and track sound sources of interest, and helping the users to accurately judge the content of the sounds and take corresponding actions, and avoiding missing important information.
According to some embodiments of the invention, the method further comprises the steps of: the audio perception module transmits the first audio signal to an audio buffer module; and if the frequency of the first audio signal exceeds a threshold value, the audio buffer module transmits the first audio signal to an audio analysis module.
In a third aspect, an earphone according to an embodiment of the present invention includes a dynamic audio perception tracking system according to an embodiment of the first aspect of the present invention.
The earphone according to the embodiment of the invention has at least the following beneficial effects: by adopting the dynamic audio perception tracking system, the sounds in the external environment can be monitored in real time under the condition that the user wears the earphone to listen to music, and the distances, the directions, the types and the importance degrees of the sounds are accurately judged, so that the user is helped to dynamically locate and track the sound sources of interest, and the user is helped to accurately judge the contents of the sounds and take corresponding actions, and important information is prevented from being missed.
In a fourth aspect, a computer readable storage medium according to an embodiment of the present invention has stored thereon computer instructions that, when executed by a processor, implement a dynamic audio perception tracking method according to an embodiment of the second aspect of the present invention.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a block diagram of a dynamic audio perception tracking system in accordance with an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an electroencephalogram detection module according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating steps of a dynamic audio perception tracking method according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the accompanying drawings are used to supplement the description of the written description so that one can intuitively and intuitively understand each technical feature and overall technical scheme of the present invention, but not to limit the scope of the present invention.
In the description of the present invention, it should be understood that references to orientation descriptions such as upper, lower, front, rear, left, right, etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description of the present invention and to simplify the description, and do not indicate or imply that the apparatus or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present invention.
In the description of the present invention, a number means one or more, a number means two or more, and greater than, less than, exceeding, etc. are understood to not include the present number, and above, below, within, etc. are understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present invention can be reasonably determined by a person skilled in the art in combination with the specific contents of the technical scheme.
In a first aspect, as shown in fig. 1, a dynamic audio perception tracking system according to an embodiment of the present invention includes an audio perception module, an brain wave detection module, an audio analysis module, and an audio tracking module; the audio perception module is used for acquiring first audio signals corresponding to different sound sources in the environment, filtering and amplifying the first audio signals and transmitting the first audio signals to ears; the brain wave detection module is used for detecting brain wave changes of the human brain for each first audio signal so as to acquire a second audio signal of interest of the human brain; the audio analysis module is used for slicing and extracting the characteristics of the waveforms of the second audio signals, marking the waveforms, and judging the priority level of each second audio signal; the audio tracking module is used for respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.
Specifically, the audio perception module comprises a first audio signal filter, an audio signal collector and an audio signal amplifier which are electrically connected in sequence. The first audio signal filter is used for filtering out some sound sources far away from the user, the audio signal collector is used for collecting some sound sources close to the user (such as talking sound, whistling sound, broadcasting sound, prompting sound, alarming sound and the like of people close to the user), and the first audio signals corresponding to the sound sources are amplified through the audio signal amplifier and transmitted to ears, and are transmitted to the brain through the ears. That is, the sound source sensing module is configured to sense sound sources from different directions in the environment, filter sound with a longer distance, collect sound sources with a shorter distance, amplify first audio signals corresponding to the sound sources, and transmit the amplified first audio signals to ears.
As shown in fig. 2, in some embodiments of the present invention, the brain wave detection module includes a detection electrode, a current timer, an brain signal amplifier, an brain frequency counter, a controller, and a second audio signal filter electrically connected in sequence. The detection electrode is used for acquiring brain waves of the human brain, the current timer and the brain electric frequency counter are used for acquiring the time domain and the frequency domain of the brain waves, the brain electric signal amplifier is used for amplifying the acquired brain wave signals, and the detection electrode, the current timer, the brain electric signal amplifier and the brain electric frequency counter are used for detecting the brain wave changes in real time and transmitting the brain wave changes to the controller; when the user is interested in the sound heard by the user, the brain wave can generate larger fluctuation due to stress reaction, so that the user can know which audio is interested in by detecting the change condition of the brain wave; the controller then invokes the second audio signal filter to filter out audio bands of no interest to the user, and only retains the audio signals of interest (i.e., the second audio signals) and transmits these second audio signals to the audio analysis module. The controller may be a common processor such as an MCU.
The audio analysis module performs slicing and feature extraction on waveforms of the second audio signals after receiving the second audio signals, and marks the signals so as to respectively cluster the second audio signals by using a corresponding self-learning algorithm; meanwhile, the audio analysis module can judge the priority level of the second audio signals according to the interested degree of the user and send the second audio signals to the audio tracking module.
The audio tracking module can adopt an MEMS six-axis inertial sensor, and the sensor integrates functional parts such as an MEMS accelerometer, a gyroscope, a pressure sensor, a magnetic sensor, a signal processing circuit and the like into a silicon chip by utilizing a three-dimensional heterogeneous integration technology, and embeds an algorithm to realize chip-level guidance, navigation, positioning and other functions. The audio tracking module responds to physical movement of a user by using the MEMS six-axis inertial sensor, takes a sound source of interest to the user as a fixed reference object, converts the reaction of linear displacement and angular speed rotation of the user into an electric signal, and realizes the function of space dynamic positioning and tracking of the sound source of interest.
Therefore, according to the dynamic audio perception tracking system provided by the embodiment of the invention, users wearing headphones for listening to music can be helped, sounds in the external environment can be detected in real time, and the distances, types, directions and importance degrees of the sounds can be accurately judged, so that the users can be helped to dynamically position and track sound sources of interest, accurately judge the contents of the sounds and take corresponding actions, and important information is prevented from being missed.
As shown in fig. 1, in some embodiments of the present invention, the audio buffer module is further configured to buffer the first audio signal, and transmit the first audio signal with the frequency of occurrence higher than the threshold value to the audio analysis module. The value of the threshold may be customized.
Specifically, after the audio sensing module collects the sound sources in the environment, the audio sensing module sends the first audio signals corresponding to the sound sources to the audio buffering module so that the audio buffering module can judge whether some sounds are frequently appeared. And if the audio buffer module finds that the occurrence frequency of a certain first audio signal is higher, the first audio signal is sent to the audio analysis module for analysis. This is because, when a certain sound frequently occurs, the brain is fatigued against the sound, and brain waves do not fluctuate significantly; therefore, the sound needs to be identified by the audio buffer module so as to be convenient for the audio tracking module to locate and track the sound. If the brain wave detection module does not detect brain wave fluctuation, and the audio buffer module does not find audio frequency of high frequency, the audio frequency sensing module enters a standby state, and consumption of a power supply is saved.
In a second aspect, as shown in fig. 3, a dynamic audio perception tracking method according to an embodiment of the present invention includes the steps of:
s100: and collecting first audio signals respectively corresponding to different sound sources in the environment, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears.
Specifically, in practical application, a first audio signal filter in an audio sensing module is used for filtering out some sound sources far away, then an audio signal collector is used for collecting some sound sources close to a user (such as talking sound, whistling sound, broadcasting sound, prompt sound, alarm sound and the like of people close to the user), and then an audio signal amplifier is used for amplifying first audio signals corresponding to the sound sources, transmitting the first audio signals to ears and transmitting the first audio signals to the brain through human ears.
S200: the brain wave variation of the human brain for each first audio signal is detected to obtain a second audio signal of interest to the human brain.
Specifically, in practical application, the change condition of brain waves is detected in real time through a detection electrode, a current timer, an electroencephalogram signal amplifier and an electroencephalogram frequency counter in the brain wave detection module, and the reaction condition of the brain waves to each first audio signal is obtained. When detecting that the brain wave has larger fluctuation change, the brain is interested in the first audio signal, at the moment, the controller can mobilize the second audio signal filter to filter out the audio wave bands which are not interested by the user, only the interested audio signals (namely the second audio signals) are reserved, and the second audio signals are transmitted to the audio analysis module.
S300: slicing and feature extraction are performed on the second audio signals, marking is performed, and the priority level of each second audio signal is judged.
Specifically, after receiving the second audio signals, the audio analysis module performs slicing and feature extraction on waveforms of the second audio signals, marks the signals, and clusters the second audio signals respectively by using a corresponding self-learning algorithm; meanwhile, the audio analysis module can judge the priority level of the second audio signals according to the interested degree of the user, so that the judgment of which sounds are more important to the user is carried out, and the second audio signals are sent to the audio tracking module.
S400: and respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.
And the audio tracking module adopts an MEMS six-axis inertial sensor to position and track the sound source of interest to the user. In the process of positioning and tracking, a sound source of interest of a user is used as a fixed reference, the MEMS six-axis inertial sensor reacts to physical movement of the user, and the reaction of linear displacement and angular speed rotation of the user is converted into an electric signal, so that the function of space dynamic positioning and tracking of the sound source of interest is realized.
According to the dynamic audio perception tracking method provided by the embodiment of the invention, users wearing headphones to listen to music can be helped, sounds in the external environment can be monitored in real time, and the distances, directions, types and importance degrees of the sounds can be accurately judged, so that the users can be helped to dynamically locate and track sound sources of interest, and the users can be helped to accurately judge the contents of the sounds and take corresponding actions, so that important information is prevented from being missed.
In some embodiments of the present invention, the above-mentioned dynamic audio perception tracking method further includes the following steps: the audio perception module transmits the acquired first audio signal to the audio buffer module, and if the occurrence frequency of the first audio signal exceeds a threshold value, the first audio signal is transmitted to the audio analysis module. The value of the threshold may be customized.
Specifically, after the audio sensing module collects the sound sources in the environment, the audio sensing module sends the first audio signals corresponding to the sound sources to the audio buffering module so that the audio buffering module can judge whether some sounds are frequently appeared. And if the audio buffer module finds that the occurrence frequency of a certain sound is higher, sending a first audio signal corresponding to the sound to the audio analysis module. This is because, when a certain sound frequently occurs, the brain is fatigued against the sound, and brain waves do not fluctuate significantly; therefore, the sound needs to be identified by the audio buffer module so as to be convenient for the audio tracking module to locate and track the sound. If the brain wave detection module does not detect brain wave fluctuation, and the audio buffer module does not find high-frequency sound, the audio sensing module enters a standby state, and power consumption is saved.
According to the earphone, the dynamic audio tracking system is adopted, sounds in the external environment can be monitored in real time under the condition that the user wears the earphone to listen to music, and the distances, directions, types and importance degrees of the sounds are accurately judged, so that the user is helped to dynamically locate and track sound sources of interest, accurately judge the content of the sounds and take corresponding actions, and important information is prevented from being missed.
In a fourth aspect, it should be appreciated that the steps of the dynamic audio perceptual tracking method described above in an embodiment of the present invention may be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may use standard programming techniques. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes (or variations and/or combinations thereof) described herein may be performed under control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications), by hardware, or combinations thereof, collectively executing on one or more processors. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable computing platform, including, but not limited to, a personal computer, mini-computer, mainframe, workstation, network or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and so forth. Aspects of the invention may be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and/or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, which when read by a computer, is operable to configure and operate the computer to perform the processes described herein. Further, the machine readable code, or portions thereof, may be transmitted over a wired or wireless network. When such media includes instructions or programs that, in conjunction with a microprocessor or other data processor, implement the steps described above, the invention described herein includes these and other different types of non-transitory computer-readable storage media. The invention also includes the computer itself when programmed according to the methods and techniques of the present invention.
The computer program can be applied to the input data to perform the functions described herein, thereby converting the input data to generate output data that is stored to the non-volatile memory. The output information may also be applied to one or more output devices such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on a display.
In the description of the present specification, a description referring to the terms "one embodiment," "further embodiment," "some specific embodiments," or "some examples," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
Claims (9)
1. A dynamic audio perception tracking system, comprising:
the audio perception module is used for acquiring first audio signals corresponding to different sound sources in the environment, filtering and amplifying the first audio signals and then transmitting the first audio signals to ears;
the brain wave detection module is used for detecting brain wave changes of the human brain for each first audio signal so as to acquire a second audio signal of interest of the human brain;
the audio analysis module is used for slicing and extracting features of the waveforms of the second audio signals, marking the waveforms, and judging the priority level of each second audio signal;
and the audio tracking module is used for respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.
2. The dynamic audio perception tracking system of claim 1, further comprising an audio buffering module for storing the first audio signal and transmitting the first audio signal with a frequency of occurrence above a threshold to the audio analysis module.
3. The dynamic audio perception tracking system of claim 1 or 2, wherein the audio perception module comprises a first audio signal filter, an audio signal collector, and an audio signal amplifier electrically connected in sequence.
4. The dynamic audio-aware tracking system of claim 1 or 2, wherein the brain wave detection module comprises a detection electrode, a current timer, an brain signal amplifier, an brain frequency counter, a controller, and a second audio signal filter electrically connected in sequence.
5. The dynamic audio-aware tracking system of claim 1 or 2, wherein the audio tracking module comprises a six-axis inertial sensor.
6. A method of dynamic audio perception tracking based on a dynamic audio perception tracking system as claimed in any one of claims 1 to 5, comprising the steps of:
collecting first audio signals corresponding to different sound sources in the environment, filtering and amplifying the first audio signals, and transmitting the first audio signals to ears;
detecting a change in brain waves of the human brain for each of the first audio signals to obtain a second audio signal of interest to the human brain;
slicing and feature extracting are carried out on the second audio signals, marking is carried out, and the priority level of each second audio signal is judged;
and respectively carrying out positioning tracking on the sound source corresponding to each second audio signal according to the priority level.
7. The method of dynamic audio perception tracking according to claim 6, further comprising the steps of:
the audio perception module transmits the first audio signal to an audio buffer module;
and if the frequency of the first audio signal exceeds a threshold value, the audio buffer module transmits the first audio signal to an audio analysis module.
8. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement a dynamic audio perception tracking method as claimed in any one of claims 6 to 7.
9. A headset comprising a dynamic audio perception tracking system as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082292.4A CN113782058B (en) | 2021-09-15 | 2021-09-15 | Dynamic audio perception tracking system and method, storage medium, and earphone |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111082292.4A CN113782058B (en) | 2021-09-15 | 2021-09-15 | Dynamic audio perception tracking system and method, storage medium, and earphone |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113782058A CN113782058A (en) | 2021-12-10 |
| CN113782058B true CN113782058B (en) | 2023-07-18 |
Family
ID=78844197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111082292.4A Active CN113782058B (en) | 2021-09-15 | 2021-09-15 | Dynamic audio perception tracking system and method, storage medium, and earphone |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113782058B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682761A (en) * | 2011-03-12 | 2012-09-19 | 谢津 | Personalized system and device for sound processing |
| CN105534521A (en) * | 2016-01-28 | 2016-05-04 | 南京邮电大学 | Electroencephalogram real-time monitoring control system and method based on positioning |
| CN107864440A (en) * | 2016-07-08 | 2018-03-30 | 奥迪康有限公司 | Hearing assistance system including EEG records and analysis system |
| CN111671399A (en) * | 2020-06-18 | 2020-09-18 | 清华大学 | Method, apparatus and electronic equipment for measuring noise perception intensity |
| CN112353391A (en) * | 2020-10-22 | 2021-02-12 | 武汉理工大学 | Electroencephalogram signal-based method and device for recognizing sound quality in automobile |
| CN113069125A (en) * | 2021-03-18 | 2021-07-06 | 上海趣立信息科技有限公司 | Head-mounted equipment control system, method and medium based on brain wave and eye movement tracking |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9380374B2 (en) * | 2014-01-17 | 2016-06-28 | Okappi, Inc. | Hearing assistance systems configured to detect and provide protection to the user from harmful conditions |
| US10536786B1 (en) * | 2018-06-27 | 2020-01-14 | Google Llc | Augmented environmental awareness system |
-
2021
- 2021-09-15 CN CN202111082292.4A patent/CN113782058B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682761A (en) * | 2011-03-12 | 2012-09-19 | 谢津 | Personalized system and device for sound processing |
| CN105534521A (en) * | 2016-01-28 | 2016-05-04 | 南京邮电大学 | Electroencephalogram real-time monitoring control system and method based on positioning |
| CN107864440A (en) * | 2016-07-08 | 2018-03-30 | 奥迪康有限公司 | Hearing assistance system including EEG records and analysis system |
| CN111671399A (en) * | 2020-06-18 | 2020-09-18 | 清华大学 | Method, apparatus and electronic equipment for measuring noise perception intensity |
| CN112353391A (en) * | 2020-10-22 | 2021-02-12 | 武汉理工大学 | Electroencephalogram signal-based method and device for recognizing sound quality in automobile |
| CN113069125A (en) * | 2021-03-18 | 2021-07-06 | 上海趣立信息科技有限公司 | Head-mounted equipment control system, method and medium based on brain wave and eye movement tracking |
Non-Patent Citations (1)
| Title |
|---|
| "Event-Related Potentials Measured From In and Around the Ear Electrodes Integrated in a Live Hearing Device for Monitoring Sound Perception";Florian D;《 Trends in Hearing》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113782058A (en) | 2021-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10841682B2 (en) | Communication network of in-ear utility devices having sensors | |
| EP3447648B1 (en) | Master device for using connection attribute of electronic accessories connections to facilitate locating an accessory | |
| US9860650B2 (en) | Hearing assistance device comprising a location identification unit | |
| US20150305426A1 (en) | Bicycle helmet with integrated electronics | |
| US10045130B2 (en) | In-ear utility device having voice recognition | |
| US20170347178A1 (en) | In-Ear Utility Device Having a Humidity Sensor | |
| US20170347179A1 (en) | In-Ear Utility Device Having Tap Detector | |
| EP2661097A3 (en) | Audio system | |
| US20230051613A1 (en) | Systems and methods for locating mobile electronic devices with ear-worn devices | |
| CN109152663B (en) | Hearing protector and data transmission device | |
| CN111462454A (en) | Reminding method, device, equipment and storage medium | |
| WO2017205558A1 (en) | In-ear utility device having dual microphones | |
| US20170347183A1 (en) | In-Ear Utility Device Having Dual Microphones | |
| AU2022201050A1 (en) | Improving situational awareness, communication, and safety in hearing protection and communication systems | |
| CN106302974B (en) | information processing method and electronic equipment | |
| CN105791518B (en) | Information processing method and electronic equipment | |
| US20220353602A1 (en) | Method for operating a hearing aid and system having the hearing aid | |
| CN113782058B (en) | Dynamic audio perception tracking system and method, storage medium, and earphone | |
| US20230381025A1 (en) | Situational awareness, communication, and safety in hearing protection and communication systems | |
| HK1212535A1 (en) | A system for fitting audio signals for in-use ear | |
| KR20150027622A (en) | Method of preventing feedback based on body position detection and apparatuses performing the same | |
| CN114302278A (en) | Headset wearing calibration method, electronic device and computer-readable storage medium | |
| CN113347304A (en) | Intelligent help calling method and device, electronic equipment and storage medium | |
| CN117641247A (en) | Earphone anti-lost method and device, electronic equipment and readable storage medium | |
| HK40002799B (en) | Master device for using connection attribute of electronic accessories connections to facilitate locating an accessory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |