CN109218882A - The ambient sound monitor method and earphone of earphone - Google Patents
The ambient sound monitor method and earphone of earphone Download PDFInfo
- Publication number
- CN109218882A CN109218882A CN201810937218.8A CN201810937218A CN109218882A CN 109218882 A CN109218882 A CN 109218882A CN 201810937218 A CN201810937218 A CN 201810937218A CN 109218882 A CN109218882 A CN 109218882A
- Authority
- CN
- China
- Prior art keywords
- signal
- earphone
- voice signal
- microphone
- environmental sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000007613 environmental effect Effects 0.000 claims abstract description 84
- 230000005236 sound signal Effects 0.000 claims abstract description 83
- 238000011946 reduction process Methods 0.000 claims abstract description 9
- 230000009467 reduction Effects 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 22
- 238000000605 extraction Methods 0.000 claims description 18
- 238000012544 monitoring process Methods 0.000 claims description 17
- 230000003321 amplification Effects 0.000 claims description 16
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 16
- 230000015654 memory Effects 0.000 claims description 14
- 230000000717 retained effect Effects 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 14
- 238000003860 storage Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1041—Mechanical or electronic switches, or control elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2410/00—Microphones
- H04R2410/01—Noise reduction using microphones having different directional characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2420/00—Details of connection covered by H04R, not provided for in its groups
- H04R2420/03—Connection circuits to selectively connect loudspeakers or headphones to amplifiers
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Headphones And Earphones (AREA)
Abstract
The invention discloses the ambient sound monitor methods and earphone of a kind of earphone.This method comprises: in the listen mode, obtaining environmental sound signal；Voice signal is extracted from the environmental sound signal；The voice signal is exported to earphone and is played out；And/or the voice signal is removed to obtain noise signal from the environmental sound signal；Noise reduction process is carried out to the noise signal, generation de-noising signal, which is exported to earphone, to be played out.
Description
Technical field
The present invention relates to sound processing techniques field, more particularly, to a kind of earphone ambient sound monitor method with
And a kind of earphone.
Background technique
Noise cancelling headphone can reduce influence of the external environmental sounds to user, and the noise reduction mode of earphone is main in the prior art
Have three ways, such as that active noise reduction, passive noise reduction and active noise reduction mix passive noise reduction.
Earphone with active noise reduction function usually has following two operating mode:
When earphone is in noise reduction mode, earphone is by noise reduction module to the collected external environmental sounds signal of microphone
Reverse phase processing is carried out, the reversed acoustic signals for generating an equivalent reverse phase are played out by earphone, to offset external environmental sounds
Signal.Under noise reduction mode, the voice signal for including in external environmental sounds signal can also be eliminated simultaneously, cause to wear earphone
Person and people are not easy not hearing the sound of other side when talking.
When earphone is in listening mode, active noise reduction function is closed, the collected external environmental sounds letter of microphone
Number being sent to earphone plays out, and user can also talk in the case where wearing earphone with others.At this point, due to Mike
The collected external environmental sounds signal of wind includes voice signal and noise signal, and voice signal and noise signal are simultaneously by earphone
It plays back, user still will receive the interference of noise signal during talk, cause user experience poor.Therefore, having must
A kind of method can be improved headphone monitor effect is provided.
Summary of the invention
It is an object of the present invention to provide a kind of ambient sound Monitor schemes for earphone, to improve the monitoring of earphone
Effect.
It is providing according to the present invention in a first aspect, providing a kind of ambient sound monitor method of earphone, comprising:
In the listen mode, environmental sound signal is obtained；
Voice signal is extracted from the environmental sound signal；
The voice signal is exported to earphone and is played out；And/or
The voice signal is removed from the environmental sound signal to obtain noise signal；
Noise reduction process is carried out to the noise signal, generation de-noising signal, which is exported to earphone, to be played out.
Optionally, the voice signal includes the ring deviateed within earphone wearer voice directions predetermined angle range
Border voice signal.
Optionally, the predetermined angle range is 0 °~45 °.
Optionally, the earphone includes the first microphone and second microphone, and first, second microphone and earphone are worn
Wearer's mouth is on the same line；The voice signal includes: with first, second microphone towards the earphone wearer
The direction of mouth is benchmark direction, deviates the environmental sound signal within the reference direction predetermined angle range；
The acquisition environmental sound signal, comprising: ambient sound message is obtained by first, second microphone respectively
Number, to obtain first via environmental sound signal and the second road environmental sound signal；
It is described to extract voice signal from the environmental sound signal, comprising:
Wave beam forming processing is carried out to the first via environmental sound signal and second road environmental sound signal, to obtain
Take the voice signal.
Optionally, the predetermined angle range is 0 °~45 °.
Optionally, first microphone is feedforward microphone, and the second microphone is call microphone.
Optionally, described export the voice signal to earphone plays out, comprising:
Gain amplification disposal is carried out to the voice signal and carries out the broadcasting again.
Optionally, described export the voice signal to earphone plays out, comprising:
The voice signal is filtered, the voice signal retained in default frequency range carries out the broadcasting.
Optionally, described export the voice signal to earphone plays out, comprising:
The voice signal is filtered, the voice signal in default frequency range is retained；
Gain amplification disposal is carried out to the voice signal in the default frequency range and carries out the broadcasting again.
Optionally, the default frequency range is 200Hz-2KHz.
The second aspect provided according to the present invention provides a kind of earphone, including microphone, loudspeaker, voice signal extract
Module monitors module, noise signal extraction module and noise reduction module；
The microphone is used in the listen mode, picking up surrounding sounds signal；
The voice signal extraction module, for extracting voice signal from the environmental sound signal；
The monitoring module, plays out for exporting the voice signal to the loudspeaker；
The noise extraction module, for removing the voice signal from the environmental sound signal to obtain noise letter
Number；
The noise reduction module generates de-noising signal and exports to the loudspeaker for carrying out noise reduction process to the noise signal
Play out.
Optionally, the voice signal includes the ring deviateed within earphone wearer voice directions predetermined angle range
Border voice signal.
Optionally, the predetermined angle range is 0 °~45 °.
Optionally, the microphone includes the first microphone and second microphone, first, second microphone and earphone
Wearer's mouth is on the same line；The voice signal includes: to be worn with first, second microphone towards the earphone
The direction of person's mouth is benchmark direction, deviates the environmental sound signal within the reference direction predetermined angle range；
The microphone picking up surrounding sounds signal, comprising: ambient sound is obtained by first, second microphone respectively
Sound signal, to obtain first via environmental sound signal and the second road environmental sound signal；
The voice signal extraction module includes Wave beam forming module；
The Wave beam forming module, for the first via environmental sound signal and second road environmental sound signal
Wave beam forming processing is carried out, to obtain the voice signal.
Optionally, the predetermined angle range is 0 °~45 °.
Optionally, first microphone is feedforward microphone, and the second microphone is call microphone.
Optionally, the monitoring module includes gain module；
The gain module believes the voice after gain for carrying out gain amplification disposal to the voice signal
Number output is played out to the loudspeaker.
Optionally, the monitoring module includes filter module；
The filter module, for being filtered to the voice signal, the voice signal in frequency range is preset in reservation,
The voice signal in the default frequency range is exported to the loudspeaker and is played out.
Optionally, the monitoring module includes filter module and gain module；
The filter module, for being filtered to the voice signal, the voice signal in frequency range is preset in reservation:
The gain module will increase for carrying out gain amplification disposal to the voice signal in the default frequency range
The voice signal in the default frequency range after benefit, which is exported to the loudspeaker, to be played out.
Optionally, the default frequency range is 200Hz-2KHz.
According to the third aspect of the invention we, a kind of earphone is provided, comprising: memory and processor, the memory are used
In store instruction, described instruction is operated for controlling the processor to execute any one of -10 institute according to claim 1
The method stated.
Optionally, the earphone includes the first microphone and second microphone, and first, second microphone and earphone are worn
Wearer's mouth is on the same line.
Optionally, first microphone is feedforward microphone, and the second microphone is call microphone.
One beneficial effect of the embodiment of the present invention is, the method for the embodiment of the present invention in the listen mode, Neng Gouti
It takes the voice signal in environmental sound signal to play out, and/or the noise signal in environmental sound signal can be dropped
It makes an uproar processing, earphone wearer is allowed clearer can to hear extraneous voice signal, preferably monitor effect to realize.
Detailed description of the invention
It is combined in the description and the attached drawing for constituting part of specification shows the embodiment of the present invention, and even
With its explanation together principle for explaining the present invention.
Fig. 1 is the schematic flow chart of the ambient sound monitor method of earphone according to an embodiment of the present invention；
Fig. 2 is the setting position view of the microphone of earphone according to an embodiment of the present invention；
Fig. 3 is the block diagram of earphone according to an embodiment of the invention；
Fig. 4 is the block diagram of earphone according to another embodiment of the present invention；
Fig. 5 is the hardware structural diagram of the earphone according to an embodiment of the present invention with active noise reduction function.
Specific embodiment
Carry out the various exemplary embodiments of detailed description of the present invention now with reference to attached drawing.It should also be noted that unless in addition having
Body explanation, the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally
The range of invention.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the present invention
And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as part of specification.
It is shown here and discuss all examples in, any occurrence should be construed as merely illustratively, without
It is as limitation.Therefore, other examples of exemplary embodiment can have different values.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
There are two types of operating modes for earphone tool with active noise reduction function, and one is noise reduction modes, and one is listening modes.
The earphone includes that loudspeaker, the microphone being exposed to outside earphone and mode switch module is arranged.The earphone passes through pattern switching
Module switches between noise reduction mode and listening mode.Mode switch module can be by being arranged button, cunning on earphone
The modes such as key are realized.
Refering to what is shown in Fig. 1, illustrating the ambient sound monitor method of the earphone of the embodiment of the present invention, the ambient sound of the present embodiment
Sound monitor method can be implemented by earphone.
3100, environmental sound signal in the listen mode, is obtained.
Specifically, automatically turning on the microphone of earphone when earphone enters listening mode, pass through microphone picking up surrounding sounds
Signal.
3200, voice signal is extracted from the environmental sound signal.
In a specific example, the voice signal includes deviateing earphone wearer voice directions predetermined angle range
Within the environmental sound signal.The predetermined angle range for example can be 0 °~45 °.During actually talk, deviate
Environmental sound signal within the earphone wearer voice directions predetermined angle range mainly includes that earphone wearer talks object
The sound that the sound of sending and earphone wearer oneself issue, therefore in step 3200, earphone wearer hair can be will deviate from
Environmental sound signal within the predetermined angle range of sound direction is extracted as voice signal.
In step 3200, voice signal can be extracted from the environmental sound signal by beam-forming technology.
3300, the loudspeaker for exporting the voice signal to earphone play out.
That is, in embodiments of the present invention, earphone in the listen mode, by voice signal from environmental sound signal
It extracts and is input to the loudspeaker of earphone and plays out, so that user can also hear oneself in the case where wearing earphone and hand over
It dates the sound of sending, realizes the normal communication with talk object.
(1) in an example of the invention, in step 3300, directly by voice signal export to the loudspeaker of earphone into
Row plays.Specifically, directly voice signal can be exported to the loudspeaker progress of earphone by the monitoring module being arranged in earphone
It plays.
(2) in an example of the invention, step 3300 may further include step 3310.
3310, voice signal is filtered, the voice signal only retained in default frequency range plays out.
In the present embodiment, voice signal is defined with predetermined angle range, the environment within the scope of this predetermined angle
Voice signal is mainly earphone wearer and the human voice that talk object issues, but may also be believed simultaneously comprising some noises
Number, such as the noise etc. that the noise and earphone own hardware factor that introduce from transmission medium generate.At this point, if directly will
Voice signal is exported to the loudspeaker of earphone and is played out it is also possible that a little noise can be introduced, and influences earphone wearer and others
Talk.Therefore, in step 3310, voice signal is filtered, the voice signal only retained in default frequency range plays out,
Wherein presetting frequency range is the frequency range according to the main frequency range setting of voice, that is to say, that this embodiment passes through filtering further filter
Except noise, retain the signal for meeting the main frequency range of voice.Sound frequency when people speaks be mostly within 200Hz-2KHz, because
This default frequency range is preferably 200Hz-2KHz, which can be used bandpass filter.
It can be seen that the ambient sound monitor method of the present embodiment, earphone can in the listen mode, from the ring got
Voice signal is extracted in the voice signal of border, and voice signal is filtered, and only retains the signal output in the main frequency range of voice
It is played out to earphone, reduces the noise signal other than voice frequency range, wearing earphone person is more clearly heard
The sound of oneself and talk object sending.
(3) in an example of the invention, step 3300 may further include step 3320.
3320, earphone carries out gain amplification disposal to voice signal to be played and plays out again.
Specifically, can be carried out by the signal amplifier being arranged in the monitoring module of earphone to voice signal to be played
Gain amplification.
It can be seen that the ambient sound monitor method of the present embodiment, earphone can in the listen mode, from the ring got
Voice signal is extracted in the voice signal of border, and gain amplification disposal is carried out to voice signal to be played and is played out again, thus
It further protrudes earphone wearer oneself and talks the sound that object issues.
(4) in another example of the invention, step 3310 can be first carried out, voice signal is filtered, retained
Voice signal in default frequency range, then step 3320 is executed, gain amplification is carried out to the voice signal in the default frequency range of reservation
Processing, then plays out again.
In the example, earphone first passes through the filter being arranged in the monitoring module of earphone and is filtered to voice signal, only
Retain the voice signal in frequency range 200Hz-2KHz, then the signal amplifier being arranged in the monitoring module for passing through earphone is to frequency range
Voice signal in 200Hz-2KHz play out after gain amplification.
It can be seen that the ambient sound monitor method of the present embodiment, earphone can in the listen mode, from the ring got
Voice signal is extracted in the voice signal of border, and voice signal is filtered, and only retains the signal in the main frequency range of voice, and right
Signal in the main frequency range of the voice of reservation carries out gain amplification, thus reducing the same of the noise signal other than voice frequency range
When, the sound that further prominent earphone wearer oneself and talk object issue provides better monitoring for earphone wearer
Effect.
3400, the voice signal is removed to obtain noise signal from the environmental sound signal.
Specifically, environmental sound signal and voice signal can be subtracted each other by the subtracter being arranged in earphone, to obtain
Noise signal.
3500, noise reduction process is carried out to the noise signal, generation de-noising signal, which is exported to earphone, to be played out.
Specifically, in step 3500, filter can be used, reverse phase processing is carried out to noise signal to generate and noise
Signal magnitude is identical and the de-noising signal of opposite in phase, and de-noising signal is exported to earphone and is played out, is supported by de-noising signal
De-noising acoustical signal achievees the effect that active noise reduction.
It should be noted that in another embodiment, can only execute step 3100-3300, not execute step 3400
With step 3500.In another embodiment, step 3100,3200,3400 and 3500 can be only executed, step is not executed
3300.These three modes can help user is clearer in the case where monitoring the mode to hear extraneous voice signal, to realize
Preferably monitor effect.
One beneficial effect of the embodiment of the present invention is, the method for the embodiment of the present invention in the listen mode, Neng Gouti
It takes the voice signal in environmental sound signal to play out, and/or the noise signal in environmental sound signal can be dropped
It makes an uproar processing, earphone wearer is allowed clearer can to hear extraneous voice signal, preferably monitor effect to realize.
Illustrate the above-mentioned steps 3200 of the embodiment of the present invention below by a specific example.Referring to shown in Fig. 2, explanation
The setting position view of the microphone of the corresponding earphone of the embodiment of the present invention.
Earphone includes the first microphone A and second microphone B.In the position for designing the first microphone A and second microphone B
When, guarantee that the position C three of the position of the first microphone A, the position of second microphone B and earphone wearer's mouth is basic
On same straight line, referring to fig. 2 in dotted line shown in.Wherein, the first microphone A and second microphone B can be omnidirectional
It is also possible to directive property.The distance between first microphone A and second microphone B can be designed according to actual needs and
It adjusts.
If earphone sheet can use the feedforward Mike of earphone as dual microphone uplink noise cancelling headphone, the first microphone A
Wind realize, second microphone B can use earphone call microphone realize, only need to guarantee feedforward microphone, call microphone,
And earphone wearer's mouth three is substantially on same straight line.
For step 3100, in the listen mode, obtain environmental sound signal, it is specific:
Earphone obtains environmental sound signal by the first microphone A and second microphone B respectively, to obtain first via environment
Voice signal and the second road environmental sound signal.First via environmental sound signal and the second road environmental sound signal are essentially identical.
For step S3200, earphone extracts voice signal from environmental sound signal, specific:
Voice signal includes by the first microphone A, second microphone B towards side on the basis of the direction of earphone wearer's mouth
To, deviate reference direction predetermined angle θ range within environmental sound signal.Referring to shown in Fig. 2, reference direction is from the first wheat
Gram wind A towards earphone wearer's mouth position C direction, predetermined angle θ can be 45 °, and extraction is constituted with AC1, AC2 solid line
Conical area range within environmental sound signal as voice signal.During actually talk, this predetermined angle
Environmental sound signal within range mainly includes that earphone wearer talks the sound that object issues and earphone wearer oneself hair
Sound out, therefore in step 3200, the environmental sound signal within reference direction predetermined angle θ range can be will deviate from and mentioned
It takes out as voice signal.
In step S3200, extracting voice signal from environmental sound signal can further comprise: to first via ambient sound
Sound signal and the second road environmental sound signal carry out Wave beam forming processing, to obtain voice signal.
Specifically, Wave beam forming processing can be carried out to first via environmental sound signal and the second road environmental sound signal,
To obtain voice signal.For example, the microphone array Beamforming Method that Wave beam forming processing can be added up based on delay-.Example
Such as, Wave beam forming module calculates the time difference that target sound source reaches the first microphone and second microphone according to predetermined angle, leads to
It crosses and is reached to first via environmental sound signal and the second road ring the time difference of the first microphone and second microphone to target sound source
Border voice signal carries out compensation of delay, is same by compensated first via environmental sound signal and the second road environmental sound signal
Step, output signal is then obtained by fixed weighting coefficient product accumulation, output signal at this time is voice signal.
Refering to what is shown in Fig. 3, illustrating earphone provided by one embodiment of the present invention:
Earphone includes microphone 10, loudspeaker 60, voice signal extraction module 20, monitors module 40, noise signal extraction mould
Block 30 and noise reduction module 50.
Microphone 10 is used in the listen mode, picking up surrounding sounds signal.
Voice signal extraction module 20, for extracting voice signal from the environmental sound signal that microphone 10 picks up.
Module 40 being monitored, being played out for exporting the voice signal to loudspeaker 60.
Noise extraction module 30, for removing the voice signal from the environmental sound signal that microphone 10 picks up to obtain
To noise signal.
Noise reduction module 50, for the noise signal carry out noise reduction process, generate de-noising signal export to loudspeaker 60 into
Row plays.
In a specific example, the voice signal includes deviateing earphone wearer voice directions predetermined angle range
Within the environmental sound signal.The predetermined angle range for example can be with 0 °~45 °.
In a specific example, monitoring module 40 includes gain module.Gain module is used for the voice signal
Gain amplification disposal is carried out, the voice signal after gain is exported to loudspeaker and is played out.
In a specific example, monitoring module 40 includes filter module.Filter module is used for the voice signal
It is filtered, retains the voice signal in default frequency range, the voice signal in the default frequency range is exported to loudspeaker
Play out.The default frequency range can be 200Hz-2KHz.
In a specific example, monitoring module 40 includes filter module and gain module.Filter module is used for institute
Predicate sound signal is filtered, and retains the voice signal in default frequency range.Gain module is used for in the default frequency range
The voice signal carry out gain amplification disposal, the voice signal in the default frequency range after gain is exported to loudspeaker
Play out.The default frequency range can be 200Hz-2KHz.
It is shown in Figure 4, illustrate the earphone that another embodiment of the present invention provides:
Earphone includes the first microphone 11, second microphone 12, loudspeaker 60, voice signal extraction module 20, monitors module
40, noise signal extraction module 30 and noise reduction module 50.Wherein, voice signal extraction module 20 includes Wave beam forming module 21,
Monitoring module 40 includes filter module 41 and gain module 42.First microphone 11, second microphone 12 and earphone wearer mouth
Buckie is originally on the same line.First microphone 11 is feedforward microphone, and second microphone 12 is call microphone.
First microphone 11, the environmental sound signal for picking up surrounding sounds signal in the listen mode, output are
First via environmental sound signal.
First microphone 12, the environmental sound signal for picking up surrounding sounds signal in the listen mode, output are
Second road environmental sound signal.
Voice signal extraction module 20, for extracting voice signal from environmental sound signal.Specifically, passing through wave beam shape
Wave beam forming processing, the predicate to obtain are carried out to first via environmental sound signal and the second road environmental sound signal at module 21
Sound signal.The voice signal includes: with first microphone 11, second microphone 12 towards the side of earphone wearer's mouth
To for benchmark direction, deviate the environmental sound signal within the reference direction predetermined angle range.The predetermined angle model
Enclosing for example can be with 0 °~45 °.
Module 40 being monitored, being played out for exporting the voice signal to loudspeaker 60.Specifically, passing through filter module
41 pairs of voice signals are filtered, and retain the voice signal in default frequency range, which is, for example, 200Hz-
2KHz.Gain amplification disposal is carried out to the voice signal in the default frequency range by gain module 42 again, after gain
The default frequency range in the voice signal export to loudspeaker 60 and play out.
Noise extraction module 30, for removing the voice signal from the environmental sound signal to obtain noise letter
Number.
Noise reduction module 50, for the noise signal carry out noise reduction process, generate de-noising signal export to loudspeaker 60 into
Row plays.
It can be seen that the earphone of the embodiment of the present invention, earphone can be in the listen mode to making an uproar in environmental sound signal
Acoustical signal carries out noise reduction process, to generate identical as the noise signal size and opposite in phase a de-noising signal with should
Noise signal is offset, meanwhile, gain amplification disposal is filtered and carried out to the voice signal in environmental sound signal, with
Allow and wear the sound that earphone person clearly hears conversation party's sending, improves the noise robustness of earphone under listening mode.
It should be noted that filter can be only included by monitoring module 40 in the earphone that another embodiment of the present invention provides
One in wave module 41 and gain module 42.
Fig. 5 is the hardware structural diagram of the earphone according to an embodiment of the present invention with active noise reduction function.According to Fig. 5
Shown, the earphone with active noise reduction function may include one or more memories 200 and one or more processors 100.
Processor 100 is such as can be central processor CPU, Micro-processor MCV.Memory 200 for example may include
ROM (read-only memory), RAM (random access memory), nonvolatile memory of hard disk etc..
Processor 100 is connect with microphone 10, loudspeaker 60, memory 200 respectively.Memory 200 is for storing computer
Instruction, the computer instruction are operated for control processor 100 to execute ambient sound prison according to an embodiment of the present invention
Method is listened,
Those skilled in the art can disclosed Technical Design instruction according to the present invention.Instruction is how to control place
Reason device is operated, and belongs to techniques known, therefore the embodiment of the present invention is not described in detail herein.
It should be noted that earphone provided by the above embodiment is corresponding with the ambient sound monitor method of aforementioned headset, it is preceding
The related content for stating the ambient sound monitor method of earphone is equally used for explaining earphone.
The present invention can be system, method and/or computer program product.Computer program product may include computer
Readable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the invention.
Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment
Equipment.Computer readable storage medium for example can be-- but it is not limited to-- storage device electric, magnetic storage apparatus, optical storage
Equipment, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium
More specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only deposits
It is reservoir (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable
Compact disk read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example thereon
It is stored with punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Calculating used herein above
Machine readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations lead to
It crosses the electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of waveguide or the propagation of other transmission mediums or is transmitted by electric wire
Electric signal.
Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing operation of the present invention can be assembly instruction, instruction set architecture (ISA) instructs,
Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages
The source code or object code that any combination is write, the programming language include the programming language-of object-oriented such as
Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer
Readable program instructions can be executed fully on the user computer, partly execute on the user computer, be only as one
Vertical software package executes, part executes on the remote computer or completely in remote computer on the user computer for part
Or it is executed on server.In situations involving remote computers, remote computer can pass through network-packet of any kind
It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit
It is connected with ISP by internet).In some embodiments, by utilizing computer-readable program instructions
Status information carry out personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or can
Programmed logic array (PLA) (PLA), the electronic circuit can execute computer-readable program instructions, to realize each side of the invention
Face.
Referring herein to according to the method for the embodiment of the present invention, the flow chart of device (system) and computer program product and/
Or block diagram describes various aspects of the invention.It should be appreciated that flowchart and or block diagram each box and flow chart and/
Or in block diagram each box combination, can be realized by computer-readable program instructions.
These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas
The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas
When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced
The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to
It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction
Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram
The instruction of the various aspects of defined function action.
Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other
In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce
Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment
Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.
The flow chart and block diagram in the drawings show the system of multiple embodiments according to the present invention, method and computer journeys
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use
The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box
It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel
Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or
The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic
The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.It is right
For those skilled in the art it is well known that, by hardware mode realize, by software mode realize and pass through software and
It is all of equal value that the mode of combination of hardware, which is realized,.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or lead this technology
Other those of ordinary skill in domain can understand each embodiment disclosed herein.The scope of the present invention is limited by appended claims
It is fixed.
Claims (14)
1. a kind of ambient sound monitor method of earphone characterized by comprising
In the listen mode, environmental sound signal is obtained；
Voice signal is extracted from the environmental sound signal；
The voice signal is exported to earphone and is played out；And/or
The voice signal is removed from the environmental sound signal to obtain noise signal；
Noise reduction process is carried out to the noise signal, generation de-noising signal, which is exported to earphone, to be played out.
2. the method according to claim 1, wherein the voice signal includes deviateing earphone wearer sounding side
To the environmental sound signal within predetermined angle range.
3. according to the method described in claim 2, it is characterized in that, the predetermined angle range is 0 °~45 °.
4. the method according to claim 1, wherein the earphone include the first microphone and second microphone,
First, second microphone and earphone wearer mouth are on the same line；The voice signal include: with described first,
Second microphone is benchmark direction towards the direction of the earphone wearer mouth, deviates the reference direction predetermined angle range
Within the environmental sound signal；
The acquisition environmental sound signal, comprising: environmental sound signal is obtained by first, second microphone respectively, with
Obtain first via environmental sound signal and the second road environmental sound signal；
It is described to extract voice signal from the environmental sound signal, comprising:
Wave beam forming processing is carried out to the first via environmental sound signal and second road environmental sound signal, to obtain
Predicate sound signal.
5. according to the method described in claim 4, it is characterized in that, the predetermined angle range is 0 °~45 °.
6. according to the method described in claim 4, it is characterized in that, first microphone be feedforward microphone, described second
Microphone is call microphone.
7. method according to claim 1-6, which is characterized in that described to export the voice signal to earphone
It plays out, comprising:
Gain amplification disposal is carried out to the voice signal and carries out the broadcasting again.
8. method according to claim 1-6, which is characterized in that described to export the voice signal to earphone
It plays out, comprising:
The voice signal is filtered, the voice signal retained in default frequency range carries out the broadcasting.
9. method according to claim 1-6, which is characterized in that described to export the voice signal to earphone
It plays out, comprising:
The voice signal is filtered, the voice signal in default frequency range is retained；
Gain amplification disposal is carried out to the voice signal in the default frequency range and carries out the broadcasting again.
10. method according to claim 9 or 10, which is characterized in that the default frequency range is 200Hz-2KHz.
11. a kind of earphone, which is characterized in that including microphone, loudspeaker, voice signal extraction module, monitor module, noise signal
Extraction module and noise reduction module；
The microphone is used in the listen mode, picking up surrounding sounds signal；
The voice signal extraction module, for extracting voice signal from the environmental sound signal；
The monitoring module, plays out for exporting the voice signal to the loudspeaker；
The noise extraction module, for removing the voice signal from the environmental sound signal to obtain noise signal；
The noise reduction module, for the noise signal carry out noise reduction process, generate de-noising signal export to the loudspeaker into
Row plays.
12. a kind of earphone characterized by comprising memory and processor, the memory for storing instruction, the finger
Order is operated for controlling the processor to execute method according to claim 1 to 10.
13. earphone according to claim 12, which is characterized in that the earphone includes the first microphone and the second Mike
Wind, first, second microphone and earphone wearer mouth are on the same line.
14. earphone according to claim 13, which is characterized in that first microphone is feedforward microphone, described the
Two microphones are call microphone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810937218.8A CN109218882B (en) | 2018-08-16 | 2018-08-16 | Earphone and ambient sound monitoring method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810937218.8A CN109218882B (en) | 2018-08-16 | 2018-08-16 | Earphone and ambient sound monitoring method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109218882A true CN109218882A (en) | 2019-01-15 |
| CN109218882B CN109218882B (en) | 2021-02-26 |
Family
ID=64988540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810937218.8A Active CN109218882B (en) | 2018-08-16 | 2018-08-16 | Earphone and ambient sound monitoring method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109218882B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110211599A (en) * | 2019-06-03 | 2019-09-06 | Oppo广东移动通信有限公司 | Application awakening method and device, storage medium and electronic equipment |
| CN110753285A (en) * | 2019-10-24 | 2020-02-04 | 歌尔股份有限公司 | Method, device and equipment for processing sound signal of noise reduction earphone and storage medium |
| CN112448787A (en) * | 2019-09-02 | 2021-03-05 | 中兴通讯股份有限公司 | Method, device and system for preventing eavesdropping and stealing recording in conference |
| CN112770214A (en) * | 2021-01-28 | 2021-05-07 | 歌尔科技有限公司 | Earphone control method and device and earphone |
| CN113099338A (en) * | 2021-03-08 | 2021-07-09 | 头领科技（昆山）有限公司 | Intelligent control's audio chip and wireless earphone of making an uproar that falls |
| CN113098901A (en) * | 2021-04-29 | 2021-07-09 | 心动互动娱乐有限公司 | High-concurrency real-time voice call method, system, device and medium |
| CN113099348A (en) * | 2021-04-09 | 2021-07-09 | 泰凌微电子(上海)股份有限公司 | Noise reduction method, noise reduction device and earphone |
| CN113596661A (en) * | 2021-07-28 | 2021-11-02 | 维沃移动通信有限公司 | Earphone, feedback noise reduction method and device |
| CN114501210A (en) * | 2020-11-13 | 2022-05-13 | 炬芯科技股份有限公司 | Active noise reduction circuit with directional permeability, method, equipment and storage medium |
| WO2022135340A1 (en) * | 2020-12-22 | 2022-06-30 | 华为技术有限公司 | Active noise reduction method, device and system |
| US12047760B2 (en) | 2019-12-25 | 2024-07-23 | Huawei Technologies Co., Ltd. | Bluetooth communication method and apparatus |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101009820A (en) * | 2007-01-26 | 2007-08-01 | 北京佳讯飞鸿电气有限责任公司 | A maintenance method and device for the call signal device |
| CN101859568A (en) * | 2009-04-10 | 2010-10-13 | 比亚迪股份有限公司 | Method and device for eliminating voice background noise |
| CN102209987A (en) * | 2008-11-24 | 2011-10-05 | 高通股份有限公司 | Systems, methods, apparatus, and computer program products for enhanced active noise cancellation |
| CN102300140A (en) * | 2011-08-10 | 2011-12-28 | 歌尔声学股份有限公司 | Speech enhancing method and device of communication earphone and noise reduction communication earphone |
| US20120029912A1 (en) * | 2010-07-27 | 2012-02-02 | Voice Muffler Corporation | Hands-free Active Noise Canceling Device |
| WO2012115785A2 (en) * | 2011-02-25 | 2012-08-30 | Beevers Manufacturing & Supply, Inc. | Electronic communication system that mimics natural range and orientation dependence |
| CN103269465A (en) * | 2013-05-22 | 2013-08-28 | 歌尔声学股份有限公司 | Headset communication method under loud-noise environment and headset |
| CN103310797A (en) * | 2013-06-28 | 2013-09-18 | 姜鸿彦 | Noise and voice signal extraction device |
| CN103310796A (en) * | 2013-06-28 | 2013-09-18 | 姜鸿彦 | Voice signal extraction method |
| US20140321654A1 (en) * | 2007-01-23 | 2014-10-30 | Samsung Electronics Co., Ltd. | Apparatus and method for transmitting/receiving voice signal through headset |
| US20160026253A1 (en) * | 2014-03-11 | 2016-01-28 | Magic Leap, Inc. | Methods and systems for creating virtual and augmented reality |
| CN105719657A (en) * | 2016-02-23 | 2016-06-29 | 惠州市德赛西威汽车电子股份有限公司 | Human voice extracting method and device based on microphone |
| CN106548781A (en) * | 2015-09-21 | 2017-03-29 | 上海日趋信息技术有限公司 | A kind of method for eliminating background noise for speech recognition system |
| CN107045778A (en) * | 2017-04-26 | 2017-08-15 | 兰州交通大学 | A kind of Multifunctional noise bucking-out system |
-
2018
- 2018-08-16 CN CN201810937218.8A patent/CN109218882B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140321654A1 (en) * | 2007-01-23 | 2014-10-30 | Samsung Electronics Co., Ltd. | Apparatus and method for transmitting/receiving voice signal through headset |
| CN101009820A (en) * | 2007-01-26 | 2007-08-01 | 北京佳讯飞鸿电气有限责任公司 | A maintenance method and device for the call signal device |
| CN102209987A (en) * | 2008-11-24 | 2011-10-05 | 高通股份有限公司 | Systems, methods, apparatus, and computer program products for enhanced active noise cancellation |
| CN101859568A (en) * | 2009-04-10 | 2010-10-13 | 比亚迪股份有限公司 | Method and device for eliminating voice background noise |
| US20120029912A1 (en) * | 2010-07-27 | 2012-02-02 | Voice Muffler Corporation | Hands-free Active Noise Canceling Device |
| WO2012115785A2 (en) * | 2011-02-25 | 2012-08-30 | Beevers Manufacturing & Supply, Inc. | Electronic communication system that mimics natural range and orientation dependence |
| CN102300140A (en) * | 2011-08-10 | 2011-12-28 | 歌尔声学股份有限公司 | Speech enhancing method and device of communication earphone and noise reduction communication earphone |
| CN103269465A (en) * | 2013-05-22 | 2013-08-28 | 歌尔声学股份有限公司 | Headset communication method under loud-noise environment and headset |
| CN103310797A (en) * | 2013-06-28 | 2013-09-18 | 姜鸿彦 | Noise and voice signal extraction device |
| CN103310796A (en) * | 2013-06-28 | 2013-09-18 | 姜鸿彦 | Voice signal extraction method |
| US20160026253A1 (en) * | 2014-03-11 | 2016-01-28 | Magic Leap, Inc. | Methods and systems for creating virtual and augmented reality |
| CN106548781A (en) * | 2015-09-21 | 2017-03-29 | 上海日趋信息技术有限公司 | A kind of method for eliminating background noise for speech recognition system |
| CN105719657A (en) * | 2016-02-23 | 2016-06-29 | 惠州市德赛西威汽车电子股份有限公司 | Human voice extracting method and device based on microphone |
| CN107045778A (en) * | 2017-04-26 | 2017-08-15 | 兰州交通大学 | A kind of Multifunctional noise bucking-out system |
Non-Patent Citations (1)
| Title |
|---|
| 曹辉: "基于局域网的音频实时记录和监听系统的设计与实现", 《中国优秀硕士学位论文全文数据库-信息科技辑》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110211599A (en) * | 2019-06-03 | 2019-09-06 | Oppo广东移动通信有限公司 | Application awakening method and device, storage medium and electronic equipment |
| CN110211599B (en) * | 2019-06-03 | 2021-07-16 | Oppo广东移动通信有限公司 | Application wake-up method, device, storage medium and electronic device |
| CN112448787A (en) * | 2019-09-02 | 2021-03-05 | 中兴通讯股份有限公司 | Method, device and system for preventing eavesdropping and stealing recording in conference |
| CN110753285A (en) * | 2019-10-24 | 2020-02-04 | 歌尔股份有限公司 | Method, device and equipment for processing sound signal of noise reduction earphone and storage medium |
| US12047760B2 (en) | 2019-12-25 | 2024-07-23 | Huawei Technologies Co., Ltd. | Bluetooth communication method and apparatus |
| CN114501210A (en) * | 2020-11-13 | 2022-05-13 | 炬芯科技股份有限公司 | Active noise reduction circuit with directional permeability, method, equipment and storage medium |
| CN114501210B (en) * | 2020-11-13 | 2025-03-28 | 炬芯科技股份有限公司 | Active noise reduction circuit, method, device and storage medium with directional transparency |
| WO2022135340A1 (en) * | 2020-12-22 | 2022-06-30 | 华为技术有限公司 | Active noise reduction method, device and system |
| CN112770214A (en) * | 2021-01-28 | 2021-05-07 | 歌尔科技有限公司 | Earphone control method and device and earphone |
| CN113099338A (en) * | 2021-03-08 | 2021-07-09 | 头领科技（昆山）有限公司 | Intelligent control's audio chip and wireless earphone of making an uproar that falls |
| CN113099348A (en) * | 2021-04-09 | 2021-07-09 | 泰凌微电子(上海)股份有限公司 | Noise reduction method, noise reduction device and earphone |
| CN113099348B (en) * | 2021-04-09 | 2024-06-21 | 泰凌微电子(上海)股份有限公司 | Noise reduction method, noise reduction device and earphone |
| CN113098901B (en) * | 2021-04-29 | 2022-09-09 | 心动互动娱乐有限公司 | High-concurrency real-time voice call method, system, device and medium |
| CN113098901A (en) * | 2021-04-29 | 2021-07-09 | 心动互动娱乐有限公司 | High-concurrency real-time voice call method, system, device and medium |
| CN113596661A (en) * | 2021-07-28 | 2021-11-02 | 维沃移动通信有限公司 | Earphone, feedback noise reduction method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109218882B (en) | 2021-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109218882A (en) | The ambient sound monitor method and earphone of earphone | |
| KR102424257B1 (en) | An audio processing device and a method for estimating a signal-to-noise-ratio of a sound signal | |
| US20190158965A1 (en) | Hearing aid comprising a beam former filtering unit comprising a smoothing unit | |
| DK2882204T3 (en) | Hearing aid device for hands-free communication | |
| CN110475178B (en) | Wireless earphone noise reduction method and device, wireless earphone and storage medium | |
| CN111464905A (en) | Hearing enhancement method and system based on intelligent wearable device and wearable device | |
| CN106797508B (en) | For improving the method and earphone of sound quality | |
| CN109195045A (en) | The method, apparatus and earphone of test earphone wearing state | |
| CN205092949U (en) | Noise -canceling headphones | |
| CN112770214A (en) | Earphone control method and device and earphone | |
| CN105229737A (en) | Noise cancelling microphone device | |
| US11689869B2 (en) | Hearing device configured to utilize non-audio information to process audio signals | |
| US11589173B2 (en) | Hearing aid comprising a record and replay function | |
| CN112399301B (en) | Earphone and noise reduction method | |
| CN112992169A (en) | Voice signal acquisition method and device, electronic equipment and storage medium | |
| CN113949955B (en) | Noise reduction processing method, device, electronic equipment, earphone and storage medium | |
| WO2023284402A1 (en) | Audio signal processing method, system, and apparatus, electronic device, and storage medium | |
| US20250063287A1 (en) | Hearing device or system comprising a noise control system | |
| US11551704B2 (en) | Method and device for spectral expansion for an audio signal | |
| EP4287646B1 (en) | A hearing aid or hearing aid system comprising a sound source localization estimator | |
| CN105141729A (en) | Noise reduction method, noise reduction device and mobile phone | |
| CN113038318B (en) | A kind of voice signal processing method and device | |
| WO2021129196A1 (en) | Voice signal processing method and device | |
| CN113450819B (en) | Signal processing method and related product | |
| US20240251208A1 (en) | Hearing aid and distance-specific amplifier |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |