EP3225037B1 - Verfahren und vorrichtung zur erzeugung eines gerichteten tonsignals aus ersten und zweiten tonsignalen - Google Patents
Verfahren und vorrichtung zur erzeugung eines gerichteten tonsignals aus ersten und zweiten tonsignalen Download PDFInfo
- Publication number
- EP3225037B1 EP3225037B1 EP14771598.1A EP14771598A EP3225037B1 EP 3225037 B1 EP3225037 B1 EP 3225037B1 EP 14771598 A EP14771598 A EP 14771598A EP 3225037 B1 EP3225037 B1 EP 3225037B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- frequency
- signal
- directional
- deq
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
- H04R2430/25—Array processing for suppression of unwanted side-lobes in directivity characteristics, e.g. a blocking matrix
Definitions
- the present invention generally relates to the field of sound acquisition. More particularly, the present invention relates to a method and an apparatus for generating a directional sound signal from first and second sound signals, which are generated by a first and a second microphone, which are separated by a distance.
- microphone arrays proved to be useful. They are designed to attenuate possible noise and interference components while retaining the desired source signal by exploiting different spatial (or directional) characteristics of the different signal sources (see, e.g., J. Benesty, J. Chen, and Y. Huang, "Microphone Array Signal Processing,” Heidelberg: Springer, 2008 for an overview).
- a simple, yet efficient approach is the first-order differential microphone array described in G. Elko and A.-T. N. Pong, "A simple adaptive first-order differential microphone,” in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pages 169 to 172, October 1995 .
- This microphone array which is schematically and exemplarily shown in Fig. 1 allows to place two symmetrical notches (directions of maximum attenuation) at angles of ⁇ and 360° - ⁇ .
- three independent enhancements to the original method are proposed and a practical implementation for handsfree communication is described.
- a possible target device is a wireless loudspeaker with two integrated miniature digital micro-electromechanical system (MEMS) microphone capsules which facilitate handsfree audio coomunication.
- MEMS micro-electromechanical system
- Fig. 1 shows schematically and exemplarily a differential microphone array according to G. Elko and A.-T. N. Pong.
- Two closely spaced omnidirectional microphones M1 and M2 are used to capture the acoustic environment.
- the corresponding digital signals x 1 ( k ) and x 2 ( k ) are sampled with a rate of f s .
- Due to the small distance D between M1 and M2, a coherent mutual subtraction - for convenience, acausal filters are assumed herein; in practice, appropriate signal alignment is required as marked by 'o' in Figs.
- the signals x f ( k ) and x b ( k ) can be interpreted as "forward and backward facing cardioid" signals as the respective directional responses of Eqs. (1) and (2) form cardioid shapes (see Fig.
- a common problem with differential microphone arrays are the tolerances of the employed microphones, leading to a “microphone mismatch” and therefore noise amplification (see M. Buck and M. R michler, "First Order Differential Microphone Arrays for Automotive Applications,” in Proceedings of International Workshop on Acoustic Echo and Noise Control (IWAENC), September 2001 ).
- the digital MEMS microphones in the possible target device usually exhibit relatively constant frequency responses; therefore, individual microphone equalization is preferably not necessary for the envisaged application.
- their power levels may still vary to a certain extent due to mounting and assembly tolerances, which is disadvantageous since it is preferred to have fully matched input levels in order to utilize the full potential of the method.
- a method for generating a directional sound signal from first and second sound signals, which are generated by a first and a second microphone, which are separated by a distance comprises:
- the present invention is based on the idea that by employing these steps, a (substantially) frequency invariant notch characteristic can be obtained even for larger microphone distances.
- a larger distance also helps to confine the noise gain of the array. Therefore, the array becomes practically usable even for higher sampling rates (e.g., 16kHz).
- difference signal also includes the case where one or both of the first and the second sound signals is/are further temporally delayed, for example, by means of a fractional delay filter h T ( k ), as described in section 2 above.
- the frequency-selective processing comprises weighting the difference signal with an approximated steering factor that is independent of frequency to generate a weighted difference signal and correcting for the approximation by adding a correction signal that is generated from the difference signal in dependence of frequency and the steering angle.
- the generation of the correction signal comprises applying two separate operations, one being dependent on frequency and independent of the steering angle and one being dependent on the steering angle but independent of frequency.
- the generation of the correction signal comprises filtering the difference signal with a filter that is dependent on frequency and independent of the steering angle to generate a filtered difference signal.
- the generation of the correction signal further comprises weighting the filtered difference signal with a factor that is dependent on the steering angle and independent of frequency.
- the factor is determined by using a polynomial approximation that is evaluated with the steering angle.
- the method further comprises filtering the directional sound signal with a low-pass filter to generate a filtered directional sound signal.
- the approximated steering factor for a time instance is adapted for the following time instance by adding an adaptation value that is scaled by a stepsize parameter, wherein the stepsize parameter is adapted in dependence of estimated energies of coherent and incoherent sound components.
- the energy of the incoherent sound components is approximated by the estimated short-term energy of the directional sound signal and the energy of the coherent sound components is approximated by a fraction of the estimated short-term energy of the difference signal.
- the method further comprises estimating a relative gain of the first and the second microphone and equalizing power levels of the first and the second microphone based on the relative gain.
- the relative gain is determined based on recursively estimated variances of the first and second sound signals.
- the first and the second microphone are omnidirectional microphones.
- an apparatus for generating a directional sound signal from first and second sound signals, which are generated by a first and a second microphone, which are separated by a distance comprising:
- a system comprising:
- this transfer function should become zero for a specific angle, i.e., the so-called steering angle ⁇ .
- ⁇ a ( ⁇ , ⁇ ) is separable with good accuracy: ⁇ ⁇ a ⁇ ⁇ ⁇ ⁇ ⁇ a ⁇ ⁇ ⁇ ⁇ a ⁇ ⁇ ⁇ ⁇ a ⁇
- the factors ⁇ a ( ⁇ ) and ⁇ a ( ⁇ ) can be computed by marginalization of the 2-dimensional function ⁇ a ( ⁇ , ⁇ ) and appropriate normalization.
- the factor ⁇ a ( ⁇ ) can now be regarded as the frequency response of a fixed filter. It can be transformed to the time domain via periodic extension, inverse DFT, cyclic shifting (to enforce causality) and an appropriate shortening to a desired length.
- the resulting FIR filter coefficients h DEQ ( k ), e.g., of order 16, are independent of the steering angle ⁇ .
- the angular dependency is then reintroduced with a polynomial approximation (e.g., order 4) of the second factor ⁇ a ( ⁇ ) after a variable transformation from ⁇ to a lin ( ⁇ ), i.e., P ⁇ a ( a lin ( ⁇ )) ⁇ ⁇ a ( ⁇ ( a lin )).
- the effect of directional equalizing can be observed in Fig. 3 (b) , which displays an almost frequency-invari
- the practical operation of the modified version of the microphone array does not significantly differ from the conventional version ( Fig. 1 ):
- the desired notch angle ⁇ is still easily controlled by adapting the scalar factor a lin .
- the polynomial P ⁇ a ( a lin ) must be evaluated.
- the distorted notch curve of the standard differential array ( Fig. 3 (a) ) not only limits the ability to suppress interfering sound sources, but it can even compromise the accurate NLMS adaptation of the steering angle ⁇ (see section 6).
- the goal of a notch adaptation algorithm is to automatically align the notch angle ⁇ of the differential array with the incidence angle ⁇ of the (main) interferer.
- the standard approach to adapt the factor a (or a lin if directional equalization is used) and therefore the notch angle ⁇ is the ( normalized ) least mean square (NLMS) algorithm.
- the goal here is to minimize the power of the output signal y ( k ), i.e.
- This equation represents the error signal of a single-tap adaptive filter with a noisy input.
- the noise signal n(k) is due to the incoherent (ambient) noise that cannot be suppressed.
- the coherent contribution to y ( k ) should ideally be zero.
- an error signal e ( k ) appears at the output.
- E ⁇ n 2 ( k ) ⁇ is the level ⁇ ⁇ y 2 of the microphone array's output y ( k ) while for E ⁇ e 2 ( k ) ⁇ , the assumption of a fixed attenuation factor for the backward cardioid signal is made, i.e. E e 2 k ⁇ ⁇ ⁇ ⁇ ⁇ x b 2 .
- the adaptation can be deliberately slowed down by the factor 0 ⁇ ⁇ ⁇ 1 to avoid artifacts that stem from the single-tap prediction which does not apply any smoothing.
- the combination of the proposed NLMS notch adaptation with the directional equalizer of section 5 is straight forward.
- the equalizer can indirectly influence and enhance the notch adaptation via the array output signal y ( k ).
- the performance of the proposed fast notch adaptation algorithm is contrasted with the conventional NLMS using a fixed stepsize in Fig. 5 .
- the graph illustrates the adaptation process for a synthetic sound field with a single sound source that arrives from changing angles ⁇ .
- ⁇ 0°
- the adaptation should not drift towards the 90° boundary but rather maintain the previously identified steering factor a .
- the underlying assumption is that an interferer does not move while being inactive.
- the fast version of the constant stepsize NLMS (Eq.
- the described differential microphone array (including the proposed enhancements) has been implemented on a signal processor of a wireless loudspeaker ( Binauric Boom Boom ) which is, at the same time, a handsfree communication device.
- a wireless loudspeaker Binauric Boom Boom
- the microphones offer SNRs of more than 60 dB which open up the possibility of a differential microphone array with a sufficiently low noise level.
- An example application scenario is a handsfree call in an office where another colleague is working on the opposite side of the desk. The colleague's noise (typing, voice, etc.) can then be canceled out when placing a call with Boom Boom.
- the signal processing software has been developed with the help of the RTProc rapid real-time prototyping framework (see H. Krüger and P. Vary) - the developer interface for algorithm parameterization is shown in Fig. 6 .
- a Matlab prototype based on framewise processing
- several other versions have been subsequently developed: A parameterizable C version, a C version with generated parameter tables, a C version based on fixed point arithmetic with an emulated instruction set and generated parameter tables, and finally optimized assembler code for the signal processor with generated parameter tables. All versions can be verified against each other and there is the possibility to step back to Matlab and add or modify features.
- a single unit or device may fulfill the functions of several items recited in the claims.
- the mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Landscapes
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Circuit For Audible Band Transducer (AREA)
Claims (15)
- Verfahren zum Erzeugen eines gerichteten Soundsignals (y(k)) aus ersten und zweiten Soundsignalen (x 1(k), x 2(k)), die durch ein erstes und ein zweites Mikrofon (M1, M2) erzeugt werden, welche durch einen Abstand (D) voneinander getrennt sind, wobei das Verfahren umfasst:- Erzeugen erster und zweiter differentieller Soundsignale (x f(k), x b,DEQ(k)) basierend auf den ersten und zweiten Soundsignalen (x 1(k), x 2(k)), und- Erzeugen des gerichteten Soundsignals (y(k)) entsprechend einem frequenzabhängigen gerichteten Antwortmuster basierend auf den ersten und zweiten differentiellen Soundsignalen (x f(k), x b,DEQ(k)),
wobei das Erzeugen des zweiten differentiellen Soundsignals (x b,DEQ(k)) ein Erzeugen eines Differenzsignals (x b(k)) des ersten und zweiten Soundsignals (x 1(k), x 2(k)) umfasst, sowie ein frequenzselektives Verarbeiten, das von einem Steuerwinkel (α) abhängt, der eine gewünschte Richtung einer maximalen Abschwächung des frequenzabhängigen gerichteten Antwortmusters angibt, wobei das frequenzselektive Verarbeiten die tatsächliche Richtung der maximalen Abschwächung des frequenzabhängigen gerichteten Antwortmusters so anpasst, dass sie im Wesentlichen unabhängig von der Frequenz (ω) über den Frequenzbereich des gerichteten Soundsignals (y(k)) zu dem Steuerwinkel (α) korrespondiert. - Verfahren nach Anspruch 1, wobei das frequenzselektive Verarbeiten ein Gewichten des Differenzsignals (x b(k)) mit einem approximierten Steuerfaktor (a lin(α)), der unabhängig von der Frequenz (ω) ist, umfasst, um ein gewichtetes Differenzsignal (a lin(α)·x b(k)) zu erzeugen, sowie ein Korrigieren der Approximation durch Hinzufügen eines Korrektursignals (P Δa (a lin(α))·h DEQ(k) ∗ x b(k)), das aus dem Differenzsignal (x b(k)) in Abhängigkeit der Frequenz (ω) und des Steuerwinkels (α) erzeugt wird.
- Verfahren nach Anspruch 2, wobei die Erzeugung des Korrektursignals (P Δa (a lin(α))·h DEQ(k) ∗ xb (k)) das Anwenden zweier separater Operationen umfasst, wobei eine abhängig von der Frequenz (ω) und unabhängig vom Steuerwinkel (α) ist und eine abhängig vom Steuerwinkel (α) aber unabhängig von der Frequenz (ω) ist.
- Verfahren nach Anspruch 2 oder 3, wobei die Erzeugung des Korrektursignals (P Δa (a lin(α))·h DEQ(k) ∗ xb (k)) ein Filtern des Differenzsignals (x b(k)) mit einem Filter (h DEQ(k)), der abhängig ist von der Frequenz (ω) und unabhängig vom Steuerwinkel (α), umfasst, um ein gefiltertes Differenzsignal zu erzeugen (h DEQ(k) * xb (k)).
- Verfahren nach Anspruch 4, wobei die Erzeugung des Korrektursignals (P Δa (a lin(α))·h DEQ(k) ∗ xb (k)) des Weiteren ein Gewichten des gefilterten Differenzsignals (h DEQ(k) ∗ xb (k)) mit einem Faktor (P Δa(a lin(α)), der abhängig ist vom Steuerwinkel (α) und unabhängig von der Frequenz (ω), umfasst.
- Verfahren nach Anspruch 5, wobei der Faktor (P Δa (a lin(α)) durch Verwenden einer polynominalen Approximation, die mit dem Steuerwinkel (α) oder dem approximierten Steuerfaktor (a lin(α)) evaluiert wird, bestimmt wird.
- Verfahren nach einem der Ansprüche 1 bis 6, wobei das Verfahren des Weiteren ein Filtern des gerichteten Soundsignals (y(k)) mit einem Tiefpass-Filter (h EQ(k)) umfasst, um ein gefiltertes gerichtetes Soundsignal (y EQ(k)) zu erzeugen.
- Verfahren nach einem der Ansprüche 2 bis 7, wobei der approximierte Steuerfaktor (a lin(α)) für einen Zeitpunkt (k - 1) für den folgenden Zeitpunkt (k) durch Hinzufügen eines Anpassungswertes, welcher durch einen Schrittweitenparameter (µ opt) skaliert ist, angepasst wird, wobei der Schrittweitenparameter (µ opt) in Abhängigkeit von geschätzten Energien von kohärenten und inkoheränten Soundkomponenten angepasst wird.
- Verfahren nach Anspruch 8, wobei die Energie der inkohärenten Soundkomponenten durch die geschätzte Kurzzeit-Energie
- Verfahren nach einem der Ansprüche 1 bis 9, wobei das Verfahren des Weiteren ein Schätzen eines relativen Gains (g c) der ersten und zweiten Mikrofone (M1, M2) umfasst, sowie ein Equalisieren von Leistungspegeln des ersten und des zweiten Mikrofons (M1, M2) basierend auf dem relativen Gain (g c).
- Verfahren nach einem der Ansprüche 1 bis 11, wobei das erste und das zweite Mikrofon (M1, M2) omnidirektionale Mikrophone sind.
- Vorrichtung (2) zum Erzeugen eines gerichteten Soundsignals (y(k)) aus ersten und zweiten Soundsignalen (x 1(k), x 2(k)), die durch ein erstes und ein zweites Mikrofon (M1, M2) erzeugt werden, welche durch einen Abstand (D) voneinander getrennt sind, wobei die Vorrichtung (2) umfasst:- erste Erzeugungsmittel zum Erzeugen erster und zweiter differentieller Soundsignale (x f(k), x b,DEQ(k)) basierend auf den ersten und zweiten Soundsignalen (x 1(k), x 2(k)), und- zweite Erzeugungsmittel zum Erzeugen des gerichteten Soundsignals (y(k)) entsprechend einem gerichteten Antwortmuster basierend auf den ersten und zweiten differentiellen Soundsignalen (x f(k), x b,DEQ(k)),
wobei das Erzeugen des zweiten differentiellen Soundsignals (x b,DEQ(k)) ein Erzeugen eines Differenzsignals (x b(k)) des ersten und zweiten Soundsignals (x 1(k), x 2(k)) umfasst, sowie ein frequenzselektives Verarbeiten, das von einem Steuerwinkel (α) abhängt, der eine gewünschte Richtung einer maximalen Abschwächung des frequenzabhängigen gerichteten Antwortmusters angibt, wobei das frequenzselektive Verarbeiten die tatsächliche Richtung der maximalen Abschwächung des frequenzabhängigen gerichteten Antwortmusters so anpasst, dass sie im Wesentlichen unabhängig von der Frequenz (ω) über den Frequenzbereich des gerichteten Soundsignals (y(k)) zu dem Steuerwinkel (α) korrespondiert. - System (1), wobei das System (1) umfasst:- ein erstes und ein zweites Mikrofon (M1, M2), welche durch einen Abstand (D) voneinander getrennt sind und erste und zweite Soundsignale (x 1(k), x 2(k)) erzeugen, und- eine Vorrichtung (2) wie im Anspruch 13 definiert.
- Computerprogramm umfassend Programmcodemittel, welches, wenn es auf einem Computer, der die Vorrichtung (2) nach Anspruch 13 steuert, ausgeführt wird, die Schritte des Verfahrens nach einem der Ansprüche 1 bis 12 ausführt.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2014/070243 WO2016045706A1 (en) | 2014-09-23 | 2014-09-23 | Method and apparatus for generating a directional sound signal from first and second sound signals |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3225037A1 EP3225037A1 (de) | 2017-10-04 |
EP3225037B1 true EP3225037B1 (de) | 2019-05-08 |
Family
ID=51585120
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14771598.1A Active EP3225037B1 (de) | 2014-09-23 | 2014-09-23 | Verfahren und vorrichtung zur erzeugung eines gerichteten tonsignals aus ersten und zweiten tonsignalen |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP3225037B1 (de) |
WO (1) | WO2016045706A1 (de) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220109511A1 (en) * | 2020-10-05 | 2022-04-07 | CUE Audio, LLC | Method and system for digital communication over an acoustic channel |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7139628B2 (ja) * | 2018-03-09 | 2022-09-21 | ヤマハ株式会社 | 音処理方法および音処理装置 |
GB2575491A (en) * | 2018-07-12 | 2020-01-15 | Centricam Tech Limited | A microphone system |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9202475B2 (en) * | 2008-09-02 | 2015-12-01 | Mh Acoustics Llc | Noise-reducing directional microphone ARRAYOCO |
-
2014
- 2014-09-23 WO PCT/EP2014/070243 patent/WO2016045706A1/en active Application Filing
- 2014-09-23 EP EP14771598.1A patent/EP3225037B1/de active Active
Non-Patent Citations (1)
Title |
---|
None * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220109511A1 (en) * | 2020-10-05 | 2022-04-07 | CUE Audio, LLC | Method and system for digital communication over an acoustic channel |
US11728905B2 (en) * | 2020-10-05 | 2023-08-15 | CUE Audio, LLC | Method and system for digital communication over an acoustic channel |
Also Published As
Publication number | Publication date |
---|---|
WO2016045706A1 (en) | 2016-03-31 |
EP3225037A1 (de) | 2017-10-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5805365B2 (ja) | ノイズ推定装置及び方法とそれを利用したノイズ減少装置 | |
US9723422B2 (en) | Multi-microphone method for estimation of target and noise spectral variances for speech degraded by reverberation and optionally additive noise | |
US8194880B2 (en) | System and method for utilizing omni-directional microphones for speech enhancement | |
EP2207168B1 (de) | Robustes Rauschunterdrückungssystem mit zwei Mikrophonen | |
CN111128210B (zh) | 具有声学回声消除的音频信号处理的方法和系统 | |
US9060052B2 (en) | Single channel, binaural and multi-channel dereverberation | |
JP5444472B2 (ja) | 音源分離装置、音源分離方法、及び、プログラム | |
EP2308044B1 (de) | Audioverarbeitung | |
EP2245861B1 (de) | Verbesserter blindquellen-separationsalgorithmus für hochkorrelierte mischungen | |
EP3462452A1 (de) | Geräuschschätzung zur verwendung mit geräuschreduzierung und echounterdrückung in persönlicher kommunikation | |
Dietzen et al. | Integrated sidelobe cancellation and linear prediction Kalman filter for joint multi-microphone speech dereverberation, interfering speech cancellation, and noise reduction | |
EP3692529B1 (de) | Vorrichtung und verfahren zur signalverbesserung | |
EP3225037B1 (de) | Verfahren und vorrichtung zur erzeugung eines gerichteten tonsignals aus ersten und zweiten tonsignalen | |
US11373668B2 (en) | Enhancement of audio from remote audio sources | |
TWI465121B (zh) | 利用全方向麥克風改善通話的系統及方法 | |
Xiao et al. | Effect of target signals and delays on spatially selective active noise control for open-fitting hearables | |
Stenzel et al. | A multichannel Wiener filter with partial equalization for distributed microphones | |
US11019433B2 (en) | Beam former, beam forming method and hearing aid system | |
Modhave et al. | Design of multichannel wiener filter for speech enhancement in hearing aids and noise reduction technique | |
Barfuss et al. | Informed spatial filtering based on constrained independent component analysis | |
Dietzen et al. | Speech dereverberation by data-dependent beamforming with signal pre-whitening | |
Geiser et al. | A differential microphone array with input level alignment, directional equalization and fast notch adaptation for handsfree communication | |
Masuyama et al. | Causal distortionless response beamforming by alternating direction method of multipliers | |
Lombard et al. | Combination of adaptive feedback cancellation and binaural adaptive filtering in hearing aids | |
Yong et al. | Effective binaural multi-channel processing algorithm for improved environmental presence |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20170810 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20181121 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 1132097 Country of ref document: AT Kind code of ref document: T Effective date: 20190515 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602014046410 Country of ref document: DE Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20190508 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190908 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190808 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190809 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190808 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1132097 Country of ref document: AT Kind code of ref document: T Effective date: 20190508 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602014046410 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
26N | No opposition filed |
Effective date: 20200211 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190923 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190923 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20190930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190908 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20140923 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190508 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602014046410 Country of ref document: DE |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240226 Year of fee payment: 10 Ref country code: GB Payment date: 20240227 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240227 Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602014046410 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20240923 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20250401 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240923 |