EP3639261A4 - CONTEXTUAL APPLICATIONS IN A MIXED REALITY ENVIRONMENT - Google Patents

CONTEXTUAL APPLICATIONS IN A MIXED REALITY ENVIRONMENT Download PDF

Info

Publication number: EP3639261A4
Authority: EP; European Patent Office
Prior art keywords: mixed reality; reality environment; contextual applications; contextual; applications
Prior art date: 2017-05-05
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Granted

Application number

EP18794296.6A

Other languages

German (de)

French (fr)

Other versions

EP3639261B1 (en

EP3639261A1 (en

Inventor

Sylvio Herve Drouin

Gregory Lionel Xavier Jean Palmaro

Dioselin Alejandra Gonzalez ROSILLO

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

Unity IPR ApS

Original Assignee

Unity IPR ApS

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2017-05-05

Filing date

2018-05-04

Publication date

2020-05-13

2018-05-04 Application filed by Unity IPR ApS filed Critical Unity IPR ApS

2020-04-22 Publication of EP3639261A1 publication Critical patent/EP3639261A1/en

2020-05-13 Publication of EP3639261A4 publication Critical patent/EP3639261A4/en

2023-08-30 Application granted granted Critical

2023-08-30 Publication of EP3639261B1 publication Critical patent/EP3639261B1/en

Status Active legal-status Critical Current

2038-05-04 Anticipated expiration legal-status Critical

Classifications

- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Landscapes

Engineering & Computer Science (AREA)
Physics & Mathematics (AREA)
Theoretical Computer Science (AREA)
General Engineering & Computer Science (AREA)
General Physics & Mathematics (AREA)
Human Computer Interaction (AREA)
Optics & Photonics (AREA)
Computer Hardware Design (AREA)
Computer Graphics (AREA)
Software Systems (AREA)
Multimedia (AREA)
User Interface Of Digital Computer (AREA)
Processing Or Creating Images (AREA)
Controls And Circuits For Display Device (AREA)
Image Analysis (AREA)

EP18794296.6A 2017-05-05 2018-05-04 Contextual applications in a mixed reality environment Active EP3639261B1 (en)

Applications Claiming Priority (3)

Application Number	Priority Date	Filing Date	Title
US201762502349P	2017-05-05	2017-05-05
US201762561017P	2017-09-20	2017-09-20
PCT/US2018/031245 WO2018204879A1 (en)	2017-05-05	2018-05-04	Contextual applications in a mixed reality environment

Publications (3)

Publication Number	Publication Date
EP3639261A1 EP3639261A1 (en)	2020-04-22
EP3639261A4 true EP3639261A4 (en)	2020-05-13
EP3639261B1 EP3639261B1 (en)	2023-08-30

Family

ID=64014833

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
EP18794296.6A Active EP3639261B1 (en)	2017-05-05	2018-05-04	Contextual applications in a mixed reality environment

Country Status (7)

Country	Link
US (3)	US10360734B2 (en)
EP (1)	EP3639261B1 (en)
JP (1)	JP6828189B2 (en)
KR (1)	KR102220949B1 (en)
CN (1)	CN111033605A (en)
CA (1)	CA3062541C (en)
WO (1)	WO2018204879A1 (en)

Families Citing this family (59)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
CN104740869B (en) *	2015-03-26	2018-04-03	北京小小牛创意科技有限公司	The exchange method and system that a kind of actual situation for merging true environment combines
JP2018005091A (en) *	2016-07-06	2018-01-11	富士通株式会社	Display control program, display control method and display controller
US10867445B1 (en) *	2016-11-16	2020-12-15	Amazon Technologies, Inc.	Content segmentation and navigation
WO2018204879A1 (en)	2017-05-05	2018-11-08	Unity IPR ApS	Contextual applications in a mixed reality environment
US10559133B2 (en) *	2018-02-07	2020-02-11	Dell Products L.P.	Visual space management across information handling system and augmented reality
CN108769814B (en) *	2018-06-01	2022-02-01	腾讯科技（深圳）有限公司	Video interaction method, device, terminal and readable storage medium
US20190378334A1 (en) *	2018-06-08	2019-12-12	Vulcan Inc.	Augmented reality portal-based applications
WO2019237085A1 (en)	2018-06-08	2019-12-12	Vulcan Inc.	Session-based information exchange
US20190385372A1 (en) *	2018-06-15	2019-12-19	Microsoft Technology Licensing, Llc	Positioning a virtual reality passthrough region at a known distance
US10712901B2 (en)	2018-06-27	2020-07-14	Facebook Technologies, Llc	Gesture-based content sharing in artificial reality environments
US10783712B2 (en) *	2018-06-27	2020-09-22	Facebook Technologies, Llc	Visual flairs for emphasizing gestures in artificial-reality environments
US10635895B2 (en)	2018-06-27	2020-04-28	Facebook Technologies, Llc	Gesture-based casting and manipulation of virtual content in artificial-reality environments
US10996831B2 (en)	2018-06-29	2021-05-04	Vulcan Inc.	Augmented reality cursors
CN113272766A (en) *	2019-01-24	2021-08-17	麦克赛尔株式会社	Display terminal, application control system, and application control method
US11054896B1 (en) *	2019-02-07	2021-07-06	Facebook, Inc.	Displaying virtual interaction objects to a user on a reference plane
JP7157871B2 (en) *	2019-03-08	2022-10-20	株式会社Ｎｔｔドコモ	Related information output device
DE102019106281A1 (en) *	2019-03-12	2020-09-17	tooz technologies GmbH	System, method, computer program, mobile device and kit for operating a device with low computing power
US10810782B1 (en) *	2019-04-01	2020-10-20	Snap Inc.	Semantic texture mapping system
US11804014B1 (en) *	2019-04-22	2023-10-31	Apple Inc.	Context-based application placement
US11163997B2 (en) *	2019-05-05	2021-11-02	Google Llc	Methods and apparatus for venue based augmented reality
US10897564B1 (en)	2019-06-17	2021-01-19	Snap Inc.	Shared control of camera device by multiple devices
US11922690B2 (en) *	2019-06-25	2024-03-05	Semiconductor Energy Laboratory Co., Ltd.	Data processing system and data processing method
US11100715B2 (en) *	2019-06-26	2021-08-24	International Business Machines Corporation	Establishment of positional timers in an augmented reality environment
US11340857B1 (en)	2019-07-19	2022-05-24	Snap Inc.	Shared control of a virtual object by multiple devices
US11176744B2 (en) *	2019-07-22	2021-11-16	Microsoft Technology Licensing, Llc	Mapping sensor data using a mixed-reality cloud
IT201900015485A1 (en) *	2019-09-03	2021-03-03	Ali Group Srl Carpigiani	SUPPORT SYSTEM FOR THE MANAGEMENT OF A FOOD PROCESSING MACHINE AND CORRESPONDING PROCEDURE.
US11461867B2 (en) *	2019-09-19	2022-10-04	Sanctuary Cognitive Systems Corporation	Visual interface and communications techniques for use with robots
CN110716645A (en) *	2019-10-15	2020-01-21	北京市商汤科技开发有限公司	Augmented reality data presentation method and device, electronic equipment and storage medium
US11270115B2 (en) *	2019-11-18	2022-03-08	Lenovo (Singapore) Pte. Ltd.	Presentation of augmented reality content based on identification of trigger accompanying video content
US11461115B2 (en) *	2020-01-28	2022-10-04	Sap Se	Third party executable asset bundle deployment
US11354867B2 (en) *	2020-03-04	2022-06-07	Apple Inc.	Environment application model
US12101360B2 (en)	2020-03-25	2024-09-24	Snap Inc.	Virtual interaction session to facilitate augmented reality based communication between multiple users
US11985175B2 (en)	2020-03-25	2024-05-14	Snap Inc.	Virtual interaction session to facilitate time limited augmented reality based communication between multiple users
US12182903B2 (en)	2020-03-25	2024-12-31	Snap Inc.	Augmented reality based communication between multiple users
US11593997B2 (en) *	2020-03-31	2023-02-28	Snap Inc.	Context based augmented reality communication
CN111462663B (en) *	2020-06-19	2020-10-27	南京新研协同定位导航研究院有限公司	Tour guide method based on MR glasses
CN111539662A (en) *	2020-07-10	2020-08-14	南京新研协同定位导航研究院有限公司	Dynamic inventory allocation method based on MR (magnetic resonance) glasses leasing business
EP3964929A1 (en) *	2020-09-08	2022-03-09	Koninklijke Philips N.V.	Controlling a 2d screen interface application in a mixed reality application
CN116076063A (en)	2020-09-09	2023-05-05	斯纳普公司	Augmented reality messenger system
KR20230066448A (en) *	2020-09-16	2023-05-15	스냅 인코포레이티드	Context-triggered augmented reality
US11617953B2 (en)	2020-10-09	2023-04-04	Contact Control Interfaces, Llc.	Virtual object interaction scripts
EP3995191A1 (en) *	2020-11-04	2022-05-11	Unity IPR APS	Method and system for merging distant spaces
JP6975489B1 (en) *	2020-12-18	2021-12-01	株式会社Gatari	Information processing system, information processing method and information processing program
KR20220121990A (en) *	2021-02-26	2022-09-02	삼성전자주식회사	Augmented Reality Devices and Electronic Devices Interacting with Augmented Reality Devices
US12056416B2 (en)	2021-02-26	2024-08-06	Samsung Electronics Co., Ltd.	Augmented reality device and electronic device interacting with augmented reality device
MX2023010660A (en) *	2021-03-12	2023-10-11	Christopher faraguna	Systems and methods for user data collection within an augmented reality game.
US20220319124A1 (en) *	2021-03-31	2022-10-06	Snap Inc.	Auto-filling virtual content
US11935199B2 (en) *	2021-07-26	2024-03-19	Google Llc	Augmented reality depth detection through object recognition
KR20240056558A (en)	2021-09-01	2024-04-30	스냅 인코포레이티드	Handcrafted Augmented Reality Experiences
WO2023034010A1 (en)	2021-09-01	2023-03-09	Snap Inc.	Physical action-based augmented reality communication exchanges
US20230066686A1 (en) *	2021-09-02	2023-03-02	Snap Inc.	Augmented reality prop interactions
US11969650B2 (en) *	2021-11-18	2024-04-30	Sony Interactive Entertainment Inc.	Feature similarity scoring of physical environment for augmented reality gameplay
US11568616B1 (en) *	2021-11-29	2023-01-31	Varjo Technologies Oy	Display apparatuses and methods for facilitating location-based virtual content
CN114971764A (en) *	2022-04-08	2022-08-30	浙江赟燊商业信息系统科技有限公司	HoloLens-based storage and matching system and method
JP7482943B2 (en)	2022-06-17	2024-05-14	ソフトバンク株式会社	Information processing device, program, system, and information processing method
US12020388B2 (en) *	2022-07-29	2024-06-25	Maplebear Inc.	Displaying an augmented reality element that provides a personalized enhanced experience at a warehouse
US20240220006A1 (en) *	2022-12-29	2024-07-04	Skonec Entertainment Co., Ltd.	Virtual reality control system
WO2024218003A1 (en) *	2023-04-17	2024-10-24	Interdigital Ce Patent Holdings, Sas	Graph representation of events for interactive environments
US20250036371A1 (en) *	2023-07-25	2025-01-30	simpleAR, Inc.	Serializing and deserializing mixed reality experiences or portions thereof

Citations (3)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US20130182012A1 (en) *	2012-01-12	2013-07-18	Samsung Electronics Co., Ltd.	Method of providing augmented reality and terminal supporting the same
US20130303285A1 (en) *	2012-05-11	2013-11-14	Sony Computer Entertainment Europe Limited	Apparatus and method for augmented reality
US20170097676A1 (en) *	2015-10-01	2017-04-06	Disney Enterprises, Inc.	Augmented Reality Controls for User Interactions with a Virtual World

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US9285589B2 (en) *	2010-02-28	2016-03-15	Microsoft Technology Licensing, Llc	AR glasses with event and sensor triggered control of AR eyepiece applications
US20110213664A1 (en) *	2010-02-28	2011-09-01	Osterhout Group, Inc.	Local advertising content on an interactive head-mounted eyepiece
JP2013004001A (en) *	2011-06-21	2013-01-07	Sony Corp	Display control device, display control method, and program
US10019962B2 (en) *	2011-08-17	2018-07-10	Microsoft Technology Licensing, Llc	Context adaptive user interface for augmented reality display
KR102062763B1 (en) *	2012-12-07	2020-01-07	삼성전자주식회사	Method and system for providing information based on context, and computer readable recording medium thereof
US9924102B2 (en) *	2013-03-14	2018-03-20	Qualcomm Incorporated	Image-based application launcher
US9240075B2 (en) *	2013-03-15	2016-01-19	Daqri, Llc	Campaign optimization for experience content dataset
CN203250508U (en) *	2013-04-24	2013-10-23	苏州创捷传媒展览股份有限公司	Three-dimensional mixed reality display device
US9536355B1 (en) *	2016-03-24	2017-01-03	Daqri, Llc	Thermal detection in an augmented reality system
CN105912130A (en) *	2016-07-06	2016-08-31	深圳云戈科技有限公司	Communication method, device and system based on mixed reality
WO2018204879A1 (en)	2017-05-05	2018-11-08	Unity IPR ApS	Contextual applications in a mixed reality environment

2018
- 2018-05-04 WO PCT/US2018/031245 patent/WO2018204879A1/en active Application Filing
- 2018-05-04 EP EP18794296.6A patent/EP3639261B1/en active Active
- 2018-05-04 CA CA3062541A patent/CA3062541C/en active Active
- 2018-05-04 US US15/972,020 patent/US10360734B2/en active Active
- 2018-05-04 JP JP2019560385A patent/JP6828189B2/en active Active
- 2018-05-04 CN CN201880044613.1A patent/CN111033605A/en active Pending
- 2018-05-04 KR KR1020197035793A patent/KR102220949B1/en not_active Expired - Fee Related
2019
- 2019-06-27 US US16/455,493 patent/US10769857B2/en active Active
2020
- 2020-07-28 US US16/941,298 patent/US10984604B2/en active Active

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US20130182012A1 (en) *	2012-01-12	2013-07-18	Samsung Electronics Co., Ltd.	Method of providing augmented reality and terminal supporting the same
US20130303285A1 (en) *	2012-05-11	2013-11-14	Sony Computer Entertainment Europe Limited	Apparatus and method for augmented reality
US20170097676A1 (en) *	2015-10-01	2017-04-06	Disney Enterprises, Inc.	Augmented Reality Controls for User Interactions with a Virtual World

Also Published As

Publication number	Publication date
US10769857B2 (en)	2020-09-08
KR102220949B1 (en)	2021-03-02
EP3639261B1 (en)	2023-08-30
KR20190141777A (en)	2019-12-24
CN111033605A (en)	2020-04-17
JP2020520000A (en)	2020-07-02
CA3062541A1 (en)	2018-11-08
US10360734B2 (en)	2019-07-23
JP6828189B2 (en)	2021-02-10
WO2018204879A1 (en)	2018-11-08
CA3062541C (en)	2022-01-11
US10984604B2 (en)	2021-04-20
US20180322706A1 (en)	2018-11-08
US20190392645A1 (en)	2019-12-26
EP3639261A1 (en)	2020-04-22
US20200357187A1 (en)	2020-11-12

Publication	Publication Date	Title
EP3639261A4 (en)	2020-05-13	CONTEXTUAL APPLICATIONS IN A MIXED REALITY ENVIRONMENT
EP3371690A4 (en)	2019-07-24	INTELLIGENT DIGITAL ASSISTANT IN A MULTITASK ENVIRONMENT
IL268397A (en)	2019-09-26	Drawing a virtual and real object on a mixed reality device
GB201717042D0 (en)	2017-11-29	Facilitating debugging serverless applications via graph rewriting
GB201704508D0 (en)	2017-05-03	Contextual task shortcuts
EP3465331A4 (en)	2020-01-29	MIXED REALITY SYSTEM
EP3566143A4 (en)	2020-01-22	TEST SOFTWARE CODE IN A PRODUCTION ENVIRONMENT
EP3574401A4 (en)	2021-01-27	RECONFIGURABLE, APPLICATION-SPECIFIC COMPUTER ACCELERATOR
EP3605134A4 (en)	2020-12-30	RADAR DEVICE
EP3594717A4 (en)	2020-12-30	RADAR DEVICE
EP3657209A4 (en)	2020-08-05	RADAR DEVICE
EP3454146A4 (en)	2020-08-26	PROCESSING STATUS DISPLAY DEVICE
EP3509589A4 (en)	2020-03-18	NEW APPLICATIONS
EP3434358A4 (en)	2019-07-03	LIQUÉFACTION MIXER
EP3801344A4 (en)	2022-03-09	DISPOSABLE STICKER IN AUGMENTED REALITY ENVIRONMENT
EP3323564A4 (en)	2019-03-13	HYDRAULIC ROADING DEVICE
ES1195485Y (en)	2018-01-17	COMPACT MOTION SIMULATOR
KR20180084875A (en)	2018-07-25	Static mixer
EP3708324A4 (en)	2021-02-17	MIXING DEVICE
EP3688112A4 (en)	2021-06-23	SINGLE-COMPONENT POLYMER MODIFICATORS
BR112016025075A2 (en)	2018-05-15	agrochemical formulation.
EP3347554A4 (en)	2019-04-24	PRE-DEVELOPED FLAT PACKING BRIDGE
EP3708323A4 (en)	2021-01-13	MIXING DEVICE
DK3482821T3 (en)	2021-05-03	STIRS
EP3647918A4 (en)	2021-04-07	INPUT DEVICE

Legal Events

Date	Code	Title	Description
2018-11-09	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE
2020-03-20	PUAI	Public reference made under article 153(3) epc to a published international application that has entered the european phase	Free format text: ORIGINAL CODE: 0009012
2020-03-20	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE
2020-04-03	REG	Reference to a national code	Ref country code: DE Ref legal event code: R079 Ref document number: 602018056519 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G09G0005000000 Ipc: G06F0003010000 Ref country code: DE Ref legal event code: R079 Ipc: G06F0003010000
2020-04-22	17P	Request for examination filed	Effective date: 20191128
2020-04-22	AK	Designated contracting states	Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
2020-04-22	AX	Request for extension of the european patent	Extension state: BA ME
2020-05-13	A4	Supplementary search report drawn up and despatched	Effective date: 20200409
2020-05-13	RIC1	Information provided on ipc code assigned before grant	Ipc: G09G 5/00 20060101ALI20200403BHEP Ipc: G06F 3/01 20060101AFI20200403BHEP Ipc: G06T 19/00 20110101ALI20200403BHEP Ipc: G02B 27/01 20060101ALI20200403BHEP
2020-08-12	DAV	Request for validation of the european patent (deleted)
2020-08-12	DAX	Request for extension of the european patent (deleted)
2021-10-22	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: EXAMINATION IS IN PROGRESS
2021-11-24	17Q	First examination report despatched	Effective date: 20211021
2023-03-19	GRAP	Despatch of communication of intention to grant a patent	Free format text: ORIGINAL CODE: EPIDOSNIGR1
2023-03-19	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: GRANT OF PATENT IS INTENDED
2023-04-19	INTG	Intention to grant announced	Effective date: 20230320
2023-04-19	RAP3	Party data changed (applicant data changed or rights of an application transferred)	Owner name: UNITY IPR APS
2023-04-19	RIN1	Information on inventor provided before grant (corrected)	Inventor name: ROSILLO, DIOSELIN ALEJANDRA GONZALEZ Inventor name: PALMARO, GREGORY LIONEL XAVIER JEAN Inventor name: DROUIN, SYLVIO HERVE
2023-07-12	P01	Opt-out of the competence of the unified patent court (upc) registered	Effective date: 20230605
2023-07-19	GRAS	Grant fee paid	Free format text: ORIGINAL CODE: EPIDOSNIGR3
2023-07-28	GRAA	(expected) grant	Free format text: ORIGINAL CODE: 0009210
2023-07-28	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: THE PATENT HAS BEEN GRANTED
2023-08-30	AK	Designated contracting states	Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
2023-08-30	REG	Reference to a national code	Ref country code: GB Ref legal event code: FG4D
2023-08-31	REG	Reference to a national code	Ref country code: CH Ref legal event code: EP
2023-09-21	REG	Reference to a national code	Ref country code: DE Ref legal event code: R096 Ref document number: 602018056519 Country of ref document: DE
2023-09-27	REG	Reference to a national code	Ref country code: IE Ref legal event code: FG4D
2023-12-27	REG	Reference to a national code	Ref country code: LT Ref legal event code: MG9D
2024-01-03	REG	Reference to a national code	Ref country code: NL Ref legal event code: MP Effective date: 20230830
2024-01-15	REG	Reference to a national code	Ref country code: AT Ref legal event code: MK05 Ref document number: 1606261 Country of ref document: AT Kind code of ref document: T Effective date: 20230830
2024-01-18	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231201
2024-01-23	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231230
2024-01-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231130 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231230 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231201 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-02-29	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-04-26	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-04-30	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240102
2024-05-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-06-03	REG	Reference to a national code	Ref country code: DE Ref legal event code: R097 Ref document number: 602018056519 Country of ref document: DE
2024-07-05	PLBE	No opposition filed within time limit	Free format text: ORIGINAL CODE: 0009261
2024-07-05	STAA	Information on the status of an ep patent application or granted ep patent	Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT
2024-07-08	PGFP	Annual fee paid to national office [announced via postgrant information from national office to epo]	Ref country code: GB Payment date: 20240408 Year of fee payment: 7
2024-07-10	PGFP	Annual fee paid to national office [announced via postgrant information from national office to epo]	Ref country code: DE Payment date: 20240410 Year of fee payment: 7
2024-07-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-08-07	26N	No opposition filed	Effective date: 20240603
2024-11-08	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-11-29	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2024-12-30	REG	Reference to a national code	Ref country code: CH Ref legal event code: PL
2025-01-08	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830
2025-01-16	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240504
2025-01-31	PG25	Lapsed in a contracting state [announced via postgrant information from national office to epo]	Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230830 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240504 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240531
2025-02-21	REG	Reference to a national code	Ref country code: BE Ref legal event code: MM Effective date: 20240531