KR20240041617A - Method and apparatus for classifying pose using uwb(ultra-wideband) signal - Google Patents

Abstract

The present disclosure discloses a method for an electronic device to classify a pose using a UWB signal. The method of the present disclosure includes obtaining first prediction data for classifying whether a signal corresponding to UWB CIR is a LOS signal or an NLOS signal based on UWB CIR data, using a learned first CNN model, the signal Obtaining second prediction data for classifying a pose based on sensor data corresponding to UWB CIR data using a learned second CNN model when classified as a LOS signal, learned when the signal is classified as an NLOS signal Obtaining third prediction data for classifying a pose based on sensor data corresponding to UWB CIR data using a third CNN model, using a first filtering method to obtain first prediction data, second prediction data, and 3. It may include performing filtering on at least two of the prediction data, and classifying the pose based on the filtered data.

Description

The present disclosure provides a method and device for classifying poses using ultra-wideband communication signals {METHOD AND APPARATUS FOR CLASSIFYING POSE USING UWB (ULTRA-WIDEBAND) SIGNAL}

This disclosure relates to a method and device for classifying poses using UWB signals.

The Internet is evolving from a human-centered network where humans create and consume information to an IoT (Internet of Things) network that exchanges and processes information between distributed components such as objects. IoE (Internet of Everything) technology, which combines IoT technology with big data processing technology through connection to cloud servers, etc., is also emerging. To implement IoT, technological elements such as sensing technology, wired and wireless communication and network infrastructure, service interface technology, and security technology are required. Recently, technologies such as sensor network, Machine to Machine (M2M), and MTC (Machine Type Communication) for connection between objects are being researched.

In an IoT environment, intelligent IT (Internet Technology) services that create new value in human life can be provided by collecting and analyzing data generated from connected objects. IoT, through the convergence and combination of existing IT (information technology) technology and various industries, includes smart homes, smart buildings, smart cities, smart cars or connected cars, smart grids, health care, smart home appliances, advanced medical services, etc. It can be applied in the field of

The present disclosure provides a method for classifying signals as line of sight (LOS)/non-LOS (NLOS) signals using UWB channel impulse response (CIR) data.

Additionally, the present disclosure provides a method for classifying poses using signal classification results and sensor data.

According to various embodiments of the present disclosure, a method of an electronic device uses a learned first convolutional neural network (CNN) model to generate a signal corresponding to the UWB CIR based on LOS data. Obtaining first prediction data to classify whether it is a (line of signt) signal or a non-LOS (NLOS) signal; If the signal is classified as the LOS signal, using a learned second CNN model, obtaining second prediction data for classifying a pose based on sensor data corresponding to the UWB CIR data; If the signal is classified as the NLOS signal, using a learned third CNN model, obtaining third prediction data for classifying a pose based on sensor data corresponding to the UWB CIR data; performing filtering on at least two of the first prediction data, the second prediction data, and the third prediction data using a first filtering method; and classifying the pose based on the filtered data, wherein the second CNN model is learned using sensor data associated with the LOS signal, and the third CNN model is trained using sensor data associated with the NLOS signal. It can be learned using data.

According to various embodiments of the present disclosure, an electronic device includes a transceiver; and a controller connected to the transceiver, wherein the controller: uses a learned first convolutional neural network (CNN) model to generate a signal corresponding to the UWB CIR based on UWB channel impulse response (CIR) data (LOS (LOS)). Obtain first prediction data to classify whether it is a line of signt (NLOS) signal or a non-LOS (NLOS) signal, and if the signal is classified as the LOS signal, use the learned second CNN model to obtain the UWB CIR Obtain second prediction data for classifying a pose based on sensor data corresponding to the data, and when the signal is classified as the NLOS signal, use a learned third CNN model to obtain second prediction data corresponding to the UWB CIR data. Obtaining third prediction data for classifying a pose based on sensor data, and filtering at least two of the first prediction data, the second prediction data, and the third prediction data using a first filtering method. and classify the pose based on the filtered data, wherein the second CNN model is learned using sensor data associated with the LOS signal, and the third CNN model is trained using sensor data associated with the NLOS signal. It can be learned using sensor data.

According to the method of the present disclosure, the accuracy of LOS/NLOS signal classification can be improved.

Additionally, according to the method of the present disclosure, the accuracy of pose classification can be improved.

1 is a block diagram schematically showing an electronic device.
2A shows an example architecture of a UWB device according to one embodiment of the present disclosure.
FIG. 2B shows an example configuration of a framework of a UWB device according to an embodiment of the present disclosure.
Figure 3A shows an example of UWB CIR data according to an embodiment of the present disclosure.
Figure 3b shows an example of effective UWB CIR data according to an embodiment of the present disclosure.
Figure 3c shows an example of a LOS signal and a NLOS signal classified using UWB CIR data according to an embodiment of the present disclosure.
Figure 4 shows DS-TWR according to an embodiment of the present disclosure.
Figure 5 shows DL-TDoA according to an embodiment of the present disclosure.
Figure 6 shows a learning procedure and distribution procedure for LOS/NLOS signal classification using UWB channel impulse response data according to an embodiment of the present disclosure.
Figure 7 shows the configuration of a user device for LOS/NLOS classification according to an embodiment of the present disclosure.
Figure 8 shows a learning procedure for pose classification based on LOS/NLOS signal classification using UWB channel impulse response data according to an embodiment of the present disclosure.
Figure 9 shows a distribution procedure for pose classification based on LOS/NLOS signal classification using UWB channel impulse response data according to an embodiment of the present disclosure.
Figure 10 shows the configuration of a user device for pose classification using LOS/NLOS classification according to an embodiment of the present disclosure.
Figure 11 shows a method for classifying a pose of an electronic device according to an embodiment of the present disclosure.
FIG. 12 is a diagram illustrating the structure of an electronic device according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

In describing the embodiments, description of technical content that is well known in the technical field to which this disclosure belongs and that is not directly related to this disclosure will be omitted. This is to convey the gist of the present disclosure more clearly without obscuring it by omitting unnecessary explanation.

For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically shown. Additionally, the size of each component does not entirely reflect its actual size. In each drawing, identical or corresponding components are assigned the same reference numbers.

The advantages and features of the present disclosure and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, and the embodiments of the present disclosure are merely intended to ensure that the present disclosure is complete and that common knowledge in the technical field to which the present disclosure pertains is provided. It is provided to fully inform those who have the scope of the disclosure, and the disclosure is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagram diagrams can be performed by computer program instructions. These computer program instructions can be mounted on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment, so that the instructions performed through the processor of the computer or other programmable data processing equipment are described in the flow chart block(s). It creates the means to perform functions. These computer program instructions may also be stored in computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular manner, so that the computer-usable or computer-readable memory The instructions stored in may also be capable of producing manufactured items containing instruction means to perform the functions described in the flow diagram block(s). Computer program instructions can also be mounted on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a process that is executed by the computer, thereby generating a process that is executed by the computer or other programmable data processing equipment. Instructions that perform processing equipment may also provide steps for executing the functions described in the flow diagram block(s).

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). Additionally, it should be noted that in some alternative execution examples it is possible for the functions mentioned in the blocks to occur out of order. For example, it is possible for two blocks shown in succession to be performed substantially at the same time, or it may be possible for the blocks to be performed in reverse order depending on the corresponding function.

At this time, the term '~unit' used in this embodiment refers to software or hardware components such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and '~unit' performs certain roles. do. However, '~part' is not limited to software or hardware. The '~ part' may be configured to reside in an addressable storage medium and may be configured to reproduce on one or more processors. Therefore, according to some embodiments, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and processes. Includes scissors, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card. Additionally, according to some embodiments, '~unit' may include one or more processors.

The term 'terminal' or 'device' used in this specification refers to a mobile station (MS), user equipment (UE), user terminal (UT), wireless terminal, access terminal (AT), terminal, and subscriber unit. It may be referred to as a Subscriber Unit, a Subscriber Station (SS), a wireless device, a wireless communication device, a Wireless Transmit/Receive Unit (WTRU), a mobile node, a mobile, or other terms. . Various embodiments of the terminal include a cellular phone, a smart phone with a wireless communication function, a personal digital assistant (PDA) with a wireless communication function, a wireless modem, a portable computer with a wireless communication function, and a digital camera with a wireless communication function. It may include devices, gaming devices with wireless communication functions, music storage and playback home appliances with wireless communication functions, Internet home appliances capable of wireless Internet access and browsing, as well as portable units or terminals that integrate combinations of such functions. there is. Additionally, the terminal may include, but is not limited to, an M2M (Machine to Machine) terminal and an MTC (Machine Type Communication) terminal/device. In this specification, the terminal may be referred to as an electronic device or simply a device.

Hereinafter, the operating principle of the present disclosure will be described in detail with reference to the attached drawings. In the following description of the present disclosure, if a detailed description of a related known function or configuration is determined to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of the functions in the present disclosure, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, embodiments of the present disclosure will be described in detail with the accompanying drawings. Hereinafter, embodiments of the present disclosure will be described using a communication system using UWB as an example, but embodiments of the present disclosure may also be applied to other communication systems with similar technical background or characteristics. For example, a communication system using Bluetooth or ZigBee may be included. Accordingly, the embodiments of the present disclosure may be applied to other communication systems through some modifications without significantly departing from the scope of the present disclosure at the discretion of a person with skilled technical knowledge.

Additionally, when describing the present disclosure, if it is determined that a detailed description of a related function or configuration may unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of the functions in the present disclosure, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

In general, wireless sensor network technology is largely divided into wireless LAN (Wireless Local Area Network; WLAN) technology and wireless personal area network (WPAN) technology depending on recognition distance. At this time, wireless LAN is a technology based on IEEE 802.11 and is a technology that can connect to the backbone network within a radius of about 100m. Additionally, wireless private networks are technologies based on IEEE 802.15 and include Bluetooth, ZigBee, and ultra wide band (UWB). A wireless network in which this wireless network technology is implemented may be comprised of a plurality of electronic devices.

According to the definition of the Federal Communications Commission (FCC), UWB can refer to a wireless communication technology that uses a bandwidth of 500 MHz or more, or has a bandwidth of 20% or more corresponding to the center frequency. UWB may also refer to the band itself to which UWB communication is applied. UWB enables safe and accurate ranging between devices. Through this, UWB enables relative position estimation based on the distance between two devices or accurate position estimation of a device based on its distance from fixed devices (where the position is known).

Specific terms used in the following description are provided to aid understanding of the present disclosure, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the present disclosure.

* “Application Dedicated File (ADF)” may be, for example, a data structure within the Application Data Structure that can host an application or application specific data.

“Application Protocol Data Unit (APDU)” may be a command and response used when communicating with the Application Data Structure within a UWB device.

“Application specific data” may be, for example, a file structure with a root level and an application level containing UWB control information and UWB session data required for a UWB session.

“Controller” may be a Ranging Device that defines and controls Ranging Control Messages (RCM) (or control messages). The controller can define and control ranging features by transmitting a control message.

“Controlee” may be a ranging device that uses ranging parameters in the RCM (or control message) received from the controller. Controlee can use the same ranging features as set through control messages from Controller.

Unlike “Static STS,” “Dynamic STS (Scrambled Timestamp Sequence) mode” may be an operation mode in which STS is not repeated during a ranging session. In this mode, STS is managed by the ranging device, and the ranging session key that creates the STS can be managed by the Secure Component.

“Applet” may be, for example, an applet running on the Secure Component containing UWB parameters and service data. The Applet may be a FiRa Applet.

“Ranging Device” may be a device capable of performing UWB ranging. In this disclosure, the Ranging Device may be an Enhanced Ranging Device (ERDEV) or a FiRa Device defined in IEEE 802.15.4z. Ranging Device may be referred to as a UWB device.

“UWB-enabled Application” may be an application for UWB service. For example, a UWB-enabled Application may be an application that uses the Framework API to configure an OOB Connector, Secure Service, and/or UWB service for a UWB session. “UWB-enabled Application” may be abbreviated as application or UWB application. A UWB-enabled Application may be a FiRa-enabled Application.

“Framework” may be a component that provides access to the Profile, individual UWB settings, and/or notifications. The Framework may be a collection of logical software components, including, for example, Profile Manager, OOB Connector, Secure Service, and/or UWB Service. The Framework may be FiRa Framework.

“OOB Connector” may be a software component for establishing an out-of-band (OOB) connection (e.g., BLE connection) between Ranging Devices. The OOB Connector may be FiRa OOB Connector.

“Profile” may be a predefined set of UWB and OOB configuration parameters. The Profile may be a FiRa Profile.

“Profile Manager” may be a software component that implements profiles available on the Ranging Device. The Profile Manager may be FiRa Profile Manager.

“Service” may be the implementation of a use case that provides a service to an end-user.

“Smart Ranging Device” may be a ranging device that can implement an optional Framework API. The Smart Ranging Device may be a FiRa Smart Device.

“Global Dedicated File (GDF)” may be the root level of application specific data containing data required to establish a USB session.

“Framework API” may be an API used by a UWB-enabled Application to communicate with the Framework.

“Initiator” may be a Ranging Device that initiates a ranging exchange. The initiator can initiate a ranging exchange by sending the first RFRAME (Ranging Exchange Message).

“Object Identifier (OID)” may be an identifier of the ADF within the application data structure.

“Out-Of-Band (OOB)” is an underlying wireless technology and may be data communication that does not use UWB.

“Ranging Data Set (RDS)” may be data (e.g., UWB session key, session ID, etc.) required to establish a UWB session for which confidentiality, authenticity, and integrity need to be protected.

“Responder” may be a Ranging Device that responds to the Initiator in a ranging exchange. The Responder can respond to the ranging exchange message received from the Initiator.

“STS” may be an encrypted sequence to increase the integrity and accuracy of ranging measurement timestamps. STS can be generated from the ranging session key.

“Secure Channel” may be a data channel that prevents overhearing and tampering.

“Secure Component” is an entity (e.g., Secure Element (SE) or Trusted Execution Environment (TEE)) with a defined level of security that interfaces with the UWBS for the purpose of providing RDS to the UWBS, e.g., when dynamic STS is used. ) can be.

“SE” may be a tamper-resistant secure hardware component that can be used as a Secure Component in a Ranging Device.

“Secure Ranging” may be ranging based on an STS generated through strong encryption operations.

“Secure Service” may be a software component for interfacing with a Secure Component such as a Secure Element or TEE.

“Service Applet” may be an applet on the Secure Component that handles service-specific transactions.

“Service Data” may be data defined by the Service Provider that needs to be transferred between two ranging devices to implement a service.

“Service Provider” may be an entity that defines and provides hardware and software required to provide specific services to end-users.

“Static STS mode” is an operation mode in which STS repeats during a session and does not need to be managed by the Secure Component.

A “Secure UWB Service (SUS) Applet” may be an applet on the SE that communicates with the applet to retrieve data needed to enable a secure UWB session with another ranging device. Additionally, SUS Applet can transmit the data (information) to UWBS.

“UWB Service” may be a software component that provides access to UWBS.

“UWB Session” may be the period from when the Controller and Controlee start communication through UWB until they stop communication. A UWB Session may include ranging, data forwarding, or both ranging/data forwarding.

“UWB Session ID” may be an ID (e.g., a 32-bit integer) that identifies the UWB Session, shared between the controller and the controller.

“UWB Session Key” may be a key used to protect the UWB Session. UWB Session Key can be used to create STS. The UWB Session Key may be UWB Ranging Session Key (URSK), and may be abbreviated as session key.

“UWB Subsystem (UWBS)” may be a hardware component that implements the UWB PHY and MAC layer (specification). UWBS can have an interface to the Framework and an interface to the Secure Component to retrieve the RDS.

*“UWB message” may be a message containing payload IE transmitted by a UWB device (eg, ERDEV). UWB messages include, for example, ranging initiation message (RIM), ranging response message (RRM), ranging final message (RFM), control message (CM), measurement report message (MRM), ranging result report message (RRRM), and control message (CUM). It may be a message such as an update message) or a one-way ranging (OWR) message. If necessary, multiple messages can be merged into one message.

* “OWR” may be a ranging method that uses messages transmitted in one direction between a ranging device and one or more other ranging devices. OWR can be used to measure Time Difference of Arrival (TDoA). Additionally, OWR can be used to measure AoA at the receiving end rather than measuring TDoA. In this case, one advertiser and one observer pair can be used.

“TWR” may be a ranging method that can estimate the relative distance between two devices by measuring ToF (time of flight) through the exchange of ranging messages between two devices. The TWR method may be one of double-sided two-way ranging (DS-TWR) and single-sided two-way ranging (SS-TWR). SS-TWR may be a procedure that performs ranging through one round-trip time measurement. For example, SS-TWR may include a transfer operation of RIM from the initiator to the responder, and a transfer operation of RRM from the responder to the initiator. DS-TWR may be a procedure that performs ranging through two round-trip time measurements. For example, DS-TWR may include a transfer operation of RIM from the initiator to the responder, a transfer operation of RRM from the responder to the initiator, and a transfer operation of RFM from the intiator to the responder. Through this ranging exchange (ranging message exchange), time of flight (ToF) can be calculated and the distance between the two devices can be estimated. Meanwhile, in the TWR process, measured AoA information (eg, AoA azimuth result, AoA elevation result) may be transmitted to another ranging device through RRRM or other messages. In this disclosure, TWR may be referred to as UWB TWR.

“DL-TDoA” may be called Downlink Time Difference of Arrival (DL-TDoA) or reverse TDoA. In the process of multiple anchor devices broadcasting messages or exchanging messages with each other, the user device (Tag device) becomes the anchor device. Overhearing messages from the device may be the default behavior. DL-TDoA may be classified as a type of one way ranging, like Uplink TDoA. A user device performing a DL-TDoA operation can overhear messages transmitted by two anchor devices and calculate Time Difference of Arrival (TDoA), which is proportional to the difference in distance between each anchor device and the user device. The user device can use the TDoA with multiple pairs of anchor devices to calculate the relative distance to the anchor device and use it for positioning. The operation of the anchor device for DL-TDoA may be similar to Double Side-Two Way Ranging (DS-TWR) defined in IEEE 802.15.4z, and other useful time information may be added to enable the user device to calculate TDoA. It may also be included. In this disclosure, DL-TDoA may be referred to as DL-TDoA localization.

“Anchor device” may be called an anchor, UWB anchor, or UWB anchor device, and may be a UWB device placed at a specific location to provide a positioning service. For example, an anchor device may be a UWB device installed by a service provider on an indoor wall, ceiling, or structure to provide an indoor positioning service. Anchor devices may be classified into Initiator anchors and Responder anchors depending on the order and role of transmitting messages.

“Initiator anchor” may be called an Initiator UWB anchor, an Initiator anchor device, etc., and may announce the start of a specific ranging round. The Initiator anchor may schedule ranging slots in which Responder anchors operating in the same ranging round respond. The initiation message of the Initiator anchor may be referred to as an Initiator DTM (Downlink TDoA Message) or Poll message. The initiation message of the Initiator anchor may include a transmission timestamp. The Initiator anchor may additionally deliver a termination message after receiving responses from Responder anchors. The end message of the Initiator anchor may be referred to as Final DTM or Final message. The end message may also include the reply time for messages sent by Responder anchors. The termination message may include a transmission timestamp.

“Responder anchor” may be called Responder UWB anchor, Responder UWB anchor device, Responder anchor device, etc. The Responder anchor may be a UWB anchor that responds to the Initiator anchor's initiation message. The message to which the Responder anchor responds may include the response time to the opening message. The message that the Responder anchor responds to may be referred to as a Responder DTM or Response message. The Responder anchor's response message may include a transmission timestamp.

“Cluster” may refer to a set of UWB anchors covering a specific area. A cluster can be composed of an initiator UWB anchor and responder UWB anchors that respond to it. For 2D positioning, one initiator UWB anchor and at least three responder UWB anchors are typically required, and for 3D positioning, one initiator UWB anchor and at least four responder UWB anchors are required. If the initiator UWB anchor and the responder UWB anchor can accurately synchronize the initiator (time synchronization) with separate wired/wireless connections, 1 initiator UWB anchor and 2 responder UWB anchors are needed for 2D positioning, and 1 for 3D positioning. Two initiator UWB anchors and three responder UWB anchors are required. Unless otherwise stated, it is assumed that there is no separate device for wired/wireless starter between UWB anchors. The area of the cluster may be a space formed by the UWB anchors that make up the cluster. To support positioning services for a wide area, multiple clusters can be configured to provide positioning services to user devices. A cluster may also be referred to as a cell. The operation of a cluster can be understood as the operation of anchor(s) belonging to the cluster.

1 is a block diagram schematically showing an electronic device.

Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes the main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing device) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself, where artificial intelligence is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network 199. It may communicate with an external electronic device 104 through (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a long-distance communication network such as a computer network (e.g., LAN or WAN)). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes: a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); and a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. You can.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to that component in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. can be used A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or two or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. . According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar manner as those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or , or one or more other operations may be added.

2A shows an example architecture of a UWB device according to one embodiment of the present disclosure.

In this disclosure, the UWB device 200 may be an electronic device that supports UWB communication. For example, the UWB device 200 may be an example of the electronic device 101 of FIG. 1 .

For example, the UWB device 200 may be a ranging device that supports UWB ranging. In one embodiment, the Ranging Device may be an enhanced ranging device (ERDEV) or a FiRa Device.

In the embodiment of FIG. 2A, UWB device 200 can interact with other UWB devices through a UWB session.

Additionally, the UWB device 200 may implement a first interface (Interface #1), which is an interface between the UWB-enabled Application 210 and the UWB Framework 220, and the first interface may be implemented as a UWB-enabled interface on the UWB device 200. Allows application 110 to use the UWB capabilities of UWB device 200 in a predetermined manner. In one embodiment, the first interface may be a Framework API or a proprietary interface, but is not limited thereto.

Additionally, the UWB device 200 may implement a second interface (Interface #2), which is an interface between the UWB Framework 210 and the UWB subsystem (UWBS) 230. In one embodiment, the second interface may be, but is not limited to, UCI (UWB Command Interface) or a proprietary interface.

Referring to FIG. 2A, the UWB device 200 may include a UWB-enabled Application 210, a Framework (UWB Framework) 220, and/or a UWBS 230 including a UWB MAC Layer and a UWB Physical Layer. there is. Depending on the embodiment, some entities may not be included in the UWB device, or additional entities (eg, security layer) may be further included.

The UWB-enabled Application 210 may trigger establishment of a UWB session by the UWBS 230 using the first interface. Additionally, the UWB-enabled Application 210 can use one of the predefined profiles. For example, the UWB-enabled Application 210 may use one of the profiles defined in FiRa or a custom profile. UWB-enabled Application 210 may use the first interface to handle related events such as service discovery, ranging notifications, and/or error conditions.

Framework 220 may provide access to Profile, individual UWB settings and/or notifications. Additionally, the Framework 220 may support at least one of the following functions: a function for UWB ranging and transaction performance, a function to provide an interface to an application and the UWBS 230, or a function to estimate the location of the device 200. Framework 220 may be a set of software components. As described above, the UWB-enabled Application 210 may interface with the Framework 220 through a first interface, and the Framework 220 may interface with the UWBS 230 through a second interface.

Meanwhile, in the present disclosure, the UWB-enabled Application 210 and/or Framework 220 may be implemented by an application processor (AP) (or processor). Accordingly, in the present disclosure, the operation of the UWB-enabled Application 210 and/or Framework 220 may be understood as being performed by the AP (or processor). In this disclosure, the framework may be referred to as an AP or processor.

UWBS 230 may be a hardware component including a UWB MAC Layer and a UWB Physical Layer. The UWBS 230 performs UWB session management and can communicate with the UWBS of other UWB devices. UWBS 230 can interface with Framework 120 through a second interface and obtain secure data from the Secure Component. In one embodiment, the Framework (or application processor) 220 may transmit a command to the UWBS 230 through UCI, and the UWBS 230 may send a response to the command to the Framework 220. It can be delivered to . UWBS 230 may deliver notification to Framework 120 through UCI.

FIG. 2B shows an example configuration of a framework of a UWB device according to an embodiment of the present disclosure.

The UWB device in FIG. 2B may be an example of the UWB device in FIG. 2A.

Referring to Figure 2b, the Framework 220 may include software components such as, for example, Profile Manager 221, OOB Connector(s) 222, Secure Service 223, and/or UWB Service 224. .

Profile Manager 221 may perform a role in managing profiles available on the UWB device. Here, a profile may be a set of parameters required to establish communication between UWB devices. For example, the profile may include parameters indicating which OOB secure channel is used, UWB/OOB configuration parameters, parameters indicating whether the use of a particular security component is mandatory, and/or parameters related to the file structure of the ADF. can do. The UWB-enabled Application 210 may communicate with the Profile Manager 221 through a first interface (eg, Framework API).

The OOB Connector 222 can perform the role of establishing an OOB connection with another device. OOB Connector 222 may handle OOB steps including a discovery step and/or a connection step. OOB component (eg, BLE component) 250 may be connected to OOB Connector 222.

Secure Service 223 may perform the role of interfacing with Secure Component 240, such as SE or TEE.

UWB Service 224 may perform the role of managing UWBS 230. The UWB Service 224 can provide access from the Profile Manager 321 to the UWBS 230 by implementing a second interface.

* The present disclosure provides a method and device for performing pose detection using a UWB signal. The UWB signal may be a UWB signal used for DL-TDoA ranging (OWR), or a UWB signal used for DS-TWR ranging. In the present disclosure, a pose may be, for example, a pose related to where the user is holding the user device (eg, handheld) or where it is placed (eg, back pocket/front pocket).

The present disclosure receives a UWB signal, obtains UWB channel impulse response (CIR) data, and determines whether the UWB signal is a line of sight (LOS) signal or a non-LOS (NLOS) signal based on the UWB CIR data. Provides a way to classify cognition. For classification of LOS signals or NLOS signals, a convolutional neural network (CNN) algorithm may be used. In this way, by classifying data with time series characteristics using a CNN model, high classification accuracy can be achieved. As input data for a CNN model used to classify LOS signals or NLOS signals, effective CIR data from which noise has been removed from UWB CIR data can be used. Through this, the classification accuracy of LOS/NLOS signals and the classification accuracy of poses can be increased.

The present disclosure provides a method of performing pose classification using a classification result of a LOS signal or an NLOS signal (signal classification result) and sensing data. Data from an IMU (inertial measurement unit) sensor may be used as sensing data. To perform signal classification results and pose classification using sensing data, two CNN models (e.g., two parallel structured CNN models) may be used, which are different from the CNN model used for classification of LOS/NLOS signals. . Among the two CNN models, one CNN model is trained using IMU data (first learning data) associated with the LOS signal, and the other CNN model is trained using IMU data (second learning data) associated with the NLOS signal. It can be. In this way, classification accuracy can be increased by classifying data with different characteristics using a separate CNN model.

The present disclosure provides a method for filtering prediction data output from each CNN model before final LOS/NLOS signal classification and final pose classification. For filtering, a moving average filter can be used. Through this, the classification accuracy of LOS/NLOS signals and the classification accuracy of poses can be increased.

The present disclosure provides a method by which various applications of a user device can use classified LOS/NLOS signals and/or classified poses. For example, the classification method of the present disclosure can be used in an application for a tagless gate, a digital car key application, or a point of service (POS) application. The application may appropriately provide a specific service or functions/parameters associated with a specific service using accurately classified LOS/NLOS signals and/or classified poses.

As an example, an application may perform adaptive ranging frequency adjustment for power saving using accurately classified LOS/NLOS signals. For example, an application can set a maximizing sleep duration through adaptive range period adjustment using classified LOS/NLOS signals.

As an example, the application can increase the accuracy of location estimation by selectively using only the LOS signal through accurate classification of LOS/NLOS signals.

As an example, an application may set an adaptive access boundary through accurate classification of LOS/NLOS signals and poses. For example, if a situation of NLOS signal/Back pocket pose is identified, the application can delay the start of the service.

Figure 3A shows an example of UWB CIR data according to an embodiment of the present disclosure.

The UWB channel impulse response (CIR) can be obtained through reception of a UWB signal. For example, when receiving a UWB signal for DL-TDoA or a UWB signal for DS-TWR, the electronic device may obtain UWB CIR data based on the received signal. In this disclosure, UWB CIR may be abbreviated as CIR.

According to one embodiment, an electronic device may obtain UWB CIR data using a correlation value between an impulse function and a received UWB signal.

An example of UWB CIR data obtained based on a UWB signal may be as shown in FIG. 3A. As illustrated in FIG. 3A, for example, 1016 CIR values may be output. For example, each CIR value may be a value obtained or measured based on each UWB signal including a final message (RIM) from the initiator received by the responder in the DS-TWR process. Alternatively, each CIR value may be a value obtained or measured based on each UWB signal including the final message from the initiator anchor received by the user device (tag device) in the DL-TDoA process, for example.

Referring to FIG. 3A, a peak in amplitude may occur after a specific CIR index (eg, CIR index 750).

Figure 3b shows an example of effective UWB CIR data according to an embodiment of the present disclosure.

In the present disclosure, effective UWB CIR may mean a CIR with a significant value among CIRs. That is, the effective UWB CIR may be a CIR obtained or measured by an actual signal rather than noise. In this disclosure, effective UWB CIR may be referred to as effective CIR (eCIR).

According to one embodiment, the electronic device may process CIR values before the peak occurs as noise and process n CIR values from the peak as eCIR.

According to one embodiment, the electronic device may determine n CIRs as eCIR, starting from the CIR index where the margin value is subtracted from the CIR index where the peak occurs. For example, if the peak occurs at CIR index 750, the marging value is 5 (margin = 5), and the n value is set to 200 (n = 200), the electronic device will detect CIRs from CIR index 745 to CIR index 945. These can be identified by eCRI. In this way, CIRs can be classified into signal and noise, and an example of this classification may be as shown in FIG. 3B.

When eCIR is classified from the obtained CIRs and only eCIR is used as input data for a CNN model for classification of LOS/NLOS signals, inaccurate noise can be removed to increase classification accuracy.

Additionally, when eCIR is classified and used from the obtained CIRs, latency for exchange of CIR data can be optimized.

Additionally, when eCIR is classified and used from the obtained CIRs, the processing efficiency of the terminal can be improved by optimizing the input matrix.

Figure 3c shows an example of a LOS signal and a NLOS signal classified using UWB CIR data according to an embodiment of the present disclosure.

As described above, CIR values used for classification of LOS signals/NLOS signals may be eCIR values. For example, as shown in FIG. 3C, n (e.g., 200) eCIR values can be used for classification of LOS/NLOS signals.

Meanwhile, CIR values (eCIR values) of the LOS signal and the NLOS signal may have different characteristics.

For example, a LOS signal may have at least one of the following characteristics.

- The peak value of CIR is higher compared to the peak value of NLOS signal

- After the peak value, the CIR value quickly falls to the noise level.

- Only one or two peaks appear.

For example, in the case of an NLOS signal, it may have at least one of the following characteristics.

- The peak value of CIR is low compared to the peak value of LOS signal

- It takes a long time for the CIR value to fall to the noise level after the peak value.

- Multiple peak values appear due to multipath signals

Due to the difference in characteristics of the CIR values of the LOS signal and the NLOS signal, the signal can be classified as a LOS signal or an NLOS signal using the measured CIR data.

According to one embodiment, the electronic device may use a deep learning algorithm to classify whether the corresponding signal is a LOS signal or an NLOS signal using CIR data (eCIR data) for the corresponding signal. For example, the electronic device can classify LOS/NLOS signals using a CNN model (CNN algorithm). A method for classifying LOS/NLOS signals using a CNN model will be described below with reference to FIGS. 4 to 11.

* Hereinafter, for convenience of explanation, it is assumed that the four cases for generating data used for classification of LOS/NLOS signals and/or classification of poses are classified as follows.

(1) Handheld (LOS) pose: A case in which the user holds the electronic device in his/her hand and does not block the UWB antenna of the electronic device.

(2) Hand NLOS case (Handheld (NLOS) pose): A case where the user holds the electronic device in his hand and blocks the UWB antenna of the electronic device.

(3) Front pocket (LOS) case (Front pocket (LOS) pose): A case where the electronic device is in the user's front pocket and a LOS signal is guaranteed.

(4) Back pocket (NLOS) case (Back pocket (NLOS) pose): A case where the electronic device is in the user's back pocket and the signal is blocked due to body effect.

In certain circumstances (e.g., an environment in which the user's electronic device approaches (approaches) an initiator for UWB ranging (e.g., an initiator for DS-TWR or an initiator anchor for DL-TDoA(OWR)), the above Data for four cases (eg, UWB CIR data and/or IMU sensor data) may be collected. The data collected in this way can be used for training and testing for each CNN model. However, the embodiment is not limited to this, and depending on the embodiment, data from other environments and/or other cases may be collected and used for training/testing of the CNN model.

First, with reference to FIGS. 4 and 5, examples of environments where LOS/NLOS classification is applicable, such as DS-TWR in FIG. 4 and DL-TDoA in FIG. 5, will be described.

Figure 4 shows DS-TWR according to an embodiment of the present disclosure.

Referring to FIG. 4, DS-TWR may be performed between the first electronic device 410 and the second electronic device 420.

In the embodiment of FIG. 4, for convenience of explanation, the first electronic device 410 performs the role of an initiator for DS-TWR, and the second electronic device 420, which is the user's electronic device, performs DS-TWR. It is assumed that it performs the role of responder. In this case, it is assumed that the second electronic device 420 approaches the first electronic device 410, which is the initiator. However, the embodiment is not limited to this, and for example, the opposite case is also possible.

As an example, the first electronic device 410 may be, for example, a UWB-based gate (UWB smart gate), and the second electronic device 420 may be a user's electronic device (e.g., a user's smart phone).

Referring to FIG. 4, in one ranging cycle (one ranging measurement cycle), the first electronic device 410, which is the initiator, may initiate the DS-TWR ranging procedure by transmitting a ranging initiation message (RIM). , the second electronic device 420, which is a responder, may transmit a ranging response message (RRM), which is a response message to the RIM, to the first electronic device 410, and the first electronic device 410 may transmit a response message to the RRM. An in-ranging final message (RFM) may be transmitted to the second electronic device 420. One ranging measurement cycle for this DS-TWR may be performed repeatedly (eg, periodically).

As an example, the second electronic device 420 may obtain a CIR value for each UWB signal including RFM received. The CIR data obtained in this way can be used for classification of LOS/NLOS signals.

As an example, the second electronic device 420 may acquire sensing data from an IMU sensor together with CIR data. As an example, the acquisition cycle of IMU sensor data may be different (eg, smaller) than the acquisition cycle of CIR data. In this case, it may be necessary to perform a sequencing operation to input IMU sensor data corresponding to CIR data as input to the CNN model.

Figure 5 shows DL-TDoA according to an embodiment of the present disclosure.

In the embodiment of FIG. 5, for convenience of explanation, one initiator anchor (initiator UWB anchor) 510 and three responder anchors (520-1,520-2,520-3) are used for DL-TDoA. It is assumed that it is used. In this case, it is assumed that the user's electronic device 530 approaches the initiator anchor 510, which is the initiator. However, the embodiment is not limited to this, and for example, different numbers of anchor devices are also possible.

Referring to FIG. 5, in one ranging cycle (one ranging measurement cycle), the initiator anchor 510, which is an initiator, can initiate the DL-TDoA ranging procedure by transmitting an initiation message, and the responder anchors, which are responders ( 520-1, 520-2, 520-3) may transmit a response message to the initiation message to the initiator anchor 510, and the initiator anchor 510 may transmit a final message, which is a response message to the response message, to each responder anchor (520- It can be sent to 1,520-2,520-3). User device 530 may receive this initiation message, response messages, and final messages and estimate its location according to a predefined manner.

As an example, user device 530 may obtain a CIR value for each UWB signal including a received final message. The CIR data obtained in this way can be used for classification of LOS/NLOS signals.

As an example, the user device 530 may acquire sensing data from an IMU sensor together with CIR data. As an example, the acquisition cycle of IMU sensor data may be different (eg, smaller) than the acquisition cycle of CIR data. In this case, it may be necessary to perform a sequencing operation to input IMU sensor data corresponding to CIR data as input to the CNN model.

Figure 6 shows a learning procedure and distribution procedure for LOS/NLOS signal classification using UWB channel impulse response data according to an embodiment of the present disclosure.

In the embodiment of Figure 6, for convenience of explanation, it is assumed that the training procedure 610 is performed on a remote server and the deployment procedure 620 is performed on the user device. However, the embodiment is not limited to this. For example, if the user device is a device with high computing power, the learning procedure may also be performed on the user device.

First, with reference to FIG. 6, the learning procedure 610 will be described.

In the learning procedure 610, the server may collect input data (611). For example, the server may collect CIR data (e.g., eCIR data including n eCIR values) as input data for learning.

The server may perform CIR normalization processing on the collected input data (612). For example, the server may normalize the collected input data to between 0 and 1. However, CIR normalization in operation 612 is an optional operation and may be omitted.

The server can input CIR normalized data or non-CIR normalized data as input data to the CNN model, and perform processing for LOS/NLOS classification using the CNN model (613).

Tables 1 and 2 below show examples of CNN model characteristics and hyperparameters used for LOS/NLOS classification.

The server can obtain the predicted value output from the CNN model (614). As an example, the prediction value may include a predicted probability value for a label (eg, label 1) corresponding to a LOS signal and a predicted probability value for a label (eg, label 2) corresponding to an NLOS signal.

The server can obtain the ground truth value for the signal (615). As an example, the actual value may include an actual probability value for a label corresponding to a LOS signal (e.g., label 1) and/or an actual probability value for a label corresponding to an NLOS signal (e.g., label 2).

The server may use a loss function to calculate the loss and gradient values between the predicted value of operation 614 and the actual value of operation 615 (616). For example, the total error can be calculated based on the error between the predicted value and the actual value for the LOS signal and the error between the predicted value and the actual value for the NLOS signal.

The server may update the parameters of the CNN model using the values obtained through operation 616. For example, the server can update the parameters of the CNN model to reduce errors using the back-propagation method. These updated parameters can be used in the CNN model for processing the next input data (e.g., the next n eCIR values).

The server performs the above-described learning procedure 610 on all collected input data sets (e.g., each input data set includes n eCIR values), respectively, and sets the parameters (CNN parameters) of the CNN model to an error rate. It can be continuously updated in the direction of reducing. The user device can acquire or download the CNN parameters learned in this way. Additionally, the user device may download information about whether normalization was used. Additionally, the user device may download standard deviations associated with the CNN model.

Below, the distribution procedure 620 will be described with reference to FIG. 6.

In the deployment procedure 620, the user device may collect real time input data (621). For example, the user device may collect a plurality of CIR values (e.g., eCIR values) as input data and process them by dividing them into a preset number of lengths (sizes) (e.g., n lengths). As an example, this length (n) may be the same as the length (n) used for learning.

The user device may perform CIR normalization on the collected input data (622). For example, the user device may normalize the collected input data to between 0 and 1. However, the CIR normalization of operation 622 is an optional operation and may be omitted.

The user device can input CIR normalized data or non-CIR normalized data as input data to the CNN model, and perform processing for LOS/NLOS classification using the CNN model (623). The CNN model of the user device may use CNN parameters obtained through the learning procedure 610 to process input data.

The user device may obtain prediction data (real-time prediction data) output from the CNN model (624). As an example, the prediction data output from the CNN model includes the prediction value (first prediction value) for the first eCIR data (the first n eCIR data among the real-time input data), the second eCIR data (the second n eCIR data among the real-time input data), and the prediction data output from the CNN model. It may include a predicted value (second predicted value) for n eCIR data), ... , a predicted value (n th predicted value) for n-th eCIR data (n-th n eCIR data among real-time input data). there is. As an embodiment, each prediction value is a predicted probability value for a label corresponding to a LOS signal (e.g., label 1) and/or a predicted probability for a label corresponding to an NLOS signal (e.g., label 2) for the corresponding eCIR data. Can contain values.

The user device may filter the predicted value (predicted data) using a filter (eg, low pass filter (LPF) or moving average filter) (625). As an embodiment, the moving average filter may have a hyperparameter with a window length parameter indicating the length of the window used to calculate the average of the data, and the user device uses the moving average filter with the window length parameter. You can filter the predicted data. For example, the user device may use a moving average filter to replace each predicted value with an average value associated with the corresponding predicted value obtained based on the window length parameter.

The user device can use the filtered data to obtain an output label for real-time input data (626). The user device uses the filtered data to determine whether the output label for the signal corresponding to the real-time input data is the label corresponding to the LOS signal (e.g. label 1) or the label corresponding to the NLOS signal (e.g. label 2). You can finally decide. Through this, the user device can finally classify whether the signal is a LOS signal or an NLOS signal. In this way, the classification accuracy of LOS/NLOS signals can be increased by providing filtering using a moving average filter for the prediction data output from the CNN model before final LOS/NLOS signal classification.

Table 3 below illustrates a comparison of the accuracy of LOS/NLOS classification with and without using an LPF filter (e.g., moving average filter).

Figure 7 shows the configuration of a user device for LOS/NLOS classification according to an embodiment of the present disclosure.

Referring to FIG. 7, the user device 700 includes a UWB antenna 710, a CIR collection unit 720, a machine learning unit 730, a classification processing unit 740, a central control unit 750, and/or a storage unit ( 750). The classification processing unit 740 may include an effective CIR generating unit 741, a normalizing unit 742, and/or a moving average filter 743.

Depending on the embodiment, some of the above-described components may be omitted or additional components may be further included. Depending on the embodiment, two or more of the above-described components may be merged into one component. Depending on the embodiment, all or part of the above-described components may be implemented by at least one processor (or control unit). For example, components other than the UWB antenna 710 and the storage unit 750 may be implemented by at least one processor (or control unit).

The UWB antenna 710 may receive at least one UWB signal from another electronic device. For example, the UWB antenna 710 may receive at least one UWB signal for DS-TWR or at least one UWB signal for DL-TDoA.

The CIR collection unit 720 may collect CIR data from at least one UWB signal received through the UWB antenna 710. The CIR collection unit 720 may transmit the collected CIR data to the effective CIR generation unit 741. Collected CIR data may include multiple CIR values.

The effective CIR generator 741 may generate effective CIR data from the collected CIR data. The effective CIR generation unit 741 may transmit the generated effective CIR data to the normalization unit 742 or directly to the machine learning unit 730. Effective CIR data may include a plurality of effective CIR values and may be processed by the machine learning unit 730 in a preset number of units (eg, n units).

The normalization unit 742 may normalize values of effective CIR (eCIR) data. For example, the normalization unit 742 may normalize the values of effective CIR data to values between 0 and 1. The normalization unit 742 may have an optional configuration. For example, the normalization unit 742 may not be included in the user device 700, or even if it is included in the user device 700, processing of the normalization unit 742 may be omitted.

The machine learning unit 730 may generate prediction data using the input valid CIR data. As an example, the machine learning unit 730 may generate prediction data using a CNN model. As an example, CNN parameters of a CNN model may be downloaded from a server on which a learning procedure has been previously performed. Accordingly, the machine learning unit 730 can use an optimized CNN model.

As an embodiment, the machine learning unit 730 may perform processing in units of a preset number of eCIRs (eg, n eCIRs) (eCIR sets) and output prediction values for each eCIR set. For example, the prediction data may include the prediction value (first prediction value) for the first n eCIRs (first eCIR set), the prediction value (second prediction value) for the next n eCIRs (second eCIR set), value), ... , may include predicted values (n-th predicted values) for the n-th n eCIRs (n-th eCIR set). Each predicted value may include a predicted probability value corresponding to LOS and a predicted probability value corresponding to NLOS for the corresponding eCIR set.

The machine learning unit 730 may transmit the prediction data to the central control unit 750 and/or the moving average filter 743.

The central control unit 750 may transfer the received prediction data to the storage unit 730. The storage unit 730 may store the received prediction data and transmit the stored prediction data to the moving average filter 743.

The moving average filter 743 uses the prediction data (e.g., current prediction data) received from the machine learning unit 730 and the prediction data (e.g., previous prediction data) received from the storage unit 760 to set a preset window. The average value can be calculated based on the length parameter, and the prediction data can be filtered using the average value. This filtered prediction data can be used for final classification into LOS signals or NLOS signals. Filtered prediction data can be used by various applications.

Figure 8 shows a learning procedure for pose classification based on LOS/NLOS signal classification using UWB channel impulse response data according to an embodiment of the present disclosure.

In the embodiment of Figure 8, for convenience of explanation, it is assumed that the learning procedure 810 is performed on a remote server.

First, with reference to FIG. 8, the learning procedure 800 will be described.

In the learning procedure 800, the server may collect input data (801). For example, the server may collect CIR data (e.g., eCIR data including n eCIR values) as input data for learning.

The server may perform CIR normalization on the collected input data (802). For example, the server may normalize the collected input data to between 0 and 1. However, CIR normalization in operation 802 is an optional operation and may be omitted.

The server may input CIR normalized data or non-CIR normalized data as input data of the first CNN model, and may perform processing for LOS/NLOS classification using the first CNN model (803). . In this disclosure, the first CNN model refers to the CNN model used for LOS/NLOS classification.

The server may obtain the prediction value (first prediction value or first prediction information) output from the first CNN model (804). As an example, the first prediction value may include a predicted probability value for a label (eg, label a) corresponding to a LOS signal and a predicted probability value for a label (eg, label b) corresponding to an NLOS signal.

The server may obtain a ground truth value (first truth value or first truth information) for the corresponding signal (805). As an embodiment, the first actual value may include a true probability value for a label corresponding to a LOS signal (e.g., label 1) and/or a true probability value for a label corresponding to an NLOS signal (e.g., label 2). there is.

The server may use the loss function to calculate the loss and gradient values between the predicted value of operation 804 and the actual value of operation 805 (806). For example, the total error can be calculated based on the error between the predicted value and the actual value for the LOS signal and the error between the predicted value and the actual value for the NLOS signal.

The server may update the parameters of the first CNN model using the values obtained through operation 806. For example, the server can update the parameters of the first CNN model to reduce errors using a back-propagation method. These updated parameters can be used in the first CNN model for processing the next input data (eg, the next n eCIR values).

The server performs the above-described learning procedure 800 on all collected eCIR sets (e.g., each eCIR set includes n eCIR values), respectively, and sets the parameters (CNN parameters) of the first CNN model to the error rate. It can be continuously updated in the direction of reducing. The user device may obtain or download CNN parameters for the first CNN model learned in this way. Additionally, the user device may download information about whether normalization was used. Additionally, the user device may download standard deviations associated with the first CNN model.

The server may determine whether the signal corresponding to the eCIR set is classified as a LOS signal or an NLOS signal, based on the prediction value for the eCIR set. For example, in the prediction value for that eCIR set, if the predicted probability value corresponding to the LOS signal is greater than the predicted probability value corresponding to the NLOS signal, the server determines that the signal corresponding to that eCIR set is classified as a LOS signal. You can. For another example, in the prediction value for the corresponding eCIR set, if the predicted probability value corresponding to the NLOS signal is greater than the predicted probability value corresponding to the LOS signal, the server determines that the signal corresponding to the corresponding eCIR set is classified as an NLOS signal. You can decide.

If the signal corresponding to the corresponding eCIR set is classified as a LOS signal, a classification procedure for pose detection based on the second CNN model can be performed. Alternatively, if the signal corresponding to the corresponding eCIR set is classified as an NLOS signal, a classification procedure for pose detection based on the third CNN model can be performed. In this disclosure, the second CNN model refers to a CNN model used for pose classification that is learned using IMU sensor data associated with LOS signals. In this disclosure, the third CNN model refers to a CNN model used for pose classification that is learned using IMU sensor data associated with NLOS signals.

Tables 4 and 5 below show examples of characteristics and hyperparameters of CNN models (second CNN model/third CNN model) used for pose classification.

The server may collect sensor input data (807). The sensor input data may be sensor input data associated with a LOS signal (first sensor input data) or sensor input data associated with an NLOS signal (second sensor input data). For example, the sensor data collected in the above-described environment may be sensor input data (first sensor input data) associated with the LOS signal, while the UWB antenna is not obscured or is placed in a back pocket. The sensor data collected in the above-described environment may be sensor input data (second sensor input data) associated with the NLOS signal.

As an example, the sensor input data may include IMU values for m timestpes (m time series IMU values).

The server may perform sequencing processing to input sensor input data corresponding to input data for the first CNN model into the second CNN model or the third CNN model (808). For example, the server may perform sequencing processing to input sensor input data collected during the same period as the period during which input data including n eCIR values were collected into a second CNN model or a third CNN model.

If the signal is classified as a LOS signal, the server can input the sequenced first sensor input data as input data for learning the second CNN model, and perform processing for pose classification using the second CNN model. You can do it (809). For example, the second CNN model uses input data to perform an operation to predict whether the pose for the corresponding sensor input data is a first pose (e.g. handheld) or a second pose (e.g. front pocket). can do. The server may obtain the prediction value (second prediction value or second prediction information) output from the second CNN model (810). As an embodiment, the prediction value is, for first sensor input data associated with LOS, a predicted probability value for a label (e.g., label 1) corresponding to the first pose (e.g., handheld (LOS)) and a second pose (e.g., : front pocket (LOS)) may contain the predicted probability value for the corresponding label (e.g. label 2).

If the signal is classified as an NLOS signal, the server can input the sequenced second sensor input data as input data for learning of the third CNN model, and perform processing for pose classification using the third CNN model. You can do it (811). For example, the third CNN model uses input data to determine whether the pose for the corresponding sensor input data is a third pose (e.g. handheld (NLOS)) or a fourth pose (e.g. back pocket (NLOS)). Predictive actions can be performed. The server may obtain the prediction value (second prediction value or second prediction information) output from the third CNN model (812). As an embodiment, the prediction value is, for second sensor input data associated with LOS, a predicted probability value for a label (e.g., label 3) corresponding to a third pose (e.g., handheld (NLOS)) and a fourth pose (e.g. : back pocket (NLOS)) may contain the predicted probability value for the corresponding label (e.g. label 4).

The server may obtain a ground truth value (second truth value or second truth information) for the pose (813). As an embodiment, the second actual value is a true probability value for a label (e.g., label 1) corresponding to the first pose, an actual probability value for a label (e.g., label 2) corresponding to the second pose, and a third pose. It may include an actual probability value for a label (e.g., label 3) corresponding to and/or an actual probability value for a label (e.g., label 4) corresponding to the fourth pose.

The server uses the loss function to calculate loss and gradient values between the second predicted value in operation 810 and the second actual value in operation 813, and/or between the third predicted value in operation 812 and The loss and gradient values between the second actual values in operation 813 may be calculated (814).

The server may update the parameters of the corresponding CNN model (second CNN model or third CNN model) using the values obtained through operation 814. For example, the server may update the parameters of the second CNN model using the loss and/or gradient obtained based on the second predicted value in operation 810 and the second actual value in operation 813. there is. Alternatively, the server may update the parameters of the third CNN model using the loss and/or gradient obtained based on the third prediction value in operation 812 and the second actual value in operation 813. As an embodiment, the server may update the parameters of the corresponding CNN model in a way that reduces errors using a back-propagation method.

These updated parameters can be used in the corresponding CNN model to process the next input data (e.g., the next m IMU sensing values).

The server performs the above-described learning procedure 800 on all collected IMU sensor data sets (e.g., each IMU sensor data set includes m IMU sensing values), respectively, to create a second CNN model and a third CNN model. The parameters (CNN parameters) can be continuously updated to reduce errors. The user device may obtain or download CNN parameters for the learned second CNN model and third CNN model.

Figure 9 shows a distribution procedure for pose classification based on LOS/NLOS signal classification using UWB channel impulse response data according to an embodiment of the present disclosure.

In the embodiment of Figure 9, for convenience of explanation, it is assumed that the distribution procedure 900 is performed on a user device.

First, with reference to FIG. 9, the distribution procedure 900 will be described.

In the deployment procedure 900, the user device may collect input data (901). For example, the user device may collect a plurality of CIR values (e.g., eCIR values) as input data and process them by dividing them into a preset number of lengths (sizes) (e.g., n lengths). As an example, this length (n) may be the same as the length (n) used for learning.

The user device may perform CIR normalization on the collected input data (902). For example, the user device may normalize the collected input data to between 0 and 1. However, CIR normalization in operation 902 is an optional operation and may be omitted.

The user device may input CIR normalized data or non-CIR normalized data as input data of the first CNN model, and may perform processing for LOS/NLOS classification using the first CNN model (903 ). In this disclosure, the first CNN model refers to the CNN model used for LOS/NLOS classification.

The user device may obtain a prediction value (first prediction value or first prediction information) output from the first CNN model (904). As an example, the first prediction value may include a predicted probability value for a label (eg, label a) corresponding to a LOS signal and a predicted probability value for a label (eg, label b) corresponding to an NLOS signal.

The user device may perform the above-described LOS/NLOS classification procedure for all collected eCIR sets (eg, each eCIR set includes n eCIR values). The user device may perform filtering on prediction data (first prediction data) including each prediction value obtained from the first CNN model using a moving average filter (905).

The user device may determine whether the signal corresponding to the eCIR set is classified as a LOS signal or an NLOS signal, based on the prediction value for the eCIR set. For example, in the prediction value for the corresponding eCIR set, if the predicted probability value corresponding to the LOS signal is greater than the predicted probability value corresponding to the NLOS signal, the user device determines that the signal corresponding to the corresponding eCIR set is classified as a LOS signal. You can decide. For another example, in the prediction value for the corresponding eCIR set, if the predicted probability value corresponding to the NLOS signal is greater than the predicted probability value corresponding to the LOS signal, the user device classifies the signal corresponding to the corresponding eCIR set as an NLOS signal. You can decide to do it.

If the signal corresponding to the corresponding eCIR set is classified as a LOS signal, a classification procedure for pose detection based on the second CNN model can be performed. Alternatively, if the signal corresponding to the corresponding eCIR set is classified as an NLOS signal, a classification procedure for pose detection based on the third CNN model can be performed. In this disclosure, the second CNN model refers to a CNN model used for pose classification learned using IMU sensor data associated with LOS signals. In this disclosure, the third CNN model refers to a CNN model used for pose classification learned using IMU sensor data associated with NLOS signals.

The user device may collect sensor input data (906). The sensor input data may be sensor input data associated with a LOS signal (first sensor input data) or sensor input data associated with an NLOS signal (second sensor input data). As an example, sensor input data may include IMU values for m timestpes.

The user device may perform sequencing processing to input sensor input data corresponding to input data for the first CNN model into the second CNN model or the third CNN model (907). For example, the user device may perform sequencing processing to input sensor input data collected during the same period as the period for collecting input data including n eCIR values into a second CNN model or a third CNN model. .

If the signal is classified as a LOS signal, the user device may input sensor input data (first sensor input data) as input data of a second CNN model, and perform processing for pose classification using the second CNN model. You can do it (908). For example, the second CNN model uses input data to determine whether the pose for the corresponding sensor input data is a first pose (e.g. handheld (LOS)) or a second pose (e.g. front pocket (LOS)). Predictive actions can be performed. The user device may obtain a prediction value (second prediction value or second prediction information) output from the second CNN model (909). As an embodiment, the prediction value is, for first sensor input data associated with LOS, a predicted probability value for a label (e.g., label 1) corresponding to the first pose (e.g., handheld (LOS)) and a second pose (e.g., : front pocket (LOS)) may contain the predicted probability value for the corresponding label (e.g. label 2).

If the signal is classified as an NLOS signal, the server can input the sequenced sensor input data (second sensor input data) as input data for learning of the third CNN model, and classify the pose using the third CNN model. Processing for can be performed (910). For example, the third CNN model uses input data to determine whether the pose for the corresponding sensor input data is a third pose (e.g. handheld (NLOS)) or a fourth pose (e.g. back pocket (NLOS)). Predictive actions can be performed. The user device may obtain the prediction value (second prediction value or second prediction information) output from the third CNN model (911). As an embodiment, the prediction value is, for second sensor input data associated with NLOS, a predicted probability value for a label (e.g., label 1) corresponding to a third pose (e.g., handheld (NLOS)) and a fourth pose (e.g. : front pocket (NLOS)) may contain the predicted probability value for the corresponding label (e.g. label 4).

The user device may perform the above-described pose classification procedure on all collected IMU sensor data sets (eg, each IMU sensor data set includes m IMU sensing values). The user device filters the prediction data (second prediction data) including each prediction value obtained from the second CNN model using a moving average filter or predicts data including each prediction value obtained from the third CNN model ( Filtering may be performed on the third prediction data (912). In this way, the classification accuracy of LOS/NLOS signals and poses can be increased by providing filtering using a moving average filter for the prediction data output from each CNN model before final LOS/NLOS signal classification and final pose classification.

The user device may determine the pose using the filtered first prediction data and the filtered second prediction data. For example, using the filtered first prediction data and the filtered second prediction data, whether the pose is a first pose (e.g., Hand (LOS) pose) or a second pose (e.g., Front (LOS)) You can decide whether you are aware of it or not.

Alternatively, the user device may determine the pose using the filtered first prediction data and the filtered third prediction data. For example, using the filtered first prediction data and the filtered third prediction data, whether the pose is a third pose (e.g., Hand (NLOS) pose) or a fourth pose (e.g., Back (NLOS)) You can decide whether you are aware of it or not.

Figure 10 shows the configuration of a user device for pose classification using LOS/NLOS classification according to an embodiment of the present disclosure.

Referring to FIG. 10, the user device 1000 includes an IMU sensor 1010, a sensor data collection unit 1020, a UWB antenna 1030, a CIR collection unit 1040, a machine learning unit 1050, and a classification processing unit ( 1060), a central control unit 1070, and/or a storage unit 1080. The classification processing unit 1060 may include a valid CIR generating unit 1061, a normalizing unit 1062, a moving average filter 1063, and/or a sequence generating unit 1064.

Depending on the embodiment, some of the above-described components may be omitted or additional components may be further included. Depending on the embodiment, two or more of the above-described components may be merged into one component. Depending on the embodiment, all or some of the above-described components may be implemented by at least one processor. For example, components other than the IMU sensor 1010, UWB antenna 1030, and storage unit 1080 may be implemented by at least one processor (or control unit).

The IMU sensor 1010 can sense the surrounding environment. In an embodiment, the IMU sensor 1010 uses a combination of an accelerometer, a gyroscope, and/or a magnetometer to measure a specific force and angular rate of a body. ) and/or it may be a sensor capable of measuring the orientation of the body. The IMU sensor 1010 may transmit sensed data to the sensor data collection unit 1020.

The sensor data collection unit 1020 may transmit the collected sensor data to the sequence generation unit 1064.

The sequence generator 1064 may perform sequencing processing to input sensor input data corresponding to input data for the first CNN model into the second CNN model or the third CNN model. For example, the sequence generator 1064 performs sequencing processing to input sensor input data collected during the same period as the period during which input data including n eCIR values were collected into the second CNN model or the third CNN model. It can be done.

The UWB antenna 1030 can receive at least one UWB signal from another electronic device. For example, the UWB antenna 1030 may receive at least one UWB signal for DS-TWR or at least one UWB signal for DL-TDoA.

The CIR collection unit 1040 may collect CIR data from at least one UWB signal received through the UWB antenna 1030. The CIR collection unit 1040 may transmit the collected CIR data to the effective CIR generation unit 1061. Collected CIR data may include multiple CIR values.

The effective CIR generator 1061 may generate effective CIR data from the collected CIR data. The effective CIR generation unit 1061 may transmit the generated effective CIR data to the normalization unit 1062 or directly to the machine learning unit 1050. Effective CIR data may include a plurality of effective CIR values and may be processed by the machine learning unit 1050 in a preset number of units (eg, n units).

The normalization unit 1062 may normalize values of effective CIR (eCIR) data. For example, the normalization unit 1062 may normalize the values of valid CIR data to values between 0 and 1. The normalization unit 1062 may have an optional configuration. For example, the normalization unit 1062 may not be included in the user device 1000, or even if it is included in the user device 1000, processing of the normalization unit 1062 may be omitted.

The machine learning unit 1050 may generate prediction data using input data. As an embodiment, the machine learning unit 730 may generate at least one prediction data using at least one CNN model, and CNN parameters of each CNN model may be downloaded from a server on which a learning procedure has been previously performed.

According to one embodiment, the machine learning unit 1050 may generate first prediction data for LOS/NLOS classification from input valid CIR data using the first CNN model. As an embodiment, the first CNN model may perform processing in units of a preset number of eCIRs (eg, n eCIRs) (eCIR sets) and output prediction values for each eCIR set. For example, the first prediction data may include a prediction value (first prediction value) for the first n eCIRs (a first eCIR set), and a prediction value (the first prediction value) for the next n eCIRs (a second eCIR set). 2 prediction value), ... , may include prediction values (nth prediction value) for the nth n eCIRs (nth eCIR set). Each predicted value may include a predicted probability value corresponding to LOS and a predicted probability value corresponding to NLOS for the corresponding eCIR set.

According to one embodiment, the machine learning unit 1050 may use a second CNN model to generate second prediction data for classifying poses related to LOS from input sensor data. As an example, the second CNN model may perform processing in units of a preset number of sensing values (e.g., m sensing values) (sensor data set) and output predicted values for each corresponding sensing data set. For example, the second prediction data is the prediction value (first prediction value) for the first m sensing values (first sensing data set), and the prediction value (first prediction value) for the next m sensing values (second sensing data set). It may include a predicted value (second predicted value), ... , a predicted value (nth predicted value) for the nth m sensing values (nth sensing data set). Each prediction value may include a prediction probability value corresponding to a first pose and a prediction probability value corresponding to a second pose associated with the LOS for the corresponding sensing data set.

According to one embodiment, the machine learning unit 1050 may use a third CNN model to generate third prediction data for classification of poses related to NLOS from input sensor data. As an example, the third CNN model may perform processing in units of a preset number of sensing values (e.g., m sensing values) (sensor data set) and output predicted values for each corresponding sensing data set. For example, the third prediction data is the prediction value (first prediction value) for the first m sensing values (first sensing data set), and the prediction value (first prediction value) for the next m sensing values (second sensing data set). It may include a predicted value (second predicted value), ... , a predicted value (nth predicted value) for the nth m sensing values (nth sensing data set). Each prediction value may include a prediction probability value corresponding to a first pose and a prediction probability value corresponding to a second pose, associated with NLOS, for the corresponding sensing data set.

The machine learning unit 1050 may transmit prediction data (eg, first prediction data, second prediction data, and/or third prediction data) to the central control unit 1070 and/or the moving average filter 1064.

The central control unit 1070 may transfer the received prediction data to the storage unit 1080. The storage unit 1080 may store the received prediction data and transmit the stored prediction data to the moving average filter 1064.

The moving average filter 1064 may filter each prediction data. This filtered prediction data can be used for final classification of poses. Filtered prediction data can be used by various applications. As an example, an application may provide different services or service-related parameters (e.g., thresholds) depending on the pose classified based on the filtered prediction data.

Figure 11 shows a method for classifying a pose of an electronic device according to an embodiment of the present disclosure.

In the embodiment of FIG. 11, the electronic device may be a user's electronic device (user device). The description of each step in FIG. 11 may refer to the description of the above-described operations/steps in FIGS. 4 to 10, unless they contradict each other.

Referring to FIG. 11, the electronic device can use the learned first CNN model to obtain first prediction data for classifying whether the signal corresponding to the UWB CIR is a LOS signal or an NLOS signal based on the UWB CIR data. There is (1110).

The electronic device may identify whether the signal is classified as a LOS signal (1120).

If the signal is classified as a LOS signal, the electronic device may use the learned second CNN model to obtain second prediction data for classifying the pose based on sensor data corresponding to UWB CIR data (1130) .

If the signal is classified as an NLOS signal, the electronic device may use the learned third CNN model to obtain third prediction data for classifying the pose based on sensor data corresponding to UWB CIR data (1140) .

The electronic device may perform filtering on at least two of the first prediction data, the second prediction data, and the third prediction data using the first filtering method (1150).

The electronic device may classify the pose based on the filtered data (1160).

As an example, a second CNN model may be trained using sensor data associated with LOS signals, and a third CNN model may be trained using sensor data associated with NLOS signals.

As an embodiment, when the signal is classified as a LOS signal, the electronic device may classify one of at least one pose associated with the LOS signal as a pose based on the filtered first prediction data and the filtered second prediction data. .

As an embodiment, when the signal is classified as an NLOS signal, the electronic device may classify one of at least one pose associated with the NLOS signal as a pose based on the filtered first prediction data and the filtered third prediction data. .

As an example, the first filtering method may be a moving average filter method.

As an example, the sensor data may be sensor data obtained from an IMU sensor.

As an embodiment, the UWB CIR data includes effective CIR data with noise removed from CIR data obtained based on UWB signals for DL-TDoA ranging, or UWB signals for DS-TWR ranging. It may include effective CIR data in which noise has been removed from CIR data obtained based on the CIR data.

FIG. 12 is a diagram illustrating the structure of an electronic device according to an embodiment of the present disclosure.

In the embodiment of FIG. 12, the electronic device may be a user's electronic device (user device) or a server.

Referring to FIG. 12, the electronic device may include a transceiver 1210, a control unit 1220, and a storage unit 1230. In the present disclosure, the control unit may be defined as a circuit or application-specific integrated circuit or at least one processor.

The transceiver 1210 can transmit and receive signals with other network entities. For example, the transceiver unit 1210 may transmit and receive data for commissioning.

The control unit 1220 may control the overall operation of the electronic device according to the embodiment proposed in this disclosure. For example, the control unit 1220 may control signal flow between each block to perform operations according to the flowchart described above. Specifically, the control unit 1220 may control, for example, the operation of the electronic device described with reference to FIGS. 1 to 11 .

The storage unit 1230 may store at least one of information transmitted and received through the transmitting and receiving unit 1210 and information generated through the control unit 1220. For example, the storage unit 1230 may store information and data necessary for pose classification described with reference to FIGS. 1 to 11 .

In the specific embodiments of the present disclosure described above, components included in the present disclosure are expressed in singular or plural numbers depending on the specific embodiment presented. However, singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components, and even components expressed in plural may be composed of singular or singular. Even expressed components may be composed of plural elements.

Meanwhile, in the detailed description of the present disclosure, specific embodiments have been described, but of course, various modifications are possible without departing from the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, but should be determined not only by the scope of the patent claims described later, but also by the scope of this patent claim and equivalents.

Claims (12)

In the method of the electronic device,
Using the learned first convolutional neural network (CNN) model, determine whether the signal corresponding to the UWB CIR is a line of signt (LOS) signal or a non-LOS (NLOS) signal based on UWB channel impulse response (CIR) data. Obtaining first prediction data for classifying;
If the signal is classified as the LOS signal, using a learned second CNN model, obtaining second prediction data for classifying a pose based on sensor data corresponding to the UWB CIR data;
If the signal is classified as the NLOS signal, using a learned third CNN model, obtaining third prediction data for classifying a pose based on sensor data corresponding to the UWB CIR data;
performing filtering on at least two of the first prediction data, the second prediction data, and the third prediction data using a first filtering method; and
Based on the filtered data, classifying the pose,
The second CNN model is learned using sensor data associated with the LOS signal, and the third CNN model is learned using sensor data associated with the NLOS signal. The method of claim 1, wherein classifying the pose comprises:
When the signal is classified as the LOS signal, classifying one of at least one pose associated with the LOS signal as the pose based on the filtered first prediction data and the filtered second prediction data. How to. The method of claim 2, wherein classifying the pose includes:
When the signal is classified as the NLOS signal, classifying one of at least one pose associated with the NLOS signal as the pose based on the filtered first prediction data and the filtered third prediction data. How to. According to paragraph 3,
The first filtering method is a moving average filter method. According to paragraph 1,
The method wherein the sensor data is sensor data obtained from an inertial measurement unit (IMU) sensor. According to paragraph 1,
The UWB CIR data includes effective CIR data with noise removed from CIR data obtained based on UWB signals for downlink-time difference of arrival (DL-TDoA) ranging, or DS-TWR (double sided-two way ranging) A method comprising effective CIR data in which noise is removed from CIR data obtained based on UWB signals for ranging. In electronic devices,
transceiver; and
A controller coupled to the transceiver, wherein the controller:
Using the learned first convolutional neural network (CNN) model, determine whether the signal corresponding to the UWB CIR is a line of signt (LOS) signal or a non-LOS (NLOS) signal based on UWB channel impulse response (CIR) data. Obtain first prediction data to classify,
If the signal is classified as the LOS signal, obtain second prediction data for classifying the pose based on sensor data corresponding to the UWB CIR data using the learned second CNN model,
If the signal is classified as the NLOS signal, obtain third prediction data for classifying a pose based on sensor data corresponding to the UWB CIR data using a learned third CNN model,
Using a first filtering method, filtering at least two of the first prediction data, the second prediction data, and the third prediction data,
Configured to classify the pose based on the filtered data,
The second CNN model is learned using sensor data associated with the LOS signal, and the third CNN model is learned using sensor data associated with the NLOS signal. 8. The method of claim 7, wherein the controller:
When the signal is classified as the LOS signal, further configured to classify one of at least one pose associated with the LOS signal as the pose based on the filtered first prediction data and the filtered second prediction data. , electronic devices. The method of claim 8, wherein the controller:
When the signal is classified as the NLOS signal, further configured to classify one of at least one pose associated with the NLOS signal as the pose based on the filtered first prediction data and the filtered third prediction data. , electronic devices. According to clause 9,
The first filtering method is a moving average filter method. In clause 7,
The sensor data is sensor data obtained from an inertial measurement unit (IMU) sensor. In clause 7,
The UWB CIR data includes effective CIR data in which noise has been removed from CIR data obtained based on UWB signals for downlink-time difference of arrival (DL-TDoA) ranging, or DS-TWR (double sided-two way ranging) An electronic device including effective CIR data in which noise is removed from CIR data obtained based on UWB signals for ranging.

Images