CN114663491A - Method and system for providing information to a user in a scene - Google Patents

Abstract

Provided is a method and system for providing information to a user in a scene in which one or more sensors and one or more visual markers are deployed that can be used to sense or determine the scene The location information of the target in the device, the method includes: receiving information sent by a user's location obtaining device, the information includes spatial location information of the location obtaining device, wherein the location obtaining device scans the visual identifying its spatial location information by using a sign; identifying a user of the location obtaining device within the sensing range of the sensor based on the spatial location information of the location obtaining device; tracking the user through the sensor and updating the spatial location information of a user; and providing information to the user through an information receiving device of the user based on the spatial location information of the user.

Description

Method and system for providing information to users in a scene

technical field

The present invention relates to the field of information interaction, and in particular, to a method and system for providing information to users in a scene.

Background technique

The statements in this section are only for providing background information related to the technical solutions of the present application to help understanding, and they do not necessarily constitute prior art to the technical solutions of the present application.

In many scenarios, based on the needs of security, surveillance, public services, etc., sensors such as cameras and radars are deployed in the scenarios to sense, locate, and track people or objects that appear in the scenarios. However, these sensors can usually only realize one-way information transmission (that is, collect relevant information in the scene), and cannot provide information to the user in the scene based on this information (eg, the user's real-time location information), such as navigation information , instruction information, commercial promotion information, etc. In the prior art, in order to provide services to users in the scene, on-site manual service is usually adopted, which requires setting up some consultation desks and arranging service personnel at a certain density in the venue, which is costly and flexible. Low.

SUMMARY OF THE INVENTION

One aspect of the present invention relates to a method for providing information to a user in a scene in which one or more sensors and one or more visual markers are deployed, the sensors being capable of being used to sense or determine all the location information of the target in the scene, the method includes: receiving information sent by the user's location obtaining device, the information includes the spatial location information of the location obtaining device, wherein the location obtaining device scans all the the visual sign to determine its spatial position information; identify the user of the position obtaining device within the sensing range of the sensor based on the spatial position information of the position obtaining device; track the user through the sensor and update spatial location information of the user; and providing information to the user through an information receiving device of the user based on the spatial location information of the user.

In one embodiment, the location obtaining device is the user's mobile phone and the information receiving device is the user's glasses. In one embodiment, the location obtaining device and the information receiving device are the same device, such as a mobile phone or glasses.

Another aspect of the invention relates to a system for providing information to a user in a scene, the system comprising: one or more sensors deployed in the scene, the sensors capable of sensing or determining all location information of an object in the scene; one or more visual markers deployed in the scene; and a server configured to implement the methods described in the embodiments of the present application.

Another aspect of the present invention relates to a storage medium in which a computer program is stored, and when the computer program is executed by a processor, can be used to implement the method described in the embodiments of the present application.

Another aspect of the present invention relates to an electronic device, including a processor and a memory, where a computer program is stored in the memory. When the computer program is executed by the processor, the computer program can be used to implement the methods described in the embodiments of the present application. method.

Through the solution of the present invention, not only the location information of the user in the scene can be collected or monitored, but also information, such as navigation information, instruction information, business promotion information, etc., can be provided to the user based on the real-time location information of the user.

Description of drawings

Embodiments of the present invention are further described below with reference to the accompanying drawings, wherein:

Figure 1 shows an exemplary visual sign;

Figure 2 shows an optical communication device that can be used as a visual sign;

Figure 3 illustrates a system for providing information to a user in a scene through glasses, according to one embodiment;

Figure 4 illustrates a method for providing information to a user in a scene through glasses, according to one embodiment;

Figure 5 illustrates a system for providing information to a user in a scene through glasses, according to one embodiment;

Figure 6 illustrates a method for providing information to a user in a scene through glasses, according to one embodiment.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Visual signs refer to signs that can be recognized by the human eye or electronic devices, which can have various forms. In some embodiments, visual markers may be used to convey information that can be obtained by smart devices (eg, cell phones, smart glasses, etc.). For example, the visual sign may be an optical communication device capable of emitting encoded optical information, or the visual sign may be a graphic with encoded information, such as a two-dimensional code (eg, QR code, applet code), barcode, or the like. Figure 1 shows an exemplary visual sign with a specific black and white pattern. FIG. 2 shows an optical communication device 100 that can be used as a visual sign, which includes three light sources (respectively, a first light source 101, a second light source 102, and a third light source 103). The optical communication device 100 also includes a controller (not shown in FIG. 2 ) for selecting a corresponding driving mode for each light source according to the information to be communicated. For example, in different driving modes, the controller can use different driving signals to control the light-emitting manner of the light source, so that when the optical communication apparatus 100 is photographed by using a device with an imaging function, the imaging of the light source can present different images. Appearance (eg, different colors, patterns, brightness, etc.). By analyzing the imaging of the light sources in the optical communication device 100 , the driving mode of each light source at the moment can be analyzed, so as to analyze the information transmitted by the optical communication device 100 at the moment.

In order to provide corresponding services to users based on the visual logo, each visual logo may be assigned an identification information (ID), which is used to uniquely identify or identify the visual logo by the manufacturer, manager or user of the visual logo, etc. . The user can use the device to capture the image of the visual sign to obtain the identification information transmitted by the visual sign, so as to access the corresponding service based on the identification information, for example, visit the webpage associated with the identification information, obtain the identification information associated with the identification information. Other information (eg, position or gesture information of the visual landmark corresponding to the identification information) and so on. The devices mentioned herein can be, for example, devices carried or controlled by users (eg, mobile phones, tablet computers, smart glasses, AR glasses, smart helmets, smart watches, cars, etc.), or machines that can move autonomously (eg, drones, driverless cars, robots, etc.). The device can acquire the image containing the visual sign by collecting the image of the visual sign through the image acquisition device on it, and can identify the information transmitted by the visual sign and determine the relative position of the device relative to the visual sign by analyzing the imaging of the visual sign in the image. position or attitude information.

Sensors capable of sensing the location of objects may be various sensors capable of sensing or determining location information of objects in a scene, such as cameras, radars (eg, lidars, millimeter-wave radars), wireless signal transceivers, and the like. A target in a scene can be a person or an object in the scene. In the following embodiments, a camera is used as an example of a sensor for description.

Figure 3 illustrates a system for providing information to a user in a scene through glasses, the system including a visual sign 301, a camera 302, and a server (not shown in Figure 3), according to one embodiment. User 303 is in the scene and wears glasses 304 . The glasses 304 are provided with an image capturing device and can recognize the visual sign 301 through the image capturing device. The glasses 304 themselves may have the ability to directly access the network, for example, the glasses 304 may access the network by means of wifi, telecommunication network or the like. The glasses 304 may also not have the ability to directly access the network, but may indirectly access the network through a connection (eg, a Bluetooth connection or a wired connection) between it and the user's other devices (eg, a mobile phone, a watch, etc.).

The visual sign 301 and the camera 302 are each installed in the real scene in a specific position and attitude (hereinafter collectively referred to as "pose"). In one embodiment, the server may obtain the respective pose information of the camera and the visual marker, and may obtain relative pose information between the camera and the visual marker based on the respective pose information of the camera and the visual marker. In one embodiment, the server may also directly obtain the relative pose information between the camera and the visual marker. In this way, the server can obtain a transformation matrix between the camera coordinate system and the visual sign coordinate system, and the transformation matrix may include, for example, a rotation matrix R and a displacement vector t between the two coordinate systems. Through the transformation matrix between the camera coordinate system and the visual marker coordinate system, the coordinates in one coordinate system can be converted into the coordinates in the other coordinate system. The camera may be a camera installed in a fixed position and having a fixed orientation, but it is understood that the camera may also be a camera that can move (for example, the position or direction can be changed), as long as its current pose information can be determined. The current pose information of the camera can be set by the server, and the movement of the camera can be controlled based on the pose information, or the movement of the camera can be controlled by the camera itself or other devices, and the current pose information of the camera can be sent to the server. In some embodiments, more than one camera may be included in the system, and more than one visual sign may also be included.

In one embodiment, a scene coordinate system (which may also be referred to as a real world coordinate system) may be established for the real scene, and the distance between the camera coordinate system and the scene coordinate system may be determined based on the pose information of the camera in the real scene and the transformation matrix between the visual landmark coordinate system and the scene coordinate system is determined based on the pose information of the visual landmark in the real scene. In this case, it is possible to convert the coordinates in the camera coordinate system or the visual landmark coordinate system to the coordinates in the scene coordinate system without transforming between the camera coordinate system and the visual landmark coordinate system, but it is understood that the camera and visual The relative pose information or transformation matrix between the landmarks can still be known by the server. Therefore, in this application, having a relative pose between the camera and the visual sign refers to objectively having a relative pose between the two, and does not require the system to pre-store the relative pose information between the two above or Use this relative pose information. For example, in one embodiment, only the pose information of the camera and the visual marker in the scene coordinate system may be stored in the system, and the relative poses of the two may not be calculated or used.

The camera is used to track objects in a real scene, which may be stationary or moving, such as persons, stationary objects, movable objects, etc. in the scene. A camera can be used to track the position of a person or object in a real scene by various methods in the prior art. For example, in the case of using a single monocular camera, the location information of objects in the scene can be determined in combination with scene information (eg, information on the plane on which a person or object in the scene is located). For the case of using a binocular camera, the position information of the target can be determined according to the position of the target in the field of view of the camera and the depth information of the target. In the case of using multiple cameras, the position information of the target can be determined according to the position of the target in the field of view of each camera.

It can be understood that the system may have multiple visual signs or multiple cameras, and the fields of view of the multiple cameras may be continuous or discontinuous.

Fig. 4 shows a method for providing information to a user in a scene through glasses according to one embodiment, the method can be implemented using the system shown in Fig. 3, and can include the following steps:

Step 401: Receive information sent by the glasses, where the information includes spatial position information of the glasses.

In one embodiment, the user 303 may use the glasses 304 to determine the spatial location information of the glasses 304 by scanning the visual markers 301 deployed in the scene. The user 303 may send information to the server through the glasses 304, and the information may include the spatial position information of the glasses 304, and the spatial position information may be the spatial position information of the glasses 304 relative to the visual sign 301 or the spatial position information of the glasses 304 in the scene . In one embodiment, an image of the visual sign 301 may be collected using the glasses 304; identification information of the visual sign 301 and spatial position information of the glasses 304 relative to the visual sign 301 are determined by analyzing the captured image of the visual sign 301; The identification information of 301 determines the position and attitude information of the visual mark 301 in space; and based on the position and attitude information of the visual mark 301 in space and the spatial position information of the glasses 304 relative to the visual mark 301, determine the position of the glasses 304 in the scene. Spatial location information. In one embodiment, the glasses 304 can send the identification information of the visual marker 301 and the spatial position information of the glasses 304 relative to the visual marker 301 to the server, so that the server can determine the spatial position information of the glasses 304 in the scene.

In one embodiment, the glasses 304 can also be used to scan the visual markers 301 to determine the gesture information of the glasses 304 relative to the visual markers 301 or the gesture information of the glasses 304 in the scene, and the gesture information can be sent to the server.

In one embodiment, the spatial position information and attitude information of the glasses may be the spatial position information and attitude information of the glasses when scanning the visual mark, or the real-time position information and attitude information at any moment after scanning the visual mark. For example, glasses can determine their initial spatial position information and attitude information when scanning visual landmarks, and then use various sensors built into the glasses (eg, acceleration sensors, magnetic sensors, orientation sensors, gravity sensors, gyroscopes, cameras, etc.) The real-time position and/or attitude of the glasses is determined by measuring or tracking its position change and/or attitude change by methods known in the art (eg, inertial navigation, visual odometry, SLAM, VSLAM, SFM, etc.).

The spatial position information of the glasses received by the server may be coordinate information, but is not limited thereto, any information that can be used to derive the spatial position of the glasses belongs to the spatial position information. In one embodiment, the spatial position information of the glasses received by the server may be an image of a visual sign captured by the glasses, and the server may determine the spatial position of the glasses according to the image. Similarly, any information that can be used to derive the pose of the glasses is pose information, which in one embodiment may be an image of a visual landmark captured by the glasses.

In one embodiment, in addition to the spatial location information of the glasses, the information sent by the glasses may also include information related to the glasses or their users, such as service request information, help information, alarm information, identification information (such as phone numbers, APP account information) etc.

In one embodiment, the glasses themselves may be capable of directly accessing the network, so that the server may receive information directly from the glasses. In another embodiment, the glasses may not have the ability to directly access the network, but indirectly access the network through a connection between it and, for example, the user's mobile phone. In this case, the server may use an intermediate device such as a mobile phone. Receive information sent by glasses.

Step 402: Identify the user of the glasses in the image captured by the camera based on the spatial position information of the glasses.

Various feasible ways can be used to identify the user of the glasses in the image captured by the camera through the spatial position information of the glasses.

In one embodiment, the imaging position of the user of the glasses in the image captured by the camera may be determined based on the spatial position information of the glasses, and the user in the image captured by the camera is identified according to the imaging position.

Since the user usually wears glasses to scan the visual sign, the spatial position of the user can be inferred according to the spatial position of the glasses, and then the imaging position of the user in the image captured by the camera can be determined according to the spatial position of the user. The imaging position of the glasses in the image captured by the camera can also be determined according to the spatial position of the glasses, and then the imaging position of the user can be inferred according to the imaging position of the glasses.

In one embodiment, a pre-established mapping relationship between one or more spatial positions (not necessarily all) in the scene and one or more imaging positions in the image captured by the camera and the space of the glasses may be used position information to determine the imaging position of the user in the image captured by the camera. For example, for a hall scene, several spatial positions on the floor of the hall can be selected, and the imaging positions of these positions in the image captured by the camera can be determined. After that, the mapping relationship between these spatial positions and the imaging positions can be established, and the An imaging position corresponding to a certain spatial position is deduced based on the mapping relationship.

In one embodiment, the imaging position of the user of the glasses in the image captured by the camera may be determined based on the spatial position information of the glasses and the pose information of the camera, where the pose information of the camera may be its pose in the scene information or its pose information relative to the visual landmark.

After the imaging position of the user in the image captured by the camera is determined, the user can be identified in the image according to the imaging position, so that the user can be tracked later. For example, a user closest to the imaging position or a user whose distance to the imaging position satisfies a predetermined condition may be selected.

In one embodiment, the spatial position information of the glasses may be compared with the spatial position information of one or more users determined according to the tracking result of the camera to identify the user of the glasses in the image captured by the camera. A camera can be used to determine the spatial position of a person or object in a real scene through various methods in the prior art. For example, in the case of using a single monocular camera, the location information of objects in the scene can be determined in combination with scene information (eg, information on the plane on which a person or object in the scene is located). For the case of using a binocular camera, the position information of the target can be determined according to the position of the target in the field of view of the camera and the depth information of the target. In the case of using multiple cameras, the position information of the target can be determined according to the position of the target in the field of view of each camera. In one embodiment, the spatial location information of one or more users may also be determined by using images captured by a camera in combination with lidar and the like.

In one embodiment, if there are multiple users near the spatial position of the glasses, the positions of the multiple users can be tracked through the camera, and the real-time spatial position information of the glasses (for example, satellite positioning information or obtained through the sensors of the glasses) can be received. position information), and identify the user wearing the glasses by comparing the positions of the multiple users with the real-time spatial position information of the glasses.

In one embodiment, if there are multiple users near the spatial position of the glasses, the characteristic information of the user wearing the glasses (for example, the characteristic information for face recognition) may be determined based on the information sent by the glasses, and collected by the camera. feature information of the multiple users, and identify the user wearing the glasses by comparing the feature information of the multiple users with the feature information of the user wearing the glasses.

Step 403: Track the user through the camera and update the spatial location information of the user.

In one embodiment, a camera may be used to track the user and update the imaging position of the user, and determine the spatial position information of the user based on the updated imaging position. Various visual tracking methods known in the art can be used to track the user in the field of view of the camera and update the imaging position of the user. The camera can remain stationary or move while tracking the user. In one embodiment, in tracking the user, multiple cameras may be used, which may have a continuous field of view or a discontinuous field of view. Where the field of view is discontinuous, the user's characteristics can be recorded and re-identified and tracked when the user re-enters the field of view of one or more cameras.

In one embodiment, a pre-established mapping relationship between one or more spatial positions (not necessarily all) in the scene and one or more imaging positions in the image captured by the camera and the imaging positions may be used, to determine the user's spatial location information. In one embodiment, the spatial position information of the user may be determined based on the pose information of the camera and the imaging position. For example, in the case of using a depth camera or a multi-camera camera, the direction of the user relative to the camera can be determined based on the imaging position, the depth information can be used to determine the distance of the user relative to the camera, so as to determine the position of the user relative to the camera, and then , the spatial position information of the user can be further determined based on the pose information of the camera. In one embodiment, the distance of the user relative to the camera may be estimated based on the imaging size of the user, and the spatial position information of the user may be determined based on the pose information of the camera and the imaging position. In one embodiment, the distance of the user relative to the camera may be determined by using a lidar or the like installed on the camera, and the spatial position information of the user may be determined based on the pose information of the camera and the imaging position. In one embodiment, if the visual fields of multiple cameras cover the user at the same time, the multiple cameras can be used to jointly determine the spatial location information of the user. In one embodiment, the spatial position information of the user may be determined based on the pose information of the camera, the imaging position, and optional other information (eg, coordinate information of the ground in the scene).

In one embodiment, the user's gesture information may also be determined based on the tracking result of the user by the camera.

Step 404: Provide information to the user through the user's glasses based on the user's spatial location information.

Knowing the user's spatial location information, the user can be provided with various required information, such as navigation information, instruction information, tutorial information, advertising information, other information related to location-based services, and the like. In one embodiment, the above information may be provided visually, audibly, or the like. In one embodiment, a virtual object may be superimposed on the display medium of the glasses, and the virtual object may be, for example, an icon (eg, a navigation icon), a picture, a text, or the like.

In one embodiment, the glasses themselves may have the ability to directly access the network, so that the glasses may directly receive indication information from the server. In another embodiment, the glasses may not have the ability to directly access the network, but indirectly access the network through a connection between it and, for example, the user's mobile phone, in this case, the glasses may pass through an intermediate device such as a mobile phone Receive instructions from the server.

In one embodiment, information may be further provided to the user in conjunction with the glasses or the gesture information of the user thereof. The posture information of the glasses or the user thereof may be determined by the glasses, or the posture information of the user may be determined by the user image captured by the camera, and the posture information may include the orientation information of the user. In one embodiment, the posture information of the glasses can be obtained through its built-in sensors, for example, by tracking the initial posture or directly determined by the built-in sensors of the glasses (for example, a gravity sensor, a magnetic sensor, an orientation sensor, etc.). . The server may directly receive the gesture information from the glasses, or receive the gesture information through an intermediate device such as a mobile phone.

The steps in the method shown in FIG. 4 may be implemented by the server in the system shown in FIG. 3 , but it is understood that one or more of these steps may also be implemented by other devices.

FIG. 5 shows a system for providing information to a user in a scene through glasses, including a visual sign 501, a camera 502, and a server (not shown in FIG. 5), according to one embodiment. A user 503 is in the scene and carries glasses 504 and a cell phone 505 . The mobile phone 505 can recognize the visual sign 501 through the image capture device on it, so the glasses 504 may not have an image capture device, or although there is an image capture device on it, the image capture device may not have the ability to recognize the visual sign 501. . The glasses 504 themselves may have the ability to directly access the network, for example, the glasses 504 may access the network by means of wifi, a telecommunication network, or the like. The glasses 504 may also not have the ability to directly access the network, but may indirectly access the network through a connection between it and the mobile phone 505 (eg, a Bluetooth connection or a wired connection).

FIG. 6 illustrates a method of providing information to a user in a scene through glasses, which may be implemented using the system shown in FIG. 5 , according to one embodiment. The method includes the following steps (part of the steps are similar to the steps in FIG. 4 , and will not be repeated here, but it can be understood that the content described for each step in FIG. 4 can also be applied to the corresponding steps in FIG. 6 ):

Step 601: Receive information sent by the user's mobile phone, where the information includes the spatial location information of the mobile phone.

The user can use the mobile phone to determine the spatial location information of the mobile phone by scanning the visual landmarks deployed in the scene. In one embodiment, the gesture information of the mobile phone can also be determined by scanning the visual sign, and the gesture information can be sent to the server.

Step 602: Identify the user of the mobile phone in the image captured by the camera based on the spatial location information of the mobile phone.

In one embodiment, the imaging position of the user of the mobile phone in the image captured by the camera may be determined based on the spatial position information of the cell phone, and the user in the image captured by the camera is identified according to the imaging position. Since the user usually holds the mobile phone to scan the visual sign, the spatial position of the user can be inferred according to the spatial position of the glasses, and then the imaging position of the user in the image captured by the camera can be determined according to the spatial position of the user. The imaging position of the mobile phone in the image captured by the camera can also be determined according to the spatial position of the mobile phone, and then the imaging position of the user can be inferred according to the imaging position of the mobile phone.

In one embodiment, the spatial location information of the mobile phone may be compared with the spatial location information of one or more users determined according to the tracking result of the camera to identify the user of the mobile phone in the image captured by the camera.

Step 603: Track the user through the camera and update the spatial location information of the user.

In one embodiment, the user's gesture information can also be determined.

Step 604: Provide information to the user through the user's glasses based on the user's spatial location information.

In one embodiment, the glasses themselves may have the ability to directly access the network, so that the glasses may directly receive indication information from the server. In another embodiment, the glasses may not have the ability to directly access the network, but indirectly access the network through a connection between it and, for example, the user's mobile phone, in this case, the glasses may pass through an intermediate device such as a mobile phone Receive instructions from the server. For example, the server may first send first information to the user's mobile phone, and then the mobile phone may send second information (the second information may be the same as or different from the first information) to the glasses based on the first information, so as to provide the user with information based on the glasses through the glasses. location services.

In one embodiment, information may be further provided to the user in conjunction with the glasses or the gesture information of the user thereof.

In one embodiment, the user may also not use the glasses, but only use the cell phone. In this way, in the above step 604, information can be provided to the user through the user's mobile phone based on the user's spatial location information. In one embodiment, the information may be further provided to the user in combination with the gesture information of the mobile phone or its user. The gesture information of the user can be determined through the mobile phone, or the user's gesture information can be determined through the user image captured by the camera. In one embodiment, the gesture information of the mobile phone can be obtained through its built-in sensor.

In this application, a device for scanning a visual sign to determine its spatial location information may be referred to as a "position acquisition device", and a device for providing information to a user may be referred to as an "information receiving device". It can be understood from the above description of this application that the location obtaining device and the information receiving device may be the same device, such as the user's mobile phone or the user's glasses; the location obtaining device and the information receiving device may also be different devices, such as a mobile phone and a user's glasses respectively. Glasses.

In the above embodiments, a camera is used as an example of a sensor for description, but it can be understood that the embodiments herein can also be applied to any other sensor capable of sensing the target position, such as lidar, millimeter-wave radar, wireless signal transceiving device, etc.

It should be noted that the glasses in this application may be AR glasses, smart glasses, or any other glasses that can be used to present information to the user. The glasses in this application also include glasses formed by adding components or inserts to ordinary optical glasses, for example, glasses formed by adding a display device to ordinary optical glasses.

In one embodiment of the present invention, the present invention may be implemented in the form of a computer program. The computer program can be stored in various storage media (eg, hard disk, optical disk, flash memory, etc.), and when the computer program is executed by the processor, can be used to implement the method of the present invention.

In another embodiment of the present invention, the present invention may be implemented in the form of an electronic device. The electronic device includes a processor and a memory, and the memory stores a computer program that, when executed by the processor, can be used to implement the method of the present invention.

References herein to "various embodiments," "some embodiments," "one embodiment," or "an embodiment," etc. refer to the fact that a particular feature, structure, or property described in connection with the embodiment is included in the in at least one embodiment. Thus, appearances of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in an embodiment" in various places throughout this document are not necessarily referring to the same implementation example. Furthermore, the particular features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, particular features, structures, or properties shown or described in connection with one embodiment may be combined, in whole or in part, with the features, structures, or properties of one or more other embodiments without limitation, so long as the combination does not limit the Logical or not working. Expressions such as "according to A", "based on A", "by A" or "using A" appearing herein are meant to be non-exclusive, that is, "according to A" may encompass "according to A only" or Covers "according to A and B" unless specifically stated to mean "according to A only". In this application, for the sake of clarity, some schematic operation steps are described in a certain order, but those skilled in the art can understand that each of these operation steps is not essential, and some of them may be omitted or replaced by other steps. These operational steps also do not have to be performed sequentially in the manner shown, rather, some of these operational steps may be performed in a different order as practically desired, or in parallel, as long as the new implementation is not illogical or inoperable.

Having thus described several aspects of at least one embodiment of this invention, it will be appreciated that various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications and improvements are intended to be within the spirit and scope of the present invention. Although the present invention has been described by way of some embodiments, the present invention is not limited to the embodiments described herein, and various changes and changes can be made without departing from the scope of the present invention.

Claims (18)

1. A method for providing information to a user in a scene in which one or more sensors and one or more visual markers are deployed that can be used to sense or determine The location information of the target, the method includes: receiving information sent by a user's location obtaining device, the information including spatial location information of the location obtaining device, wherein the location obtaining device determines its spatial location information by scanning the visual sign; Identifying a user of the location obtaining device within the sensing range of the sensor based on the spatial location information of the location obtaining device; Track the user through the sensor and update the spatial location information of the user; and Information is provided to the user through the user's information receiving device based on the user's spatial location information. 2. The method of claim 1, wherein the sensor comprises one or more of the following: Camera; radar; Wireless signal transceiver. 3. The method of claim 1, further comprising: Determine the gesture information of the user through the sensor; or The gesture information of the information receiving device is determined by the user's information receiving device. 4. The method according to claim 3, wherein the providing information to the user through the user's information receiving device based on the user's spatial location information comprises: Information is provided to the user through the user's information receiving device based on the user's spatial position information and the gesture information. 5. The method of claim 1, wherein the sensor comprises a camera, and wherein the location acquisition within a sensing range of the sensor is identified based on spatial location information of the location acquisition device Users of the device include: determining the imaging position of the user of the position obtaining device in the image captured by the camera based on the spatial position information of the position obtaining device; and The user in the image captured by the camera is identified according to the imaging position. 6. The method according to claim 5, wherein the determining, based on the spatial position information of the position obtaining device, the imaging position of the user of the position obtaining device in the image captured by the camera comprises: The determination of the said the imaging position of the user of the position obtaining device in the image captured by the camera; or The imaging position of the user in the image captured by the camera is determined based on the spatial position information of the position obtaining device and the pose information of the camera. 7. The method of claim 1, wherein the identifying a user of the location obtaining device within a sensing range of the sensor based on the spatial location information of the location obtaining device comprises: comparing the spatial location information of the location obtaining device with the spatial location information of one or more users determined according to the sensing result of the sensor to identify the location obtaining within the sensing range of the sensor user of the device. 8. The method of claim 1, wherein the sensor comprises a camera, and wherein the tracking the user via the sensor and updating the user's spatial location information comprises: tracking the user through the camera and updating the imaging position of the user; and Spatial location information of the user is determined based on the updated imaging location. 9. The method of claim 8, wherein the determining the spatial location information of the user based on the updated imaging location comprises: Determine the space of the user based on a pre-established mapping relationship between one or more spatial positions in the scene and one or more imaging positions in the image captured by the camera and the updated imaging position location information; or The spatial position information of the user is determined based on the pose information of the camera and the updated imaging position. 10. The method of claim 1, wherein the location obtaining device to determine its spatial location information by scanning the visual landmark comprises: capturing an image of the visual landmark using the location obtaining device; determining the identification information of the visual sign and the position of the position obtaining device relative to the visual sign by analyzing the image; Obtain the position and attitude information of the visual sign in space through the identification information of the visual sign; Based on the position and attitude information of the visual marker in space and the position of the position obtaining device relative to the visual marker, the spatial position information of the position obtaining device is determined. 11. The method of claim 1, wherein identifying a user of the location obtaining device within a sensing range of the sensor based on the spatial location information of the location obtaining device comprises: If there are multiple users near the position obtaining device, then: track the positions of the multiple users through the sensor; receive real-time spatial position information of the position obtaining device; compare the positions of the multiple users The real-time spatial location information of the location obtaining device is used to identify the user of the location obtaining device. 12. The method of claim 1, wherein the sensor comprises a camera, and wherein the location acquisition within a sensing range of the sensor is identified based on spatial location information of the location acquisition device Users of the device include: If there are multiple users near the location obtaining device, then: determine the feature information of the users of the location obtaining device based on the information sent by the location obtaining device; collect the feature information of the multiple users through the camera ; identifying the user of the location obtaining device by comparing the feature information of the plurality of users with the feature information of the user of the location obtaining device. 13. The method of claim 1, wherein the providing information to the user through the user's information receiving device comprises: superimposing a virtual object on a display medium of the information receiving device. 14. The method of claim 1, wherein the location obtaining device is the user's cell phone and the information receiving device is the user's glasses. 15. The method of claim 1, wherein the location obtaining device and the information receiving device are the same device. 16. A system for providing information to a user in a scene, the system comprising: one or more sensors deployed in the scene, the sensors can be used to sense or determine location information of objects in the scene; one or more visual markers deployed in the scene; and A server configured to implement the method of any of claims 1-15. 17. A storage medium in which a computer program is stored which, when executed by a processor, can be used to implement the method of any one of claims 1-15. 18. An electronic device, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it can be used to implement the method of any one of claims 1-15 .

Images