KR102585102B1 - System for providing augmented reality based on gps information using metaverse service - Google Patents

Abstract

The location-based augmented reality system using the metaverse service of the present invention includes a user terminal including a location tracking unit capable of recording location information of latitude and longitude, a camera unit for taking images, and a display unit for displaying a screen; As a metaverse service that allows the user terminal to access the network, the location information and video transmitted from the user terminal are used to set a corresponding location within the metaverse service, and the object included in the video is located at the location within the corresponding metaverse service. It includes a metaverse service server that displays an object corresponding to the metaverse service server, the metaverse service server includes a location confirmation unit that checks location information transmitted from the user terminal and sets a corresponding location within the metaverse service, and An object confirmation unit that checks the object in the video transmitted from the user terminal and determines whether it is implemented in the metaverse service, and the object confirmed by the object confirmation unit is identified based on the corresponding location information in the metaverse service. An object implementation unit that enables display, and if the object implementation unit determines that display within the metaverse service is difficult, it uses objects stored in advance or designated by the user to correspond to objects in the video transmitted from the user terminal. It includes an object creation unit that creates objects for the metaverse service.

Description

Location-based augmented reality system using metaverse service {SYSTEM FOR PROVIDING AUGMENTED REALITY BASED ON GPS INFORMATION USING METAVERSE SERVICE}

The present invention uses the metaverse service to display offline objects in the metaverse service based on the user's location, or displays objects in the metaverse service on the user terminal, thereby providing offline and metaverse services based on the location of the user terminal. It is about a system that enables various applications.

Metaverse is a compound word of 'meta', meaning virtuality and transcendence, and 'universe', meaning world and universe, and means a three-dimensional virtual world linked to reality.

Recently, as the global COVID-19 pandemic has prolonged, contact and movement between people has been restricted, and 5G communication networks that can transmit large amounts of information at high speed have been expanded, various forms of virtual worlds supporting lifestyle or game-type virtual worlds have been created. Metaverse services are emerging.

These metaverse services support functions that allow changing the interface design that configures the metaverse environment. Users can utilize these functions to vary the interface design that configures the metaverse environment as the user wants without physical restrictions. By changing it or applying decorative effects, you can get fun and satisfaction.

When using this metaverse service, environments and objects that are closer to offline/reality can be used online, and ways to provide a variety of services beyond simple fun are being studied.

The present invention seeks to propose a system that enables various services by linking objects in the real world and objects displayed within the metaverse service based on the user's location confirmed based on the user terminal.

In particular, since object images taken by users can be used based on the location of the user terminal, content provision within the metaverse service can be based on the user's requirements or tastes, making provision and use of services more efficient. I would like to propose a system that makes this possible.

The location-based augmented reality system using the metaverse service of the present invention includes a user terminal including a location tracking unit capable of recording location information of latitude and longitude, a camera unit for taking images, and a display unit for displaying a screen; As a metaverse service that allows the user terminal to access the network, the location information and video transmitted from the user terminal are used to set a corresponding location within the metaverse service, and the object included in the video is located at the location within the corresponding metaverse service. It includes a metaverse service server that displays an object corresponding to the metaverse service server, the metaverse service server includes a location confirmation unit that checks location information transmitted from the user terminal and sets a corresponding location within the metaverse service, and An object confirmation unit that checks the object in the video transmitted from the user terminal and determines whether it is implemented in the metaverse service, and the object confirmed by the object confirmation unit is identified based on the corresponding location information in the metaverse service. An object implementation unit that enables display, and if the object implementation unit determines that display within the metaverse service is difficult, it uses objects stored in advance or designated by the user to correspond to objects in the video transmitted from the user terminal. It includes an object creation unit that creates objects for the metaverse service.

In addition, the metaverse service server includes a customized analysis unit that analyzes the tastes of each user using the location information transmitted from the user terminal and objects in the captured video, and the taste information of each user analyzed by the customized analysis unit. Based on this, it further includes a customized suggestion unit that provides a customized service to the user within the metaverse service, and a content DB that stores objects that can be displayed or provided or content that can be serviced within the metaverse service.

In addition, the customized suggestion unit is characterized in that it suggests an offline location to the user terminal based on the taste information for each user, or suggests a location or product within the metaverse service.

The proposed system of the present invention enables various services by linking objects in the real world and objects displayed within the metaverse service based on the user's location confirmed based on the user terminal. There is an advantage.

Figure 1 is a block diagram showing the overall configuration of a location-based augmented reality system using the metaverse service of the present invention.
Figures 2 to 4 are diagrams showing an example in which an object is displayed within the metaverse service based on the location of the user terminal according to this embodiment.
Figure 5 is a diagram showing an example of an object in the metaverse service being called and displayed on a user terminal according to this embodiment.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention.

Meanwhile, the blocks in the attached block diagram or the steps in the flow chart are computer program instructions that are mounted on the processor or memory of equipment capable of data processing, such as general-purpose computers, special-purpose computers, portable laptop computers, and network computers, and perform designated functions. It can be interpreted to mean. Because these computer program instructions can be stored in a memory provided in a computer device or in a computer-readable memory, the functions described in the blocks of a block diagram or the steps of a flow diagram can be produced as a manufactured product containing instruction means to perform them. It could be. In addition, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). Additionally, it should be noted that in some alternative embodiments, it is possible for functions mentioned in blocks or steps to be executed in a different order. For example, two blocks or steps shown in succession may be performed substantially simultaneously or in reverse order, and in some cases, some blocks or steps may be omitted.

Hereinafter, in order to enable those skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram showing the overall configuration of a location-based augmented reality system using the metaverse service of the present invention.

A Metaverse user who uses the Metaverse service using a computer (PC) or smart device accesses the Metaverse service operation server (10) via the Internet/network to go to work at a company within the Metaverse or to attend an event/event within the Metaverse. By attending the event, you can optionally participate in various events, purchase products, etc. Objects, which are objects displayed within this metaverse service, and various services utilizing the objects are provided to offline users, that is, based on the location of the user terminal (1) and within the video captured by the user terminal (1). Let's explain how to link objects with the metaverse service.

The user terminal 1, which can connect to the metaverse service server 10 through the Internet, network, etc., includes a camera unit 11 that allows the user to photograph real objects offline, and a camera unit 11 that allows the user, that is, the user terminal It includes a location tracking unit 12 that tracks location information (GPS), which is latitude and longitude information, and a display unit 13 that displays offline objects or objects provided from a network-connected metaverse service. In the case of a terminal equipped with a camera unit, a location tracking unit, and a display unit, it can be a variety of electronic devices such as smart devices such as smartphones and tablet PCs.

Meanwhile, the metaverse service server 10, to which the user terminal 1 can connect to the network, has a location confirmation unit that checks the location information transmitted from the user terminal 1 and confirms or sets the corresponding location within the metaverse service. (110), an object confirmation unit 120 that checks the object in the video transmitted from the user terminal 1 and determines whether it is implemented in the metaverse service, and detects the object based on the corresponding location information in the metaverse service. An object implementation unit 130 that causes the object to be displayed, and when the object implementation unit 130 determines that it is difficult to display within the metaverse service, it is transmitted from the user terminal using objects stored in advance or designated by the user. An object creation unit 140 that creates an object for the metaverse service corresponding to an object in the image, and an object creation unit 140 that generates an object in the image captured by the user and the location information transmitted from the user terminal 1 to create patterns or tastes for each user. A customized analysis unit 160 that analyzes, a customized suggestion unit 170 that provides a customized service to the user within the metaverse service based on the taste information for each user analyzed by the customized analysis unit 160, and a metaverse service. It includes a content DB 150 in which objects that can be displayed or provided or content that can be serviced within the bus service are stored.

In order to provide services according to this embodiment, each configuration of the metaverse service will be described in more detail together with the user terminal.

In detail, the location confirmation unit 110 receives the image captured by the user terminal 1 or GPS information generated when the camera unit 11 operates to photograph an offline object, that is, location information, and receives the corresponding It verifies the location of the user terminal and matches the confirmed offline location with the location within the metaverse service.

That is, the metaverse service of this embodiment is configured to be linked to offline location information, so that objects in offline locations and objects in locations within the metaverse service can also be configured to be linked to each other. For example, in the case of a video taken by a user at Seoul Station, offline location information corresponding to Seoul Station is stored in the video. When the video (object) with the stored location information is delivered to the metaverse service, the location confirmation unit 110 sets and matches the location within the metaverse service that corresponds to the offline location.

In addition, the object confirmation unit 120 checks whether there is a displayable object, that is, a metaverse object, at a location within the metaverse service corresponding to the offline location transmitted from the user terminal, and checks the above for the confirmed metaverse object. It is controlled to be displayed at the corresponding location within the metaverse service through the object implementation unit 130. If a displayable object is not set at the corresponding location by the object confirmation unit 120, that is, a location within the metaverse service matched to an offline location by the location confirmation unit 110, the object creation unit ( 140) controls the creation of the object.

For example, when a user photographs a work of art at a specific offline location using the user terminal 1, the object confirmation unit 120 detects the object within the metaverse service confirmed through the location confirmation unit 110. After checking whether a displayable artwork is stored, the object implementation unit 130 displays the artwork image, which is the corresponding metaverse object, at a location within the metaverse service corresponding to the offline location. If the corresponding metaverse object is not stored, object creation is performed by the object creation unit 140. At this time, the object creation unit 140 creates an object ( A work of art) is extracted, the video corresponding to the work of art is loaded from another network-connected site or DB, stored, and created as a new metaverse object that can be displayed at the corresponding location (location within the metaverse service).

Through this process, users capture the videos they have shot, that is, videos of interest, and register them in the Metaverse service, so that they can track the location they have moved to at any time and the metadata corresponding to the video shot at that location. You can enjoy bus objects. By linking the offline location, the location within the metaverse service corresponding to each object, which is an offline object, and the metaverse object, operators providing metaverse services can also provide a variety of services.

For example, the object implementation unit 130 or the customized suggestion unit 170 uses the user's location and the captured image to provide a metaverse object at the corresponding location, while providing various related information or providing information of the user's interests. By reflecting this, more related products (displayable objects) can be provided.

In the case of Figure 3, it is a diagram showing an object displayed within the metaverse service when a user photographs furniture using a user terminal, and the object confirmation unit 120 is a diagram showing an object displayed within the metaverse service corresponding to the photographing location. Objects that can be displayed at a location (for example, furniture) are displayed within the metaverse service. At this time, a pre-set user avatar is displayed together so that the user can estimate the size of the furniture, or the corresponding metaverse object (furniture) is displayed together within the background displayed at the location within the metaverse service, so that the user can adjust the mood or mood. Make it easy to check the color.

By ensuring that the actual offline location and the location within the metaverse service are interconnected, when businesses selling or advertising products display their products at a specific offline location, users can select the desired product (object) using their user terminal. Filmed and filmed videos can be used within the Metaverse service, making it possible to provide a service that allows not only advertising/sales of actual products but also transactions for Metaverse objects.

Meanwhile, the customized analysis unit 160 is based on the user's offline location confirmed through the location confirmation unit 110 and location information within the Metiverse service, and objects identified in the video captured by the user, It performs the role of analyzing preference information for each user.

In particular, the customized analysis unit 160 analyzes preference information for each user, and based on the results analyzed by the customized analysis unit 160, the customized suggestion unit 170 determines an offline location that the user may be interested in ( It is possible to suggest a location (place) or, conversely, a location (place) within the Metaverse service that may be of interest within the Metaverse service, suggest offline goods (actual goods), or products or services within the Metaverse service. It is also possible to suggest . For example, if we determine that a user is interested in interior design through frequent furniture photography, we suggest the location of a furniture store that would be useful to visit based on the user's offline residence, or furniture companies that can be displayed within the Metaverse service. It is possible to show fields and furniture.

This customized analysis unit 160 classifies and extracts areas that other users are interested in at that location based on location information such as the user's shooting location and moving location, or based on objects present in the image captured by the user. Thus, when another user photographs the same objects, items and objects of interest can be classified and extracted.

Based on this analysis, the customized suggestion unit 170 not only suggests services within the metaverse service, but also recommends places or products that users can check/experience offline, and this process Through this, the Metaverse service is recognized as a space closer to actual offline living patterns rather than a simple virtual space, and this makes it possible to provide not only simple fun but also a variety of services.

Figure 4 is a diagram showing an example in which an object is displayed within the metaverse service based on the location of the user terminal according to this embodiment.

For the video captured by the user terminal 1 at a specific location, the location confirmation unit 110 and the object confirmation unit 120 of the Metiverse service server 1 check offline location information and objects in the video, and Check the corresponding location within the service, and confirm and create the object to be displayed at the corresponding location so that it is displayed.

In particular, when an image containing a chroma key background is captured, the object confirmation unit 120 makes it easier to extract the object and ensures that the object is displayed in the background image displayed at the corresponding location within the metaverse service. By doing so, it is possible to provide a more vivid sense of reality when using services within the metaverse service.

In order to display objects in the real world, that is, offline, within the metaverse service, by using location information where offline objects are located, the metaverse service extracts objects that may be in that offline location and quickly verifies and verifies the objects. Displaying may be possible, and if the offline location is a place such as an exhibition, memorial hall, or concert hall with a specific purpose, customized services are possible because the user's interests can be identified not only by location but also by object.

Conversely, when there is a request to transmit an object in the metaverse service from the user terminal 1, the object data is transmitted from the metaverse service server 10 to the user terminal 1, and the user An object received from an offline background, that is, a virtual image, may be output through the display unit 13.

Figure 5 is a diagram showing an example of an object in the metaverse service being called and displayed on a user terminal according to this embodiment.

That is, when a request for object transmission is transmitted from the user terminal 1 to the metaverse service server 10, the location confirmation unit 110 of the metaverse service server 10 determines the offline location of the user terminal 1. and check the location within the corresponding metaverse service. Then, the object confirmation unit 120 confirms and determines an object that can be displayed at a location within the metaverse service and then transmits it to the user terminal 1, and displays the display unit 13 of the user terminal 1. ), the received object may be displayed in the offline background captured by the camera unit 11.

Through this process, objects provided by the Metaverse service can be received and displayed with just a camera and display without the need to take a separate identification code such as a QR code, so it not only provides a virtual reality experience to users, but also provides It has the advantage of enabling a variety of business services.

In the above, the present invention has been described with specific details such as specific components and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , a person skilled in the art to which the present invention pertains can make various modifications and variations from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all modifications equivalent to or equivalent to the scope of the claims fall within the scope of the spirit of the present invention. They will say they do it.

Claims (5)

A user terminal including a location tracking unit capable of recording location information of latitude and longitude, a camera unit for taking images, and a display unit for displaying a screen;
As a metaverse service that allows the user terminal to access the network, the location information and video transmitted from the user terminal are used to set a corresponding location within the metaverse service, and the object included in the video is located at the location within the corresponding metaverse service. Includes a metaverse service server that displays objects corresponding to
The metaverse service server,
A location confirmation unit that checks the location information transmitted from the user terminal and sets a corresponding location within the metaverse service, and an object that checks the object in the video transmitted from the user terminal and determines whether it is implemented in the metaverse service. A confirmation unit, an object implementation unit that displays the object confirmed by the object confirmation unit based on corresponding location information in the metaverse service, and the object implementation unit determines that it is difficult to display the object within the metaverse service. In the case of doing so, an object creation unit that creates an object for the metaverse service corresponding to an object in the video transmitted from the user terminal using objects pre-stored or designated by the user, location information transmitted from the user terminal, and A customized analysis unit that analyzes the tastes of each user using objects in the captured video, and a customized suggestion unit that provides customized services to the user within the metaverse service based on the taste information of each user analyzed by the customized analysis unit. and a content DB in which objects that can be displayed or provided or content that can be serviced within the metaverse service are stored,
A location-based augmented reality system using the metaverse service, wherein the customized suggestion unit suggests an offline location to the user terminal based on the taste information for each user, or suggests a location or product within the metaverse service. According to claim 1,
The location confirmation unit reads offline location information included in the video transmitted from the user terminal and matches the corresponding location set in the metaverse service,
If the object confirmation unit determines that there is no displayable object within the metaverse service for the object included in the video, the object is created using content preset by the user or stored in the content DB. A location-based augmented reality system using the metaverse service, characterized by controlling the generation unit. According to claim 1,
When the metaverse service server receives a transmission request for an object that can be displayed on the user terminal from the user terminal,
The location confirmation unit reads the current location information of the user terminal and confirms the corresponding location within the metaverse service, and the object confirmation unit extracts an object that can be displayed at the corresponding location and transmits it to the user terminal. Location-based augmented reality system using bus service.
delete delete