CN103049729B - Based on the augmented reality method of Quick Response Code, system and terminal - Google Patents
Based on the augmented reality method of Quick Response Code, system and terminal Download PDFInfo
- Publication number
- CN103049729B CN103049729B CN201210587153.1A CN201210587153A CN103049729B CN 103049729 B CN103049729 B CN 103049729B CN 201210587153 A CN201210587153 A CN 201210587153A CN 103049729 B CN103049729 B CN 103049729B
- Authority
- CN
- China
- Prior art keywords
- quick response
- response code
- code
- image
- scene image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
- H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
- H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Graphics (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a kind of augmented reality method based on Quick Response Code, using Quick Response Code as the identification in augmented reality system and positioning mark, Quick Response Code in real scene image is decoded and directly generates Quick Response Code front elevation identical with it with re-encoding, tracking and matching is carried out with the image in 2 D code in the Quick Response Code front elevation generated and scene image, calculate homography matrix, accordingly, the invention also discloses the augmented reality system based on Quick Response Code and mobile terminal, both broken in traditional augmented reality application, the limitation that corresponding sample image just can carry out tracking and matching must be prestored in a database, turn avoid use legacy identification thing time to remote server template query with mate step, the system response delay because network transmission problems causes can be reduced, save the network traffic of user.
Description
Technical field
The present invention relates to mobile augmented reality field, particularly relate to a kind of augmented reality method based on Quick Response Code, system and mobile terminal.
Background technology
Quick Response Code is also known as two-dimensional bar code, record data symbol information at plane (on two-dimensional directional) the chequered with black and white figure that distributes according to certain rules with certain specific geometric figure, it is ingenious " 0 ", " 1 " bit stream concept utilizing formation computer-internal logical foundations on coding, and the geometrical body using several corresponding with scale-of-two is to represent word numerical information.
In recent years, Quick Response Code is widely applied, the Mobile solution being specifically designed to scanning recognition Quick Response Code on the market emerges in an endless stream, but after these application scanning Quick Response Codes, all directly present the Word message or the resource link such as video, webpage network address that obtain after Quick Response Code is resolved, use and dazzle not.
While Quick Response Code use is more and more general, augmented reality (AR, AugumentedReality) start slowly to enter public view, its core is added to by virtual information in scene that true environment presents in real time, utilize virtual information to supplement real scene, strengthen, allow virtual information synchronously show in real world.In existing augmented reality, virtual information be realized to be added on real scene, the relative position relation between video camera and real scene must be calculated, namely carry out registration by real scene image and sample image, obtain homography matrix.Therefore for existing augmented reality, if mobile terminal or augmented reality server end do not store the sample image of certain real scene or the characteristic point information of this sample image, then the fusion of virtual information and this real scene cannot be realized.
Summary of the invention
The object of this invention is to provide a kind of augmented reality method based on Quick Response Code, system and mobile terminal, when there is no sample image, the Quick Response Code front elevation consistent with the Quick Response Code in scene image is generated by Quick Response Code and recompile in decoding scene image, and the image in 2 D code in the scene image of to catch Quick Response Code front elevation and photographing module carries out feature detection and mates, calculate homography matrix, realize the certain deviation position in real scene Quick Response Code position or Quick Response Code, present the video that Quick Response Code is relevant, image, text, the multimedia messages of 3D model.
In order to realize foregoing invention object, the invention provides a kind of augmented reality method based on Quick Response Code, comprising:
Photographing module catches the real scene image containing Quick Response Code;
Quick Response Code in scanning scene image, and Quick Response Code is decoded, obtain the coded message of Quick Response Code, described coded message comprises: code system, version and resource information;
Re-encoding is carried out to the coded message of the Quick Response Code obtained, generates the Quick Response Code front elevation identical with the Quick Response Code code system in scene image and version; Resolve described resource information, to obtain virtual information corresponding to Quick Response Code simultaneously;
Image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out feature detection, obtains the feature interpretation of the two respectively; And carry out image registration according to the feature interpretation of the two, calculate the attitude of video camera, obtain homography matrix;
According to homography matrix, the certain deviation position of Quick Response Code position or Quick Response Code in real scene, plays up and virtual information corresponding with Quick Response Code described in output display.
Preferably, when resolve described resource information obtain be content of text time, content of text is played up as texture; Be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load according to virtual information type predetermined manner.
Preferably, described virtual information type comprises: one or more in video, image, text, 3D model.
Wherein, the Quick Response Code in described real scene image is conventional Quick Response Code or customization of two-dimension code; Resource information in customization of two-dimension code comprises resource identifier, resource type, resource load interface dimensions, one or more in rendering position drift rate.
Preferably, the image in 2 D code in described scene image of catching Quick Response Code front elevation and photographing module carries out feature detection, obtains the feature interpretation of the two respectively, is one of two kinds of modes below:
Mode one: the image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out full figure feature detection, obtains the feature interpretation of the two respectively;
Mode two: feature detection is carried out in the stabilized zone of the image in 2 D code in the scene image of only catching Quick Response Code front elevation and photographing module, obtains the feature interpretation of the two respectively.
Accordingly, the invention allows for a kind of augmented reality system based on Quick Response Code, comprising:
Photographing module, for catching the real scene image containing Quick Response Code;
Quick Response Code decoder module, for scanning the Quick Response Code in scene image, and decodes to Quick Response Code, and obtain the coded message of Quick Response Code, described coded message comprises: code system, version and resource information;
Quick Response Code coding module, carries out re-encoding to the coded message that described Quick Response Code decoder module parses, and generates the Quick Response Code front elevation identical with the Quick Response Code code system in scene image and version;
Source obtaining module, for resolving the resource information in described coded message, to obtain virtual information corresponding to Quick Response Code;
Image characteristics extraction module, carries out feature detection for the image in 2 D code in the scene image of catching Quick Response Code front elevation and photographing module, obtains the feature interpretation of the two respectively;
Image trace registration module, for carrying out image registration according to the image in 2 D code feature interpretation in Quick Response Code front elevation and scene image, calculating the attitude of video camera, obtaining homography matrix;
Play up display module, for according to homography matrix, the certain deviation position of Quick Response Code position or Quick Response Code in real scene, plays up and virtual information corresponding with Quick Response Code described in output display.
Preferably, described image characteristics extraction module, the image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out feature detection, is one of two kinds of modes below:
Mode one: the image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out full figure feature detection, obtains the feature interpretation of the two respectively;
Mode two: feature detection is carried out in the stabilized zone of the image in 2 D code in the scene image of only catching Quick Response Code front elevation and photographing module, obtains the feature interpretation of the two respectively.
Preferably, the Quick Response Code in described real scene image is conventional Quick Response Code or customization of two-dimension code; Resource information in customization of two-dimension code comprises resource identifier, resource type, resource load interface dimensions, one or more in rendering position drift rate.
Preferably, when described source obtaining module resolve that described resource information obtains be content of text time, content of text is played up as texture; And be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load according to virtual information type predetermined manner, described virtual information type comprises: one or more in video, image, text, 3D model.
Accordingly, the invention allows for a kind of mobile terminal, described mobile terminal comprises the above-mentioned augmented reality system based on Quick Response Code.
Compared with prior art, the present invention has following beneficial effect:
1, invention is directly according to the Quick Response Code in scene image, regenerate the Quick Response Code front elevation consistent with Quick Response Code in scene image, tracking and matching is carried out with the image in 2 D code in the Quick Response Code front elevation generated and scene image, calculate the attitude of video camera, obtain homography matrix, do not need to store Quick Response Code sample image in database, all applicable to arbitrary Quick Response Code, break in traditional augmented reality application, the limitation that corresponding sample image just can carry out tracking and matching must have been prestored in a database.
2, the present invention is not owing to needing sample image, therefore avoid from the inquiry of remote server with mate step, the system response delay because network transmission problems causes can be reduced, save the network traffic of user.
3, the degree of depth of the present invention excavates Quick Response Code as the application potential of information entry, makes the relevant information of Quick Response Code, resource present to user with more lively form.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings:
Fig. 1 is the augmented reality method flow schematic diagram of the embodiment of the present invention based on Quick Response Code;
Fig. 2 is the process and the effect schematic diagram that carry out virtual information superposition according to Fig. 1 method;
Fig. 3 is the image in 2 D code schematic diagram of several conventional code system;
Fig. 4 is can the Quick Response Code application schematic diagram that blocked by small icon of occlusion area.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
Those skilled in the art should know: in augmented reality process, are superimposed upon accurately on target object by virtual information, need to calculate the attitude of video camera, determine that the coordinate from target object is tied to the homography matrix of image coordinate system.
Present inventor finds, existing augmented reality is all generally carry out registration by real scene image and sample image, obtains homography matrix.This mode requires the image to carrying out augmented reality, its sample image for registration must be stored at mobile terminal or server end, if mobile terminal or augmented reality server end do not store the sample image of certain real scene or the characteristic point information of this sample image, then cannot realize the fusion of virtual information and this real scene, the popularization of augmented reality limit by sample image.In addition sample image be stored in mobile terminal can occupied terminal storage space, and can not the storage of satisfying magnanimity marker; And if sample image is stored in remote server, the retrieval of sample image with download and delay system can respond and waste the network traffic of user.
Present inventor finds, Quick Response Code has ease for use and popularization, and Quick Response Code is as information entry, can be generated by any information, it can carry the URI of any one section of text or any cost, the most important is: because Quick Response Code decoding can obtain code system, the information such as version and error-correction level of this Quick Response Code, can generate and the identical Quick Response Code front elevation of pattern in 2 D code before according to identical code system and version recompile again after Quick Response Code decoding, or generate and the identical Quick Response Code front elevation of Quick Response Code stabilized zone pattern before.Therefore propose using Quick Response Code as the identification in augmented reality system and positioning mark, to the Quick Response Code decoding re-encoding in real scene image, generate the Quick Response Code front elevation identical with the image in 2 D code in real scene image, it can be used as sample image and the Quick Response Code in scene to carry out feature point tracking to mate, find the unique point matched in the two, carry out the calculating of homography matrix.
Described Quick Response Code stabilized zone be relative to Quick Response Code can for occlusion area, because Quick Response Code has certain error correcting capability, it can be correctly decoded when certain area is blocked (as Fig. 4 can placed the Quick Response Code of a small icon by occlusion area at Quick Response Code, this Quick Response Code can be correctly decoded), therefore we claim to be blocked and do not affect region that Quick Response Code is correctly decoded for can occlusion area, except can the region except occlusion area be stabilized zone.
Introduce the present invention in detail below in conjunction with accompanying drawing and introduce scheme.
See Fig. 1, Fig. 2, for the embodiment of the present invention is based on the augmented reality method flow schematic diagram of Quick Response Code, comprise following S101 ~ S105 step:
S101: photographing module catches the real scene image containing Quick Response Code, described Quick Response Code can be conventional Quick Response Code, also can be customization of two-dimension code, conventional Quick Response Code refers to the network Quick Response Code comprising a text field or URI link in resource information, customization of two-dimension code refers to that its resource information comprises resource identifier, resource type, resource load in some other configuration information such as interface dimensions, rendering position drift rate one or more;
S102: the Quick Response Code in scanning scene image, and Quick Response Code is decoded, obtain the coded message of Quick Response Code, described coded message comprises: code system, version and resource information etc., and described resource information refers to the relevant information that scanning Quick Response Code obtains;
S103: re-encoding is carried out to the coded message of the Quick Response Code obtained, generates the Quick Response Code front elevation identical with the Quick Response Code code system in scene image and version; Resolve described resource information, to obtain virtual information corresponding to Quick Response Code, the resource information that Quick Response Code is corresponding may be text message simultaneously, also may be resource URI, when for resource URI, according to UIR address access remote server, obtain the virtual information content that URI is corresponding;
In this step, re-encoding is carried out to the coded message of the Quick Response Code obtained, generate the Quick Response Code front elevation identical with the Quick Response Code code system in scene image and version, this Quick Response Code front elevation may for the identical Quick Response Code of the pattern in 2 D code in scene image front elevation (scene Quick Response Code can occlusion area without any blocking time), also may for in scene image the identical Quick Response Code front elevation of Quick Response Code stabilized zone pattern (scene Quick Response Code can occlusion area partly or entirely blocked time, because Quick Response Code has error correcting capability, can be correctly decoded when having small icon blocks certain area, but re-encoding is carried out to decoded information, the Quick Response Code front elevation comprising data consistent with former Quick Response Code can be generated, but can not exist in the front elevation generated and block small icon in former Quick Response Code).
S104: the image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out feature detection, obtains the feature interpretation of the two respectively; And carry out image registration (i.e. Feature Points Matching finds the unique point matched in the two) according to the feature interpretation of the two, calculate the attitude of video camera, obtain homography matrix;
In this step, image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out feature detection, can be image in 2 D code in the scene image that Quick Response Code front elevation and photographing module are caught carry out full figure feature detection (this kind of mode scene Quick Response Code can occlusion area better without effect during any blocking, its to scene Quick Response Code can shield portions may there is certain matching error by region or when all blocking), also can be that the stabilized zone of the image in 2 D code in the scene image of only catching Quick Response Code front elevation and photographing module carries out feature detection (this kind of mode can whether to be blocked no requirement (NR) by occlusion area to scene Quick Response Code, all can matched well image).
Wherein, the method for feature detection can be: FAST, Harris, Shi-Thomas etc., and feature interpretation can use SIFT, SURF, ORB, BRIEF, FREAK etc., and these technology all belong to prior art, are not repeated herein.Quick Response Code in real scene is planar object, that define a world coordinate system, the Quick Response Code front elevation generated through decoding re-encoding belongs to image coordinate system, image in 2 D code in the scene image of catching for Quick Response Code front elevation and photographing module matches 4 groups of character pair points, the coordinate that the coordinate of these four unique points on world coordinate system and image coordinate are fastened can be set up following corresponding relation:
Wherein, the image coordinate of (u, v) representation feature point, (x
m, y
m, z
m) coordinate of representation feature point on world coordinate system, be known parameters, H represents homography matrix to be calculated, and the unique point matched is more, and result of calculation is more accurate.
S105: according to homography matrix, the certain deviation position of Quick Response Code position or Quick Response Code in real scene, plays up and virtual information corresponding with Quick Response Code described in output display.
In this step, if Quick Response Code is conventional Quick Response Code, be directly added to virtual information Quick Response Code position; And if Quick Response Code is customization of two-dimension code, sets resource in its resource information and load interface dimensions, rendering position drift rate etc., be then presented at folded for virtual information apart from setting deviation post place, target location according to the size set and drift rate.In addition, in this step, when resolve described resource information obtain be content of text time, content of text is played up as texture; Resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load according to virtual information type predetermined manner, described virtual information type comprises: one or more in video, image, text, 3D model, such as: when virtual information is text message, content of text is directly played up as texture; When virtual information is 3D model, need first to resolve 3D model; When virtual information is video information, needs first to decode to video, using each for video two field picture as texture, be mapped on described 3D model by sequence frame by frame, carry out graph rendering.
Embodiment of the present invention method, can with Quick Response Codes such as PDF417, QRCode, DataMatrix, GridMatrix and Aztec for application, and the Quick Response Code schematic diagram of several conventional code system can see Fig. 3.
In the embodiment of the present invention, Quick Response Code can be conventional Quick Response Code, also can be customization of two-dimension code, so-called conventional Quick Response Code refers to that Quick Response Code derives from internet, and the resource information comprised in it is generally a character string, such as, be the URI of one section of text or Quick Response Code related resource; So-called customization of two-dimension code refers to, according to the resource identification that certain consolidation form is formed, the resource information namely in Quick Response Code comprises resource identifier, resource type, resource load in some other configuration information such as interface dimensions, rendering position drift rate one or more.Such as, ID:xxx_URI:xxx_TYPE:xxx_WIDTH:xxx_HEIGHT:xxx_OFFSET:xxx, (this address may at remote server to determine resource address by ID and URI, also may be local in client), can be pre-set the load mode (load mode of picture, text, audio frequency, video and 3D model is different) of related resource by resource type, other configuration informations include resource and load the skew etc. relative to Quick Response Code of the size at interface and rendering position.
The invention allows for a kind of augmented reality system based on Quick Response Code corresponding with said method, comprise: photographing module, Quick Response Code decoder module, Quick Response Code coding module, source obtaining module, image characteristics extraction module, image trace registration module and play up display module, wherein:
Described photographing module, for catching the real scene image containing Quick Response Code, Quick Response Code in described real scene image is conventional Quick Response Code or customization of two-dimension code, and the resource information in customization of two-dimension code comprises resource identifier, resource type, resource load interface dimensions, one or more in rendering position drift rate;
Described Quick Response Code decoder module, for scanning the Quick Response Code in scene image, and decodes to Quick Response Code, and obtain the coded message of Quick Response Code, described coded message comprises: code system, version and resource information;
Described Quick Response Code coding module, for the coded message parsed according to described Quick Response Code decoder module, encode by code system identical with it and version, generate and the identical Quick Response Code front elevation of pattern in 2 D code in scene image, or with the identical Quick Response Code front elevation of Quick Response Code stabilized zone pattern in scene image, described source obtaining module, for resolving the resource information in described coded message, to obtain virtual information corresponding to Quick Response Code;
Described image characteristics extraction module, carries out feature detection for the image in 2 D code in the scene image of catching Quick Response Code front elevation and photographing module, obtains the feature interpretation of the two respectively;
Described image trace registration module, for carrying out image registration according to the image in 2 D code feature interpretation in Quick Response Code front elevation and scene image, calculating the attitude of video camera, obtaining homography matrix;
Describedly play up display module, for according to homography matrix, the certain deviation position of Quick Response Code position or Quick Response Code in real scene, play up and virtual information corresponding with Quick Response Code described in output display, when described source obtaining module resolve that described resource information obtains be content of text time, content of text is played up as texture; And be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load according to virtual information type predetermined manner, described virtual information type comprises: one or more in video, image, text, 3D model.
The invention allows for a kind of mobile terminal, described mobile terminal comprises the above-mentioned augmented reality system based on Quick Response Code.
The present invention is using Quick Response Code as the identification in augmented reality system and positioning mark, Quick Response Code front elevation is directly generated by the decoding of Quick Response Code and re-encoding, for the calculating of homography matrix, both broken in traditional augmented reality application, the limitation that corresponding sample image just can carry out tracking and matching must be prestored in a database, turn avoid use legacy identification thing time to remote server template query with mate step, the system response delay because network transmission problems causes can be reduced, save the network traffic of user.
All features disclosed in this instructions, or the step in disclosed all methods or process, except mutually exclusive feature and/or step, all can combine by any way.
Arbitrary feature disclosed in this instructions (comprising any accessory claim, summary and accompanying drawing), unless specifically stated otherwise, all can be replaced by other equivalences or the alternative features with similar object.That is, unless specifically stated otherwise, each feature is an example in a series of equivalence or similar characteristics.
The present invention is not limited to aforesaid embodiment.The present invention expands to any new feature of disclosing in this manual or any combination newly, and the step of the arbitrary new method disclosed or process or any combination newly.
Claims (10)
1., based on an augmented reality method for Quick Response Code, it is characterized in that, comprising:
Photographing module catches the real scene image containing Quick Response Code;
Quick Response Code in scanning scene image, and Quick Response Code is decoded, obtain the coded message of Quick Response Code, described coded message comprises: code system, version and resource information;
Re-encoding is carried out to the coded message of the Quick Response Code obtained, generates the Quick Response Code front elevation identical with the Quick Response Code code system in scene image and version; Resolve described resource information, to obtain virtual information corresponding to Quick Response Code simultaneously;
Image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out feature detection, obtains the feature interpretation of the two respectively; And carry out image registration according to the feature interpretation of the two, calculate the attitude of video camera, obtain homography matrix;
According to homography matrix, the certain deviation position of Quick Response Code position or Quick Response Code in real scene image, plays up the virtual information that also output display is corresponding with Quick Response Code.
2. the method for claim 1, is characterized in that:
When resolve described resource information obtain be content of text time, content of text is played up as texture;
Be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load according to virtual information type predetermined manner.
3. method as claimed in claim 2, is characterized in that: described virtual information type comprises: one or more in video, image, text, 3D model.
4. method as claimed any one in claims 1 to 3, is characterized in that:
Quick Response Code in described real scene image is conventional Quick Response Code or customization of two-dimension code;
Resource information in customization of two-dimension code comprises resource identifier, resource type, resource load interface dimensions, one or more in rendering position drift rate.
5. method as claimed any one in claims 1 to 3, it is characterized in that, image in 2 D code in described scene image of catching Quick Response Code front elevation and photographing module carries out feature detection, obtain the feature interpretation of the two respectively, be specially: the image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out full figure feature detection, obtains the feature interpretation of the two respectively.
6., based on an augmented reality system for Quick Response Code, it is characterized in that, comprising:
Photographing module, for catching the real scene image containing Quick Response Code;
Quick Response Code decoder module, for scanning the Quick Response Code in scene image, and decodes to Quick Response Code, and obtain the coded message of Quick Response Code, described coded message comprises: code system, version and resource information;
Quick Response Code coding module, carries out re-encoding to the coded message that described Quick Response Code decoder module parses, and generates the Quick Response Code front elevation identical with the Quick Response Code code system in scene image and version;
Source obtaining module, for resolving the resource information in described coded message, to obtain virtual information corresponding to Quick Response Code;
Image characteristics extraction module, carries out feature detection for the image in 2 D code in the scene image of catching Quick Response Code front elevation and photographing module, obtains the feature interpretation of the two respectively;
Image trace registration module, for carrying out image registration according to the image in 2 D code feature interpretation in Quick Response Code front elevation and scene image, calculating the attitude of video camera, obtaining homography matrix;
Play up display module, for according to homography matrix, the certain deviation position of Quick Response Code position or Quick Response Code in real scene image, plays up and the output display virtual information corresponding with Quick Response Code.
7. system as claimed in claim 6, it is characterized in that, described image characteristics extraction module, image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out feature detection, be specially: the image in 2 D code in the scene image of catch Quick Response Code front elevation and photographing module carries out full figure feature detection, obtains the feature interpretation of the two respectively.
8. system as claimed in claims 6 or 7, is characterized in that:
Quick Response Code in described real scene image is conventional Quick Response Code or customization of two-dimension code;
Resource information in customization of two-dimension code comprises resource identifier, resource type, resource load interface dimensions, one or more in rendering position drift rate.
9. system as claimed in claims 6 or 7, is characterized in that,
When described source obtaining module resolve that described resource information obtains be content of text time, content of text is played up as texture; And be resource URI when what resolve that described resource information obtains, then access this URI and obtain virtual information, and load according to virtual information type predetermined manner, described virtual information type comprises: one or more in video, image, text, 3D model.
10. a mobile terminal, is characterized in that, described mobile terminal comprises the augmented reality system based on Quick Response Code according to any one of claim 6 to 9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210587153.1A CN103049729B (en) | 2012-12-30 | 2012-12-30 | Based on the augmented reality method of Quick Response Code, system and terminal |
| PCT/CN2013/081876 WO2014101435A1 (en) | 2012-12-30 | 2013-08-20 | Augmented reality method, system, and terminal based on two-dimensional code |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210587153.1A CN103049729B (en) | 2012-12-30 | 2012-12-30 | Based on the augmented reality method of Quick Response Code, system and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049729A CN103049729A (en) | 2013-04-17 |
| CN103049729B true CN103049729B (en) | 2015-12-23 |
Family
ID=48062362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210587153.1A Active CN103049729B (en) | 2012-12-30 | 2012-12-30 | Based on the augmented reality method of Quick Response Code, system and terminal |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN103049729B (en) |
| WO (1) | WO2014101435A1 (en) |
Families Citing this family (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049729B (en) * | 2012-12-30 | 2015-12-23 | 成都理想境界科技有限公司 | Based on the augmented reality method of Quick Response Code, system and terminal |
| CN103996063A (en) * | 2014-06-12 | 2014-08-20 | 北京金山网络科技有限公司 | Method and device for processing data |
| CN104504402A (en) * | 2015-01-15 | 2015-04-08 | 刘畅 | Data processing method and system based on image search |
| CN104504155B (en) * | 2015-01-15 | 2018-06-08 | 刘畅 | A kind of data capture method and system based on picture search |
| CN105989390A (en) * | 2015-02-11 | 2016-10-05 | 北京鼎九信息工程研究院有限公司 | Generation method of two-dimensional code and apparatus thereof |
| WO2016136435A1 (en) * | 2015-02-24 | 2016-09-01 | 株式会社日立ハイテクノロジーズ | Automatic analysis apparatus |
| CN104850582B (en) * | 2015-04-13 | 2017-11-07 | 西安教育文化数码有限责任公司 | A kind of index type augmented reality system |
| CN104834680B (en) * | 2015-04-13 | 2017-11-07 | 西安教育文化数码有限责任公司 | A kind of index type augmented reality method |
| CN105446626A (en) * | 2015-12-04 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | Augmented reality technology based commodity information acquisition method and system and mobile terminal |
| CN105787534B (en) * | 2016-02-29 | 2018-07-10 | 上海导伦达信息科技有限公司 | Quick Response Code and the identification of AR digital contents are merged with study and with augmented reality implementation method |
| CN106251404B (en) * | 2016-07-19 | 2019-02-01 | 央数文化(上海)股份有限公司 | Orientation tracking, the method and relevant apparatus, equipment for realizing augmented reality |
| CN111291584B (en) * | 2016-07-22 | 2023-05-02 | 创新先进技术有限公司 | Method and system for identifying two-dimensional code position |
| CN106372144A (en) * | 2016-08-26 | 2017-02-01 | 江西科骏实业有限公司 | Two-dimensional code processing apparatus and method |
| CN106408667B (en) * | 2016-08-30 | 2019-03-05 | 西安小光子网络科技有限公司 | Customization practical method based on optical label |
| CN108665035B (en) * | 2017-03-31 | 2020-12-22 | 清华大学 | Code mark generation method and device |
| CN107464288A (en) * | 2017-07-24 | 2017-12-12 | 腾讯科技（深圳）有限公司 | Model display method and device |
| US10825563B2 (en) * | 2018-05-14 | 2020-11-03 | Novarad Corporation | Aligning image data of a patient with actual views of the patient using an optical code affixed to the patient |
| CN108986163A (en) * | 2018-06-29 | 2018-12-11 | 南京睿悦信息技术有限公司 | Augmented reality location algorithm based on more mark identifications |
| CN109840951A (en) * | 2018-12-28 | 2019-06-04 | 北京信息科技大学 | The method and device of augmented reality is carried out for plane map |
| CN111859199A (en) * | 2019-04-30 | 2020-10-30 | 苹果公司 | Locate content in the environment |
| TWI785332B (en) * | 2020-05-14 | 2022-12-01 | 光時代科技有限公司 | Three-dimensional reconstruction system based on optical label |
| CN112560851A (en) * | 2020-12-11 | 2021-03-26 | 上海影创信息科技有限公司 | Virtual object creating and positioning method and system |
| CN115221988B (en) * | 2021-04-20 | 2025-07-18 | 腾讯科技（深圳）有限公司 | Two-dimensional code generation method, device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944187A (en) * | 2010-09-07 | 2011-01-12 | 龚湘明 | Two-dimensional micro code and processing method and device thereof |
| CN102323880A (en) * | 2011-06-30 | 2012-01-18 | 中兴通讯股份有限公司 | Mobile phone application interface development method and terminal based on browser parsing mode |
| CN102800065A (en) * | 2012-07-13 | 2012-11-28 | 苏州梦想人软件科技有限公司 | Augmented reality equipment and method based on two-dimensional code identification and tracking |
| CN102821323A (en) * | 2012-08-01 | 2012-12-12 | 成都理想境界科技有限公司 | Video playing method, video playing system and mobile terminal based on augmented reality technique |
| US20120327117A1 (en) * | 2011-06-23 | 2012-12-27 | Limitless Computing, Inc. | Digitally encoded marker-based augmented reality (ar) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8350871B2 (en) * | 2009-02-04 | 2013-01-08 | Motorola Mobility Llc | Method and apparatus for creating virtual graffiti in a mobile virtual and augmented reality system |
| CN103049729B (en) * | 2012-12-30 | 2015-12-23 | 成都理想境界科技有限公司 | Based on the augmented reality method of Quick Response Code, system and terminal |
-
2012
- 2012-12-30 CN CN201210587153.1A patent/CN103049729B/en active Active
-
2013
- 2013-08-20 WO PCT/CN2013/081876 patent/WO2014101435A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944187A (en) * | 2010-09-07 | 2011-01-12 | 龚湘明 | Two-dimensional micro code and processing method and device thereof |
| US20120327117A1 (en) * | 2011-06-23 | 2012-12-27 | Limitless Computing, Inc. | Digitally encoded marker-based augmented reality (ar) |
| CN102323880A (en) * | 2011-06-30 | 2012-01-18 | 中兴通讯股份有限公司 | Mobile phone application interface development method and terminal based on browser parsing mode |
| CN102800065A (en) * | 2012-07-13 | 2012-11-28 | 苏州梦想人软件科技有限公司 | Augmented reality equipment and method based on two-dimensional code identification and tracking |
| CN102821323A (en) * | 2012-08-01 | 2012-12-12 | 成都理想境界科技有限公司 | Video playing method, video playing system and mobile terminal based on augmented reality technique |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2014101435A1 (en) | 2014-07-03 |
| CN103049729A (en) | 2013-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103049729B (en) | Based on the augmented reality method of Quick Response Code, system and terminal | |
| CN103049728B (en) | Based on the augmented reality method of Quick Response Code, system and terminal | |
| CN113127771A (en) | Application point burying method and device, computing equipment and system | |
| CN108491897B (en) | Information identification method, server, client and system | |
| CN104281827B (en) | Identify the method and device of Quick Response Code | |
| CN105447546A (en) | Two-dimensional code capable of being embedded with large-ratio graph as well as coding and decoding method and device therefor | |
| CN105577684A (en) | Anti-crawling methods, server, client and system | |
| CN102800065A (en) | Augmented reality equipment and method based on two-dimensional code identification and tracking | |
| CN105718979B (en) | A method of generating two-dimension code image | |
| CN103279503A (en) | Method and system for acquiring two-dimension code information from webpage | |
| CN102306174A (en) | Method and equipment for interacting with user based on web page elements | |
| CN103150392A (en) | Method for comparing prices of similar commodities | |
| CN110502711B (en) | Page display method and device, electronic equipment and readable storage medium | |
| CN103344241A (en) | Method and system for obtaining navigation message from public electronic map | |
| CN105808587A (en) | Method and system for embedding information into webpage and gateway device | |
| CN103686454A (en) | Information acquisition method and apparatus | |
| CN103929669A (en) | Interactive video generator, player, generating method and playing method | |
| WO2019174429A1 (en) | Video map engine system | |
| Ruban et al. | The model and the method for forming a mosaic sustainable marker of augmented reality | |
| CN106372144A (en) | Two-dimensional code processing apparatus and method | |
| WO2024113874A1 (en) | Encoding method and decoding method for annular 2-dimensional bar code | |
| CN105664490A (en) | Method and system for realizing game interaction | |
| CN102254204B (en) | Coding method for graphemic code | |
| CN108985420A (en) | A kind of decoding method and decoding system of information code | |
| US8086051B2 (en) | Method of decoding fetched scene on encoded dotmap and electronic device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |