[go: up one dir, main page]

CN112702643B - Barrage information display method and device and mobile terminal - Google Patents

Barrage information display method and device and mobile terminal Download PDF

Info

Publication number
CN112702643B
CN112702643B CN201911007282.7A CN201911007282A CN112702643B CN 112702643 B CN112702643 B CN 112702643B CN 201911007282 A CN201911007282 A CN 201911007282A CN 112702643 B CN112702643 B CN 112702643B
Authority
CN
China
Prior art keywords
information
bullet screen
position information
displayed
coordinate system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911007282.7A
Other languages
Chinese (zh)
Other versions
CN112702643A (en
Inventor
段云鹏
涂中英
孙英男
龚亚辉
鲁炳坦
王炜煜
施波迪
罗勇
方海波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Bilibili Technology Co Ltd
Original Assignee
Shanghai Bilibili Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Bilibili Technology Co Ltd filed Critical Shanghai Bilibili Technology Co Ltd
Priority to CN201911007282.7A priority Critical patent/CN112702643B/en
Priority to US17/073,957 priority patent/US11651560B2/en
Publication of CN112702643A publication Critical patent/CN112702643A/en
Application granted granted Critical
Publication of CN112702643B publication Critical patent/CN112702643B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/52—Network services specially adapted for the location of the user terminal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/75—Indicating network or usage conditions on the user display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508—Management of client data or end-user data
    • H04N21/4524—Management of client data or end-user data involving the geographical location of the client
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4884—Data services, e.g. news ticker for displaying subtitles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/004—Annotating, labelling

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Computation (AREA)
  • Marketing (AREA)
  • Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses a bullet screen information display method and device, a mobile terminal and a readable storage medium, and belongs to the technical field of video display. The bullet screen information display method of the invention comprises the following steps: acquiring camera position information of the mobile terminal in a camera coordinate system; converting the camera position information into spatial position information in a world coordinate system; acquiring the bullet screen information to be displayed; and displaying the bullet screen information to be displayed according to the space position information. The bullet screen information display method and device can display bullet screen information in the real space, and user experience is improved.

Description

Barrage information display method and device and mobile terminal
Technical Field
The present invention relates to the field of video display technologies, and in particular, to a barrage information display method and apparatus, a mobile terminal, and a computer readable storage medium.
Background
The bullet screen information may appear in a scrolling, hovering, or other special effects manner on the video's playing interface, the content of which is typically a brief comment sent by the user viewing the video. In the prior art, display of barrage information is realized mainly by superposing barrage information in a video playing interface. When a plurality of pieces of barrage information are displayed at the same time, the barrage information can be displayed to a user in a directional scrolling or line-by-line display mode.
However, in the existing bullet screen information display method, bullet screen information is only displayed in a playing interface, and the interaction mode of the user is single, so that the participation of the user is not strong, and the user experience is poor.
Disclosure of Invention
Aiming at the problems that in the existing barrage information display method, barrage information is only displayed in a playing interface and the interaction mode of a user is single, the barrage information display method, device, mobile terminal and computer readable storage medium are provided.
The invention provides a bullet screen information display method, which comprises the following steps:
acquiring camera position information of the mobile terminal in a camera coordinate system;
converting the camera position information into spatial position information in a world coordinate system;
acquiring the bullet screen information to be displayed;
and displaying the bullet screen information to be displayed according to the space position information.
Optionally, the step of converting the camera position information into spatial position information in a world coordinate system comprises:
calculating bullet screen generating position information of the bullet screen information to be displayed in the camera coordinate system according to the camera position information;
and determining the spatial position information according to the bullet screen generating position information.
Optionally, the step of displaying the bullet screen information to be displayed according to the spatial position information includes:
acquiring a motion trail and a motion speed of the bullet screen information to be displayed;
and playing the bullet screen information to be displayed according to the motion trail and the motion speed.
Optionally, the step of obtaining the bullet screen information to be displayed includes:
acquiring an object image of an object to be identified;
performing image recognition on the object image through an object recognition technology to recognize the object type;
and acquiring comment information associated with the object category from a server according to the object category, wherein the comment information is bullet screen information to be information.
Optionally, the step of obtaining the bullet screen information to be displayed includes:
acquiring geographic position information of the mobile terminal;
and acquiring comment information related to the geographical position information from a server according to the geographical position information, wherein the comment information is bullet screen information to be information.
Optionally, the comment information includes venue comment information and/or video URL information.
Optionally, the bullet screen information to be displayed includes a hyperlink bullet screen, and the method further includes:
And when the clicking operation of the user on the played hyperlink barrage is received, displaying the page corresponding to the hyperlink barrage.
The invention also provides a barrage information display device, which comprises:
the position acquisition module is used for acquiring camera position information of the mobile terminal in a camera coordinate system;
a determination module for converting the camera position information into spatial position information in a world coordinate system;
the bullet screen acquisition module is used for acquiring the bullet screen information to be displayed;
and the playing module is used for displaying the bullet screen information to be displayed according to the space position information.
The invention also provides a mobile terminal which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps of the method when executing the computer program.
The invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method.
The beneficial effects of the technical scheme are that:
in the technical scheme, camera position information of the mobile terminal in a camera coordinate system is obtained; converting the camera position information into spatial position information in a world coordinate system; acquiring the bullet screen information to be displayed; and displaying the bullet screen information to be displayed according to the space position information. According to the embodiment of the invention, the bullet screen information is displayed through an augmented reality technology (Augmented Reality, AR for short), so that the bullet screen information can be displayed in a real space, a user can truly experience the feeling that the bullet screen is in a floating state, the user experience is improved, and in addition, the bullet screen information is displayed through the AR technology, so that more various interactions can be performed.
Drawings
FIG. 1 is a block diagram of one embodiment of a system block diagram of a bullet screen information display according to the present invention;
FIG. 2 is a flowchart of an embodiment of a barrage information display method according to the present invention;
FIG. 3 is a detailed flow chart of the step of converting the camera position information into spatial position information in world coordinate system of FIG. 2 in accordance with the present invention;
FIG. 4 is a detailed flowchart of the step of obtaining the bullet screen information to be displayed in FIG. 2 according to the present invention;
FIG. 5 is a detailed flowchart of the step of obtaining the bullet screen information to be displayed in FIG. 2 according to the present invention;
FIG. 6 is a detailed flowchart of the step of displaying the bullet screen information to be displayed according to the spatial location information in FIG. 2 according to the present invention;
FIG. 7 is a flowchart of another embodiment of a barrage information display method according to the present invention;
FIG. 8 is a block diagram of an embodiment of a barrage information display apparatus according to the present invention;
fig. 9 is a schematic diagram of a hardware structure of a mobile terminal for executing a barrage information display method according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of an interface for displaying bullet screen information according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of another interface for displaying bullet screen information according to an embodiment of the present invention.
Detailed Description
Advantages of the invention are further illustrated in the following description, taken in conjunction with the accompanying drawings and detailed description.
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
In the description of the present invention, it should be understood that the numerical references before the steps do not identify the order in which the steps are performed, but are merely used to facilitate description of the present invention and to distinguish between each step, and thus should not be construed as limiting the present invention.
Referring to fig. 1, fig. 1 is a schematic view illustrating an application environment of a barrage information display method according to an embodiment of the present application. In an exemplary embodiment, the mobile terminal 2 may obtain data from the server 4 via the network 6. The mobile terminal 2 may be an electronic device having a data transmission function, such as a mobile phone, a tablet personal computer (tablet personal computer), a laptop computer (laptop computer), a notebook computer, or a desktop computer. The network 6 may be the internet.
Example 1
Fig. 2 schematically shows a flowchart of a barrage information display method according to a first embodiment of the present application. The method is applied to the mobile terminal 2, and it will be appreciated that the flow chart in the embodiment of the method is not intended to limit the order in which the steps are performed. An exemplary description will be made below with the mobile terminal 2 as an execution subject.
As shown in fig. 2, the bullet screen information display method includes steps S20 to S23, in which:
Step S20, obtaining camera position information of the mobile terminal in a camera coordinate system;
specifically, the camera position information is coordinates of the mobile terminal in a camera coordinate system. In this embodiment, when the mobile terminal displays the barrage information using augmented reality (Augmented Reality, abbreviated as AR), the mobile terminal starts an AR barrage program and invokes its own optical system (such as a camera set, including a lens, a shutter, an image sensor, etc.), together with an inertial measurement unit (Inertial Measurement Unit, IMU) and an AR software tool development kit, to collect and calculate the position information (x, y, z) and the direction (pitch, yaw, roll) of the device, wherein the inertial measurement unit may be an accelerometer or a gyroscope. The AR technology refers to a technology that combines and interacts with a real world scene through the accurate calculation of the position and angle of a camera image and the addition of an image analysis technology.
The positional information (x, y, z) is coordinates in the camera coordinate system.
The camera coordinate system is a three-dimensional rectangular coordinate system established by taking the focusing center of the camera as an origin and taking the optical axis as a Z axis. The origin of the camera coordinate system (viewing coordinate system) is the optical center of the camera, the X-axis and the Y-axis are parallel to the X, Y-axis of the image, and the z-axis is the optical axis of the camera, which is perpendicular to the graphics plane. The intersection point of the optical axis and the image plane is the origin of the image coordinate system, and the image coordinate system is a two-dimensional rectangular coordinate system.
Step S21, converting the camera position information into spatial position information in a world coordinate system.
Specifically, the spatial position information is coordinates of the bullet screen information to be information displayed in a world coordinate system. The coordinates in the camera coordinate system can be converted into coordinates in the world coordinate system by a conversion formula of the camera coordinate system and the world coordinate system.
In an exemplary embodiment, referring to fig. 3, the step of converting the camera position information into spatial position information in a world coordinate system includes S30 to S31, wherein:
and step S30, calculating bullet screen generating position information of the bullet screen information to be displayed in the camera coordinate system according to the camera position information.
Specifically, the barrage generation position information is coordinates of the barrage information to be information displayed in a camera coordinate system.
In this embodiment, in order to display the bullet screen information more randomly and dispersedly, the initialization position optimization can be performed on the bullet screen display position, that is, the bullet screen generation position can be preset. In one embodiment, the initialization position optimization may be specifically performed as follows: first, camera position information of a mobile terminal is acquired, wherein in a camera coordinate system, the camera position information is represented by a homogeneous coordinate system Secondly, after obtaining the camera position information of the mobile terminal, carrying out translation and/or rotation transformation on the camera position p1, finally calculating the position information obtained after translation and/or rotation transformation, and generating the position information as random and scattered barrages, wherein in a camera coordinate system, the position information is transformedThe camera position information is represented by the homogeneous coordinate system>In this embodiment, the specific translation distance and rotation angle may be preset, or may be set by the user, which is not limited in this embodiment.
And S31, determining the spatial position information according to the barrage generation position information.
Specifically, a transformation relation of the bullet screen information to be displayed is pre-stored in the mobile terminal, and the transformation relation is used for determining the generation position of the bullet screen information to be displayed, and the transformation relation can be: translates a units to the x-axis and rotates b degrees in the pitch direction. In this embodiment, the transformation relationship is correspondingly changed according to the movement and rotation of the camera, and the changed transformation relationship can be automatically maintained by the operating system and the SDK of the mobile terminal.
After the bullet screen generating position information is acquired, the spatial position information corresponding to the bullet screen generating position information can be calculated according to the transformation relation. That is, the conversion of the camera coordinate system and the world coordinate system is calculated from this conversion relation.
And S22, acquiring the bullet screen information to be displayed.
Specifically, a bullet screen (barrage) refers to a comment subtitle that pops up when a video is viewed over a network. Like the clips highlighted between the rows in the novice, the points of review that fly through between the screens in the video are called barrages, which are intended to provide dense shots with a large or small number of cannons. The bullet screen, as the name implies, is a screen formed by more bullets, and a large number of spitting comments appear to be the bullet screen in the flying shooting game when the comments are floated from the screen.
And the bullet screen information to be displayed is comment information of the user. In an embodiment, the comment information may specifically be a comment of a user on a certain article, or may be a comment of a user on a certain place, or a comment on a certain event, etc. In this embodiment, the comment information may exist in text form, and in another embodiment, the comment information may also exist in any form that can be converted into a 3D model, such as a picture, a 3D model, or a video.
And the bullet screen information to be displayed is stored in the server, and can be obtained from the server through a bullet screen obtaining instruction when the bullet screen information needs to be displayed.
In an exemplary embodiment, referring to fig. 4, the step of obtaining the bullet screen information to be displayed includes S40 to S42, where:
step S40, an object image of the object to be identified is acquired.
Specifically, when a user needs to acquire comment information of the user for a certain object to be identified, the user can take a picture of the object to be identified through a camera of the mobile terminal, so that the mobile terminal acquires an object image of the object to be identified. In another embodiment, the object image of the object to be identified may also be obtained directly from an album in the mobile terminal.
In step S41, image recognition is performed on the object image by using an object recognition technology, so as to recognize the object category.
Specifically, the object image can be subjected to image recognition by using an object recognition model obtained through training of a deep learning technology, so that the category of the object is recognized. In the training process of the object image recognition model, in order to improve the accuracy of model recognition, the used pictures in the training sample data set can be subjected to data enhancement processing by utilizing methods such as cutting, rotation and the like, so that the diversity of the sample data set is improved, and the object recognition model obtained through training of the training sample set can have better recognition rate.
And step S42, comment information associated with the object category is obtained from a server according to the object category, and the comment information is bullet screen information to be information.
Specifically, after the category of the object is identified, category information of the object is transmitted to a server, and the server searches comment information associated with the category of the object according to the acquired category information of the object.
For example, when a user sees a task, the user can take a picture of the task through the mobile terminal, the mobile terminal identifies the task picture, and after identifying the category of the task picture, the task category information is transmitted to the server, and the server searches comment information associated with the task according to the category information. The mobile terminal can use an offline image recognition technology to recognize the handy commodity so as to acquire a data set with higher quality and improve recognition accuracy.
It can be understood that each user can issue comment information on the sponsored commodity through playing the user interface of the barrage information to be displayed, after the user issues the comment information through the user interface, the mobile terminal transmits the comment information to the server, and the server stores the relation between the comment information of the user and the corresponding sponsor to the comment database.
After the bullet screen information is obtained, the bullet screen information may be displayed, and in one embodiment, the bullet screen information is displayed as shown in fig. 10.
In another exemplary embodiment, referring to fig. 5, the step of acquiring the bullet screen information to be displayed includes steps S50 to S51, wherein:
and S50, obtaining the geographical position information of the mobile terminal.
Specifically, the geographic position information of the mobile terminal can be obtained by calling a positioning module in the mobile terminal.
And step S51, comment information related to the geographical position information is acquired from a server according to the geographical position information, wherein the comment information is bullet screen information to be information.
Specifically, when the user publishes comment information, the user can acquire the geographical position where the user locates when publishing the comment information, and after the user publishes comment information through the mobile terminal, the mobile terminal uploads comment information containing the geographical position information to the server, and after receiving the comment information containing the geographical position information, the server stores the comment information containing the geographical position information. It can be understood that after the server stores the comment information containing the geographical location information, a background manager can audit the comment information containing the geographical location information published by the user, and delete the comment information when finding unsuitable comment information.
Since the comment information posted by the user contains the geographical position information, after the geographical position information of the mobile terminal is acquired, comment information associated with the geographical position information can be acquired from the server according to the geographical position information.
In this embodiment, the comment information may be site comment information, that is, a comment of a user for a certain site, where the site comment may be in a text form, or may be any form that may be converted into a 3D model, such as a picture, a 3D model, or a video. When the comment information exists in the form of video, the comment information also comprises video URL information of the video.
Illustratively, when a user eats at a restaurant, the user may post comment information for the restaurant, e.g., the restaurant tastes good, and the user may post a video of the restaurant environment for the restaurant environment. When a user publishes comment information through the mobile terminal, the mobile terminal can acquire geographical position information where the user publishes the comment information, and then uploads the comment information containing the geographical position information to a server. Thus, after the mobile terminal acquires the geographic position information, comment information associated with the geographic position information can be acquired from the server according to the acquired geographic position information.
It can be understood that each user can issue place comment information for a certain place by playing the user interface of the bullet screen information to be displayed, after the user issues the place comment information through the user interface, the mobile terminal transmits the place comment information to the server, and the server stores the relationship between the place comment information of the user and the corresponding geographic position information to the comment database. When comment information posted by a user is video, after uploading the video to a server, the server associates and stores URL information corresponding to the video and corresponding geographic position information to a comment database.
After the bullet screen information is obtained, the bullet screen information may be displayed, and in one embodiment, the bullet screen information is displayed as shown in fig. 11.
And step S23, displaying the bullet screen information to be displayed according to the space position information.
Specifically, after the spatial position information of the bullet screen information to be displayed is obtained, the mobile terminal may play each piece of bullet screen information to be displayed according to the display size, the font color, the font format and the display special effect of each piece of bullet screen information, where the display special effect includes a stay, a scroll or other special effect modes.
In an exemplary embodiment, referring to fig. 6, the step of displaying the bullet screen information to be displayed according to the spatial location information includes steps S60 to S61, wherein:
and step S60, acquiring the movement track and the movement speed of the bullet screen information to be displayed.
And step S61, playing the bullet screen information to be displayed according to the motion trail and the motion speed.
Specifically, the motion track and the motion speed of the bullet screen information may be preset, for example, the preset bullet screen motion track is from right to left relative to the current position of the camera, the motion vector is denoted as a, and after the preset motion vector a is acquired, the preset motion vector a is converted into a vector a' in the world coordinate system.
After the spatial position information of the bullet screen information to be displayed is obtained and the vector a 'in the world coordinate system corresponding to the movement track of the bullet screen is calculated, a bullet screen model is generated in the spatial position information, and then the bullet screen model moves at the movement speed S according to the vector a' in the world coordinate system, so that the bullet screen information to be displayed is played.
In this embodiment, based on different operating systems or SDKs, the barrage model may achieve different degrees of customization effects. Different fonts, colors, illumination shadows and other effects can be used for the barrage model, and besides the text model, a picture texture can also be used for covering the 3D model.
In one embodiment, after the bullet screen is moved to the predetermined location, bullet screen resources may be recovered.
In another embodiment, after displaying the bullet screen information, the user may perform an interactive operation on the bullet screen information, for example, the user may change the moving track of a bullet screen by clicking or sliding. For example, the user can switch the moving track of one barrage to another moving track through sliding operation, after receiving the switching operation of the user on the moving track of the barrage, the mobile terminal can redetermine the direction vector corresponding to the moving track of the barrage according to the sliding operation of the user, and after determining the direction vector corresponding to the new moving track, convert the direction vector into the direction vector corresponding to the world coordinate system, and finally play the barrage information according to the direction vector in the world coordinate system.
In the embodiment, camera position information of the mobile terminal in a camera coordinate system is obtained; converting the camera position information into spatial position information in a world coordinate system; acquiring the bullet screen information to be displayed; and displaying the bullet screen information to be displayed according to the space position information. According to the embodiment of the invention, the bullet screen information is displayed through an augmented reality technology (Augmented Reality, AR for short), so that the bullet screen information can be displayed in a real space, a user can truly experience the feeling that the bullet screen is in a floating state, the user experience is improved, and in addition, the bullet screen information is displayed through the AR technology, so that more various interactions can be performed.
Example two
Fig. 7 schematically shows a flowchart of a barrage information display method according to a second embodiment of the present application. The bullet screen information display method may further include steps S70 to S74, wherein:
step S70, obtaining camera position information of the mobile terminal in a camera coordinate system.
Step S71 converts the camera position information into spatial position information in a world coordinate system.
Step S72, obtaining the bullet screen information to be displayed.
And step S73, displaying the bullet screen information to be displayed according to the space position information.
Specifically, steps S70 to S73 in the present embodiment are substantially the same as steps S20 to S23 in the first embodiment, and are not described in detail in the present embodiment.
Step S74, when the clicking operation of the user on the played hyperlink barrage is received, displaying the page corresponding to the hyperlink barrage.
Specifically, the information of the bullet screen to be displayed may be a hyperlink bullet screen, when the played information of the bullet screen to be displayed includes the hyperlink bullet screen, the user may click on the hyperlink bullet screen played in the video interface, and after receiving the click operation of the user on the hyperlink bullet screen, the mobile terminal obtains the corresponding page content according to the hyperlink, and then displays the page content.
In this embodiment, by providing the hyperlink barrage, the user may click on the hyperlink barrage, so as to enter the page corresponding to the hyperlink, thereby increasing the interaction mode of the barrage and improving the user experience.
Example III
Fig. 8 schematically shows a block diagram of a barrage information display apparatus according to the third embodiment of the present application, which may be divided into one or more program modules, which are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. Program modules in the embodiments of the present application refer to a series of computer program instruction segments capable of performing particular functions, and are more suitable than programs themselves for describing the execution of a video processing system in a storage medium, and the following description will specifically describe the functions of each program module in the embodiments.
As shown in fig. 8, the bullet screen information display apparatus 800 may include a position acquisition module 801, a determination module 802, a bullet screen acquisition module 803, and a play module 804, where:
a position obtaining module 801, configured to obtain camera position information of a mobile terminal in a camera coordinate system;
specifically, the camera position information is coordinates of the mobile terminal in a camera coordinate system. In this embodiment, when the mobile terminal displays the bullet screen information using the AR technology, the mobile terminal starts the AR bullet screen program and invokes its own optical system (such as a camera set, including a lens, a shutter, an image sensor, etc.) together with an inertial measurement unit (Inertial Measurement Unit, IMU) and an AR software tool development kit, where the inertial measurement unit may be an accelerometer or a gyroscope, to collect and calculate the position information (x, y, z) and the direction (pitch, yaw, roll) of the device.
The positional information (x, y, z) is coordinates in the camera coordinate system.
The camera coordinate system is a three-dimensional rectangular coordinate system established by taking the focusing center of the camera as an origin and taking the optical axis as a Z axis. The origin of the camera coordinate system (viewing coordinate system) is the optical center of the camera, the X-axis and the Y-axis are parallel to the X, Y-axis of the image, and the z-axis is the optical axis of the camera, which is perpendicular to the graphics plane. The intersection point of the optical axis and the image plane is the origin of the image coordinate system, and the image coordinate system is a two-dimensional rectangular coordinate system.
A determination module 802 for converting the camera position information into spatial position information in a world coordinate system.
The spatial position information is the coordinates of the bullet screen information to be information displayed in a world coordinate system. The coordinates in the camera coordinate system can be converted into coordinates in the world coordinate system by a conversion formula of the camera coordinate system and the world coordinate system.
In an exemplary embodiment, the determining module 802 is further configured to calculate, according to the camera position information, bullet screen generating position information of the bullet screen information to be displayed in the camera coordinate system.
Specifically, the barrage generation position information is coordinates of the barrage information to be information displayed in a camera coordinate system.
In this embodiment, in order to display bullet screen information more randomly and dispersedlyAnd (3) carrying out initial position optimization on the display position of the barrage, namely presetting the generation position of the barrage. In one embodiment, the initialization position optimization may be specifically performed as follows: first, camera position information of a mobile terminal is acquired, wherein in a camera coordinate system, the camera position information is represented by a homogeneous coordinate systemSecondly, after obtaining the camera position information of the mobile terminal, carrying out translation and/or rotation transformation on the camera position p1, finally calculating the position information obtained after the translation and/or rotation transformation, and generating the position information as random and scattered barrages, wherein in a camera coordinate system, the transformed camera position information is represented by a homogeneous coordinate system>In this embodiment, the specific translation distance and rotation angle may be preset, or may be set by the user, which is not limited in this embodiment.
The determining module 802 is further configured to determine the spatial location information according to the barrage generation location information.
Specifically, a transformation relation of the bullet screen information to be displayed is pre-stored in the mobile terminal, and the transformation relation is used for determining the generation position of the bullet screen information to be displayed, and the transformation relation can be: translates a units to the x-axis and rotates b degrees in the pitch direction. In this embodiment, the transformation relationship is correspondingly changed according to the movement and rotation of the camera, and the changed transformation relationship can be automatically maintained by the operating system and the SDK of the mobile terminal.
After the bullet screen generating position information is acquired, the spatial position information corresponding to the bullet screen generating position information can be calculated according to the transformation relation. That is, the conversion of the camera coordinate system and the world coordinate system is calculated from this conversion relation.
And the barrage acquisition module 803 is used for acquiring the barrage information to be displayed.
Specifically, a bullet screen (barrage) refers to a comment subtitle that pops up when a video is viewed over a network. Like the clips highlighted between the rows in the novice, the points of review that fly through between the screens in the video are called barrages, which are intended to provide dense shots with a large or small number of cannons. The bullet screen, as the name implies, is a screen formed by more bullets, and a large number of spitting comments appear to be the bullet screen in the flying shooting game when the comments are floated from the screen.
And the bullet screen information to be displayed is comment information of the user. In an embodiment, the comment information may specifically be a comment of a user on a certain article, or may be a comment of a user on a certain place, or a comment on a certain event, etc. In this embodiment, the comment information may exist in text form, and in another embodiment, the comment information may also exist in any form that can be converted into a 3D model, such as a picture, a 3D model, or a video.
And the bullet screen information to be displayed is stored in the server, and can be obtained from the server through a bullet screen obtaining instruction when the bullet screen information needs to be displayed.
In an exemplary embodiment, the barrage acquisition module 803 is further configured to acquire an object image of the object to be identified.
Specifically, when a user needs to acquire comment information of the user for a certain object to be identified, the user can take a picture of the object to be identified through a camera of the mobile terminal, so that the mobile terminal acquires an object image of the object to be identified. In another embodiment, the object image of the object to be identified may also be obtained directly from an album in the mobile terminal.
The barrage acquisition module 803 is further configured to perform image recognition on the object image by using an object recognition technology, so as to identify an object type.
Specifically, the object image can be subjected to image recognition by using an object recognition model obtained through training of a deep learning technology, so that the category of the object is recognized. In the training process of the object image recognition model, in order to improve the accuracy of model recognition, the used pictures in the training sample data set can be subjected to data enhancement processing by utilizing methods such as cutting, rotation and the like, so that the diversity of the sample data set is improved, and the object recognition model obtained through training of the training sample set can have better recognition rate.
The barrage obtaining module 803 is further configured to obtain comment information associated with the object category from a server according to the object category, where the comment information is barrage information to be information.
Specifically, after the category of the object is identified, category information of the object is transmitted to a server, and the server searches comment information associated with the category of the object according to the acquired category information of the object.
For example, when a user sees a task, the user can take a picture of the task through the mobile terminal, the mobile terminal identifies the task picture, and after identifying the category of the task picture, the task category information is transmitted to the server, and the server searches comment information associated with the task according to the category information. The mobile terminal can use an offline image recognition technology to recognize the handy commodity so as to acquire a data set with higher quality and improve recognition accuracy.
It can be understood that each user can issue comment information on the sponsored commodity through playing the user interface of the barrage information to be displayed, after the user issues the comment information through the user interface, the mobile terminal transmits the comment information to the server, and the server stores the relation between the comment information of the user and the corresponding sponsor to the comment database.
In another exemplary embodiment, the barrage obtaining module 803 is further configured to obtain geographic location information where the mobile terminal is located.
Specifically, the geographic position information of the mobile terminal can be obtained by calling a positioning module in the mobile terminal.
The barrage obtaining module 803 is further configured to obtain comment information associated with the geographical location information from a server according to the geographical location information, where the comment information is barrage information to be information.
Specifically, when the user publishes comment information, the user can acquire the geographical position where the user locates when publishing the comment information, and after the user publishes comment information through the mobile terminal, the mobile terminal uploads comment information containing the geographical position information to the server, and after receiving the comment information containing the geographical position information, the server stores the comment information containing the geographical position information. It can be understood that after the server stores the comment information containing the geographical location information, a background manager can audit the comment information containing the geographical location information published by the user, and delete the comment information when finding unsuitable comment information.
Since the comment information posted by the user contains the geographical position information, after the geographical position information of the mobile terminal is acquired, comment information associated with the geographical position information can be acquired from the server according to the geographical position information.
In this embodiment, the comment information may be site comment information, that is, a comment of a user for a certain site, where the site comment may be in a text form, or may be any form that may be converted into a 3D model, such as a picture, a 3D model, or a video. When the comment information exists in the form of video, the comment information also comprises video URL information of the video.
Illustratively, when a user eats at a restaurant, the user may post comment information for the restaurant, e.g., the restaurant tastes good, and the user may post a video of the restaurant environment for the restaurant environment. When a user publishes comment information through the mobile terminal, the mobile terminal can acquire geographical position information where the user publishes the comment information, and then uploads the comment information containing the geographical position information to a server. Thus, after the mobile terminal acquires the geographic position information, comment information associated with the geographic position information can be acquired from the server according to the acquired geographic position information.
It can be understood that each user can issue place comment information for a certain place by playing the user interface of the bullet screen information to be displayed, after the user issues the place comment information through the user interface, the mobile terminal transmits the place comment information to the server, and the server stores the relationship between the place comment information of the user and the corresponding geographic position information to the comment database. When comment information posted by a user is video, after uploading the video to a server, the server associates and stores URL information corresponding to the video and corresponding geographic position information to a comment database.
And the playing module 804 is configured to display the bullet screen information to be displayed according to the spatial position information.
Specifically, after the spatial position information of the bullet screen information to be displayed is obtained, the mobile terminal may play each piece of bullet screen information to be displayed according to the display size, the font color, the font format and the display special effect of each piece of bullet screen information, where the display special effect includes a stay, a scroll or other special effect modes.
In an exemplary embodiment, the playing module 804 is further configured to obtain a motion track and a motion speed of the bullet screen information to be displayed, and play the bullet screen information to be displayed according to the motion track and the motion speed.
Specifically, the motion track and the motion speed of the bullet screen information may be preset, for example, the preset bullet screen motion track is from right to left relative to the current position of the camera, the motion vector is denoted as a, and after the preset motion vector a is acquired, the preset motion vector a is converted into a vector a' in the world coordinate system.
After the spatial position information of the bullet screen information to be displayed is obtained and the vector a 'in the world coordinate system corresponding to the movement track of the bullet screen is calculated, a bullet screen model is generated in the spatial position information, and then the bullet screen model moves at the movement speed S according to the vector a' in the world coordinate system, so that the bullet screen information to be displayed is played.
In this embodiment, based on different operating systems or SDKs, the barrage model may achieve different degrees of customization effects. Different fonts, colors, illumination shadows and other effects can be used for the barrage model, and besides the text model, a picture texture can also be used for covering the 3D model.
In one embodiment, after the bullet screen is moved to the predetermined location, bullet screen resources may be recovered.
In another embodiment, after displaying the bullet screen information, the user may perform an interactive operation on the bullet screen information, for example, the user may change the moving track of a bullet screen by clicking or sliding. For example, the user can switch the moving track of one barrage to another moving track through sliding operation, after receiving the switching operation of the user on the moving track of the barrage, the mobile terminal can redetermine the direction vector corresponding to the moving track of the barrage according to the sliding operation of the user, and after determining the direction vector corresponding to the new moving track, convert the direction vector into the direction vector corresponding to the world coordinate system, and finally play the barrage information according to the direction vector in the world coordinate system.
In the embodiment, camera position information of the mobile terminal in a camera coordinate system is obtained; converting the camera position information into spatial position information in a world coordinate system; acquiring the bullet screen information to be displayed; and displaying the bullet screen information to be displayed according to the space position information. According to the embodiment of the invention, the bullet screen information is displayed through an augmented reality technology (Augmented Reality, AR for short), so that the bullet screen information can be displayed in a real space, a user can truly experience the feeling that the bullet screen is in a floating state, the user experience is improved, and in addition, the bullet screen information is displayed through the AR technology, so that more various interactions can be performed.
In an exemplary embodiment, as shown in fig. 8, the bullet screen information display apparatus 800 may further include a receiving module 805.
The receiving module 805 is configured to display a page corresponding to the hyperlink barrage when receiving a click operation of the user on the played hyperlink barrage.
Specifically, the information of the bullet screen to be displayed may be a hyperlink bullet screen, when the played information of the bullet screen to be displayed includes the hyperlink bullet screen, the user may click on the hyperlink bullet screen played in the video interface, and after receiving the click operation of the user on the hyperlink bullet screen, the mobile terminal obtains the corresponding page content according to the hyperlink, and then displays the page content.
In this embodiment, by providing the hyperlink barrage, the user may click on the hyperlink barrage, so as to enter the page corresponding to the hyperlink, thereby increasing the interaction mode of the barrage and improving the user experience.
Example IV
Fig. 9 schematically illustrates a hardware architecture diagram of a mobile terminal adapted to implement a barrage information display method according to a fourth embodiment of the present application. In the present embodiment, the mobile terminal 2 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a preset or stored instruction. For example, it may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a rack server (including a stand-alone server or a server cluster composed of a plurality of servers), etc. As shown in fig. 9, the mobile terminal 2 includes at least, but is not limited to: memory 910, processor 920, and network interface 930 may be communicatively linked to each other by a system bus. Wherein:
the memory 910 includes at least one type of computer-readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 910 may be an internal storage module of the mobile terminal 2, such as a hard disk or a memory of the mobile terminal 2. In other embodiments, the memory 910 may also be an external storage device of the mobile terminal 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the mobile terminal 2. Of course, the memory 910 may also include both an internal memory module of the mobile terminal 2 and an external memory device. In this embodiment, the memory 910 is generally used to store an operating system and various application software installed in the mobile terminal 2, such as program codes of a bullet screen information display method, and the like. In addition, the memory 910 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 920 may be a central processing unit (Central Processing Unit, simply CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 920 is generally used to control overall operation of the mobile terminal 2, such as performing control and processing related to data interaction or communication with the mobile terminal 2, and the like. In this embodiment, the processor 920 is configured to execute program codes or process data stored in the memory 910.
The network interface 930 may include a wireless network interface or a wired network interface, which network interface 930 is typically used to establish a communication link between the mobile terminal 2 and other mobile terminals. For example, the network interface 930 is used to connect the mobile terminal 2 with an external terminal through a network, establish a data transmission channel and a communication link between the mobile terminal 2 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, abbreviated as GSM), wideband code division multiple access (Wideband Code Division Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc.
It should be noted that fig. 9 only shows a mobile terminal having components 910-930, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead.
In this embodiment, the bullet screen information display method stored in the memory 910 may also be divided into one or more program modules and executed by one or more processors (the processor 920 in this embodiment) to complete the present invention.
Example six
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the bullet screen information display method of the embodiment.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the mobile terminal, such as a hard disk or a memory of the mobile terminal. In other embodiments, the computer readable storage medium may also be an external storage device of a mobile terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. that are provided on the mobile terminal. Of course, the computer-readable storage medium may also include both an internal memory unit of the mobile terminal and an external memory device thereof. In this embodiment, the computer readable storage medium is generally used to store an operating system and various types of application software installed in the mobile terminal, such as program codes of the bullet screen information display method in the embodiment, and the like. Furthermore, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or a plurality of modules or steps in them may be fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (9)

1. A barrage information display method, comprising:
Acquiring camera position information of the mobile terminal in a camera coordinate system;
converting the camera position information into spatial position information in a world coordinate system, wherein the spatial position information is coordinates of bullet screen information to be displayed in the world coordinate system;
acquiring the bullet screen information to be displayed;
displaying the bullet screen information to be displayed according to the space position information;
the step of converting the camera position information into spatial position information in a world coordinate system includes:
calculating bullet screen generating position information of the bullet screen information to be displayed in the camera coordinate system according to the camera position information;
determining the spatial position information according to the bullet screen generating position information;
the calculating, according to the camera position information, bullet screen generating position information of the bullet screen information to be displayed in the camera coordinate system includes: representing the camera position information by adopting a homogeneous coordinate system; performing translation and/or rotation transformation processing on the camera position information; and calculating homogeneous coordinates obtained after translation and/or rotation transformation treatment, and taking the homogeneous coordinates as bullet screen generation position information.
2. The bullet screen information display method of claim 1 wherein said step of displaying said bullet screen information to be displayed according to said spatial location information comprises:
acquiring a motion trail and a motion speed of the bullet screen information to be displayed;
and playing the bullet screen information to be displayed according to the motion trail and the motion speed.
3. The bullet screen information display method according to any one of claims 1 to 2, wherein the step of acquiring the bullet screen information to be displayed includes:
acquiring an object image of an object to be identified;
performing image recognition on the object image through an object recognition technology to recognize the object type;
and acquiring comment information associated with the object category from a server according to the object category, wherein the comment information is bullet screen information to be information.
4. The bullet screen information display method according to any one of claims 1 to 2, wherein the step of acquiring the bullet screen information to be displayed includes:
acquiring geographic position information of the mobile terminal;
and acquiring comment information related to the geographical position information from a server according to the geographical position information, wherein the comment information is bullet screen information to be information.
5. The bullet screen information display method of claim 4 wherein said comment information includes location comment information and/or video URL information.
6. The method of claim 1, wherein the bullet screen information to be displayed comprises a hyperlink bullet screen, the method further comprising:
and when the clicking operation of the user on the played hyperlink barrage is received, displaying the page corresponding to the hyperlink barrage.
7. A barrage information display device, comprising:
the position acquisition module is used for acquiring camera position information of the mobile terminal in a camera coordinate system;
the determining module is used for converting the camera position information into spatial position information in a world coordinate system, wherein the spatial position information is coordinates of bullet screen information to be displayed in the world coordinate system;
the bullet screen acquisition module is used for acquiring the bullet screen information to be displayed;
the playing module is used for displaying the bullet screen information to be displayed according to the space position information;
the determining module is also used for calculating bullet screen generating position information of the bullet screen information to be displayed in the camera coordinate system according to the camera position information; determining the spatial position information according to the bullet screen generating position information;
The determining module is further used for representing the camera position information by adopting a homogeneous coordinate system; performing translation and/or rotation transformation processing on the camera position information; and calculating homogeneous coordinates obtained after translation and/or rotation transformation treatment, and taking the homogeneous coordinates as bullet screen generation position information.
8. A mobile terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the bullet screen information display method of any one of claims 1 to 6 when the computer program is executed.
9. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program when executed by a processor implements the steps of the barrage information display method of any one of claims 1 to 6.
CN201911007282.7A 2019-10-22 2019-10-22 Barrage information display method and device and mobile terminal Active CN112702643B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201911007282.7A CN112702643B (en) 2019-10-22 2019-10-22 Barrage information display method and device and mobile terminal
US17/073,957 US11651560B2 (en) 2019-10-22 2020-10-19 Method and device of displaying comment information, and mobile terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911007282.7A CN112702643B (en) 2019-10-22 2019-10-22 Barrage information display method and device and mobile terminal

Publications (2)

Publication Number Publication Date
CN112702643A CN112702643A (en) 2021-04-23
CN112702643B true CN112702643B (en) 2023-07-21

Family

ID=75492149

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911007282.7A Active CN112702643B (en) 2019-10-22 2019-10-22 Barrage information display method and device and mobile terminal

Country Status (2)

Country Link
US (1) US11651560B2 (en)
CN (1) CN112702643B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11393202B2 (en) * 2020-02-28 2022-07-19 CareAR Holdings LLC Augmented reality support platform
CN114302184A (en) * 2021-12-28 2022-04-08 阿里巴巴（中国）有限公司 Commodity information display method and equipment
CN116996699A (en) * 2022-10-12 2023-11-03 广州腾讯科技有限公司 Video comment display method, device, terminal and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106982387A (en) * 2016-12-12 2017-07-25 阿里巴巴集团控股有限公司 It has been shown that, method for pushing and the device and barrage application system of barrage
CN108347657A (en) * 2018-03-07 2018-07-31 北京奇艺世纪科技有限公司 A kind of method and apparatus of display barrage information
CN108629843A (en) * 2017-03-24 2018-10-09 成都理想境界科技有限公司 A kind of method and apparatus for realizing augmented reality

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7131060B1 (en) * 2000-09-29 2006-10-31 Raytheon Company System and method for automatic placement of labels for interactive graphics applications
JP5621421B2 (en) * 2010-09-06 2014-11-12 ソニー株式会社 Image processing apparatus, program, and image processing method
JP5799521B2 (en) * 2011-02-15 2015-10-28 ソニー株式会社 Information processing apparatus, authoring method, and program
US9147221B2 (en) * 2012-05-23 2015-09-29 Qualcomm Incorporated Image-driven view management for annotations
US10139985B2 (en) * 2012-06-22 2018-11-27 Matterport, Inc. Defining, displaying and interacting with tags in a three-dimensional model
CN103797783B (en) * 2012-07-17 2017-09-29 松下知识产权经营株式会社 Comment information generating means and comment information generation method
CN103797812B (en) * 2012-07-20 2018-10-12 松下知识产权经营株式会社 Band comments on moving image generating means and with comment moving image generation method
US9361733B2 (en) * 2013-09-02 2016-06-07 Lg Electronics Inc. Portable device and method of controlling therefor
JP2015095147A (en) * 2013-11-13 2015-05-18 ソニー株式会社 Display control device, display control method, and program
US20150262428A1 (en) * 2014-03-17 2015-09-17 Qualcomm Incorporated Hierarchical clustering for view management augmented reality
JP6265027B2 (en) * 2014-04-22 2018-01-24 富士通株式会社 Display device, position specifying program, and position specifying method
KR20160015972A (en) * 2014-08-01 2016-02-15 엘지전자 주식회사 The Apparatus and Method for Wearable Device
JP2017123050A (en) * 2016-01-07 2017-07-13 ソニー株式会社 Information processor, information processing method, program, and server
US20200037034A1 (en) * 2017-03-09 2020-01-30 Eben-Haeser Greyling System and Method for Navigating in a Digital Environment
CN111512272B (en) * 2017-12-20 2024-09-20 索尼公司 Information processing apparatus, information processing method, and program
CN108845737B (en) * 2018-06-29 2019-08-09 掌阅科技股份有限公司 Picture staggered floor display methods, electronic equipment and storage medium
TWI691891B (en) * 2018-09-07 2020-04-21 財團法人工業技術研究院 Method and apparatus for displaying information of multiple objects
US12067683B2 (en) * 2018-09-14 2024-08-20 Streem, Llc Location persistent augmented reality object and annotation placement
US10810430B2 (en) * 2018-12-27 2020-10-20 At&T Intellectual Property I, L.P. Augmented reality with markerless, context-aware object tracking
US10866563B2 (en) * 2019-02-13 2020-12-15 Microsoft Technology Licensing, Llc Setting hologram trajectory via user input
US11064108B2 (en) * 2019-08-21 2021-07-13 Sony Corporation Frame rate control for media capture based on rendered object speed

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106982387A (en) * 2016-12-12 2017-07-25 阿里巴巴集团控股有限公司 It has been shown that, method for pushing and the device and barrage application system of barrage
CN108629843A (en) * 2017-03-24 2018-10-09 成都理想境界科技有限公司 A kind of method and apparatus for realizing augmented reality
CN108347657A (en) * 2018-03-07 2018-07-31 北京奇艺世纪科技有限公司 A kind of method and apparatus of display barrage information

Also Published As

Publication number Publication date
CN112702643A (en) 2021-04-23
US11651560B2 (en) 2023-05-16
US20210118233A1 (en) 2021-04-22

Similar Documents

Publication Publication Date Title
US11860923B2 (en) Providing a thumbnail image that follows a main image
US12211160B2 (en) Techniques for capturing and displaying partial motion in virtual or augmented reality scenes
Dahne et al. Archeoguide: System architecture of a mobile outdoor augmented reality system
JP2023036602A (en) Augmented and virtual reality
AU2014295814B2 (en) Geo-located activity visualisation, editing and sharing
CN104376118B (en) The outdoor moving augmented reality method of accurate interest point annotation based on panorama sketch
US10325410B1 (en) Augmented reality for enhancing sporting events
CN112927349B (en) Three-dimensional virtual special effect generation method and device, computer equipment and storage medium
CN112702643B (en) Barrage information display method and device and mobile terminal
TWI617930B (en) Method and system for sorting a search result with space objects, and a computer-readable storage device
CN110215706B (en) Method, device, terminal and storage medium for determining position of virtual object
KR20150075532A (en) Apparatus and Method of Providing AR
CN108352086A (en) It determines and solar flux information is presented
CN111833457A (en) Image processing method, apparatus and storage medium
US20190147631A1 (en) Computer Readable Media, Information Processing Apparatus and Information Processing Method
CN111031293A (en) Display method, device and system and computer-readable storage medium for panoramic surveillance
Schmalstieg et al. Mobile phones as a platform for augmented reality
CN112215964A (en) Scene navigation method and device based on AR
US20220165032A1 (en) Content distribution system, content distribution method, and content distribution program
US20240276058A1 (en) Video-based interaction method and apparatus, computer device, and storage medium
CN113052753B (en) Panoramic topological structure generation method, device and equipment and readable storage medium
CN113362474A (en) Augmented reality data display method and device, electronic equipment and storage medium
CN112788443B (en) Interaction method and system based on optical communication device
Rainio et al. Presenting historical photos using augmented reality
KR101909994B1 (en) Method for providing 3d animating ar contents service using nano unit block

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant