CN118045356A - A method, device, medium and equipment for displaying game auxiliary information - Google Patents

Abstract

The invention provides a method, a device, a medium and equipment for displaying auxiliary game information; relates to the technical field of computers. According to the invention, the target reference image is determined by matching the current user interface with the reference image in the game auxiliary video, and the game auxiliary information is determined by the target reference image so as to prompt the operation of the user, so that the user does not need to repeatedly switch the game application interface and the game attack interface in the game process, the utilization mode of the game attack is simplified, the convenience of the user in game operation is improved, and the game experience of the user is further improved.

Description

A method, device, medium and equipment for displaying game auxiliary information

Technical Field

The present invention relates to the field of computer technology, and in particular to a method, device, medium and equipment for displaying game auxiliary information.

Background technique

There are currently many game strategy videos on video software. Players can refer to the game strategy videos to perform game operations, thereby improving their winning rate in the game.

Game strategy videos generally exist in video software. When players refer to game strategy videos to perform game operations, they need to switch between the video software and the game software, or split the screen of the terminal device so that the terminal device can display the video software and the game software at the same time. However, whether frequently switching software or referring to game strategy videos to perform game operations in split screen, it will affect the player's gaming experience.

It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in the field.

Summary of the invention

The purpose of the present invention is to provide a method for displaying game auxiliary information, a display device for game auxiliary information, a computer-readable storage medium and a device to address the problem in the prior art that players have poor gaming experience when performing game operations with reference to game auxiliary videos.

In order to solve the above technical problems, the present invention adopts the following technical solutions:

According to a first aspect of an embodiment of the present invention, a method for displaying game auxiliary information is provided, wherein a user interface including a virtual game scene and a virtual object is provided through a terminal device, and the method comprises:

Get a screenshot of the current user interface;

Matching the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image;

Determining the behavior data of the virtual object within a preset time period according to the current time point of the target reference image in the game auxiliary video, and generating corresponding game auxiliary information according to the behavior data;

The game auxiliary information is displayed in the current user interface to provide operation prompts to the user.

In an exemplary embodiment of the present invention, the behavior data includes a displacement vector of the virtual object, and the game auxiliary information includes a movement route of the virtual object;

The determining, according to the current time point of the target reference image in the game auxiliary video, the behavior data of the virtual object within a preset time period, and generating corresponding game auxiliary information according to the behavior data, includes:

Acquire, according to the current time point, a plurality of intermediate reference image frames in the game auxiliary video corresponding to the preset time period;

Extracting feature points of the virtual object in each of the intermediate reference images, and determining position information of the feature points;

Calculating an optical flow field between two adjacent intermediate reference images, and determining a displacement vector of the virtual object between the two adjacent intermediate reference images according to the position information of the feature points and the optical flow field;

A moving route of the virtual object is generated according to the displacement vector.

In an exemplary embodiment of the present invention, generating a movement route of the virtual object according to the displacement vector includes:

A plurality of continuous displacement vectors are smoothed to obtain a moving route of the virtual object.

In an exemplary embodiment of the present invention, matching the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image includes:

Extracting a plurality of first feature points from the virtual game scene area in the captured image, and extracting a plurality of second feature points from the virtual game scene area in each of the reference images;

Describing the local area around each of the first feature points and the local area around each of the second feature points respectively, to obtain a corresponding plurality of first descriptors and a corresponding plurality of second descriptors;

The similarity between each of the first descriptors and each of the second descriptors corresponding to the local area is calculated, and a reference image whose similarity satisfies a preset similarity condition is used as the target reference image. In an exemplary embodiment of the present invention, the displaying of the game auxiliary information in the current user interface includes:

A display window is provided in the current user interface, and the game auxiliary information is displayed in the display window.

In an exemplary embodiment of the present invention, the game auxiliary information further includes an estimated travel time of the moving route of the virtual object;

When displaying the game auxiliary information in the current user interface, the method further includes:

Calculating the estimated travel time according to the moving route and moving speed of the virtual object in the current user interface;

Displaying the estimated travel time of the moving route of the virtual object in the current user interface;

And the estimated travel time of the moving route of the virtual object is updated in real time according to the movement of the virtual object.

In an exemplary embodiment of the present invention, before matching the screen capture image with a plurality of reference images in the game auxiliary video, the method further includes:

Decomposing the game auxiliary video into multiple frames of original images;

Calculating a grayscale histogram of each of the original images, and determining a cumulative histogram according to the frequency of each grayscale level in the grayscale histogram;

A mapping function is determined according to the cumulative histogram, and the grayscale of each pixel in each of the original images is changed according to the mapping function to obtain the corresponding reference images.

According to a second aspect of an embodiment of the present invention, a device for displaying game auxiliary information is provided, wherein a user interface including a virtual game scene and virtual objects is provided through a terminal device, and the device comprises:

An image acquisition module, used to acquire a screen capture image of the current user interface;

An image matching module, used to match the captured screen image with multiple frames of reference images in the game auxiliary video, and determine a target reference image that matches the virtual game scene in the captured screen image;

an auxiliary information generation module, configured to determine the behavior data of the virtual object within a preset time period according to the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information according to the behavior data;

The auxiliary information display module is used to display the game auxiliary information in the current user interface to provide operation prompts to the user.

According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processing unit, implements any of the above methods for displaying game auxiliary information.

According to a fourth aspect of the present invention, there is provided an electronic device, comprising a processing unit; and a storage unit for storing executable instructions of the processing unit; wherein the processing unit is configured to execute any one of the above-described methods for displaying game auxiliary information by executing the executable instructions.

The present invention determines a target reference image by matching the current user interface with a reference image in a game auxiliary video, and determines game auxiliary information through the target reference image to provide operation prompts to the user. The user does not need to repeatedly switch between the game application interface and the game strategy interface during the game, which simplifies the way of using the game strategy, improves the convenience of the user in operating the game, and thus enhances the user's gaming experience.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings herein are incorporated into the invention and constitute a part of the invention, illustrate embodiments consistent with the invention, and are used together with the invention to explain the principles of the invention. Obviously, the accompanying drawings described below are only some embodiments of the invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

FIG. 1 is a schematic diagram showing a system architecture to which a method for displaying auxiliary game information according to an embodiment of the present invention can be applied.

FIG. 2 is a schematic flow chart showing a method for displaying game auxiliary information according to an embodiment of the present invention.

FIG. 3 shows a schematic diagram of a display interface for game auxiliary information designed by a method according to an embodiment of the present invention.

FIG. 4 shows a schematic diagram of a display device for game auxiliary information according to an embodiment of the present invention.

FIG. 5 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present invention.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solutions of the present invention may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present invention.

In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and/or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

Next, the embodiments of the present invention are described in detail.

FIG. 1 is a schematic diagram showing a system architecture to which a method for displaying auxiliary game information according to an embodiment of the present invention can be applied.

As shown in FIG1 , the system architecture 100 may include one or more of the terminal devices such as a desktop computer 101, a portable computer 102, a smart phone 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal device and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal device may be various electronic devices with data processing functions, which have a display screen on the electronic device, and the display screen is used to display game auxiliary videos and game operation interfaces to users. The electronic devices include but are not limited to the above-mentioned desktop computers, portable computers, smart phones, tablet computers, etc. The server 105 may communicate with the game software and video software to obtain relevant data of the current user interface and game auxiliary videos. It should be understood that the number of terminal devices, networks, and servers in FIG1 is only schematic. According to the implementation requirements, there may be any number of terminal devices, networks, and servers. For example, the server 105 may be a server cluster composed of multiple servers.

The method for displaying game auxiliary information provided in the embodiment of the present invention is generally executed by a terminal device, and accordingly, the display device for displaying game auxiliary information is generally set in the terminal device. However, it is easy for those skilled in the art to understand that the method for displaying game auxiliary information provided in the embodiment of the present invention can also be executed by the server 105, and accordingly, the display device for displaying game auxiliary information can also be set in the server 105, which is not particularly limited in this exemplary embodiment.

This exemplary embodiment provides a method for displaying game auxiliary information. Referring to FIG. 2 , the method for displaying game auxiliary information may include the following steps S210 to S240:

Step S210. Obtaining a screenshot of the current user interface;

Step S220. Matching the captured image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured image;

Step S230. Determine the behavior data of the virtual object within a preset time period according to the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information according to the behavior data;

Step S240. Display the game auxiliary information in the current user interface to provide operation prompts to the user.

The present invention determines a target reference image by matching the current user interface with a reference image in a game auxiliary video, and determines game auxiliary information through the target reference image to provide operation prompts to the user. The user does not need to repeatedly switch between the game application interface and the game strategy interface during the game, which simplifies the way of using the game strategy, improves the convenience of the user in operating the game, and thus enhances the user's gaming experience.

The above steps of this exemplary embodiment are described in more detail below.

In step S210, a screen capture image of the current user interface is obtained.

In an example implementation of the present invention, the user interface may be an interface when a user operates a game through a terminal device, and the screenshot image may be an image obtained by taking a screenshot of the interface of the terminal device. Before displaying game auxiliary information in the current user interface to provide operation prompts to the user, it is necessary to first determine the state information of the game through the screenshot image of the current user interface. The state information of the game may include the state information of the virtual game scene and virtual objects in the current user interface.

In step S220, the screenshot image is matched with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the screenshot image.

In an exemplary embodiment of the present invention, the game auxiliary video may be a game strategy video in a video software. After obtaining the game auxiliary video, the game auxiliary video may be first decomposed into multiple frames of original images, and then the grayscale histogram of each original image is calculated, and a cumulative histogram is determined according to the frequency of each grayscale level in the grayscale histogram, and a mapping function is determined according to the cumulative histogram, and the grayscale level of each pixel in each original image is changed according to the mapping function to obtain corresponding reference images.

Specifically, the user may first download and save the game auxiliary video to the terminal device, and then read the game auxiliary video frame by frame through the function provided by the video processing library to decompose the game auxiliary video into multiple frames of original images.

For example, after obtaining each original image, the number of pixels of each gray level in the original image can be counted first, and the gray level histogram of the original image can be drawn. Then, each gray level in the gray level histogram can be grouped, and the frequency of each gray level in each group can be calculated respectively. The frequency of each gray level in each group can be accumulated to obtain the cumulative frequency, and the cumulative histogram can be drawn according to the gray levels in each group and the cumulative frequency of each gray level. The cumulative histogram is normalized so that its value is between 0 and 255, and then the mapping function is determined according to the normalized cumulative histogram. The mapping function can be determined by the following formula:

in, Indicates the new gray level, /> Indicates the current gray level, /> Indicates the minimum gray level, /> Indicates the total number of pixels, /> Indicates the number of gray levels, /> Indicates the cumulative pixel ratio of each gray level in the original image, /> Indicates rounding the calculation result in the following brackets.

After the mapping function is determined, each gray level of each pixel in each original image can be mapped to a new gray level, thereby obtaining corresponding reference images.

In addition to using histograms to enhance the contrast of each original image, the image quality of the original image can also be improved by other image enhancement algorithms. For example, the image quality can be improved by grayscale stretching, bilateral filtering, Fourier transform and other methods, which are not specifically limited in the present invention. Specifically, the contrast of each original image can be enhanced by grayscale stretching; then each original image is smoothed by bilateral filtering, while retaining edge information and reducing the impact of noise; finally, each original image is converted from the spatial domain to the frequency domain by Fourier transform, and each original image is filtered in the frequency domain, and then each original image is converted back to the spatial domain to achieve image enhancement. Enhancing the contrast of each original image to obtain each reference image can improve the accuracy in the subsequent matching process of the captured screen image and the target reference image, thereby more accurately generating the corresponding game auxiliary information.

After obtaining each reference image, multiple first feature points can be extracted from the virtual game scene area in the screenshot image, and multiple second feature points can be extracted from the virtual game scene area in each reference image. The local area around each first feature point and the local area around each second feature point are described respectively to obtain corresponding multiple first descriptors and multiple second descriptors, and the similarity between each first descriptor and each second descriptor corresponding to the local area is calculated, and the reference image whose similarity meets the preset similarity condition is used as the target reference image.

Specifically, first, multiple first feature points of the virtual game scene area in the captured image and multiple second feature points of the virtual game scene area in each reference image can be extracted through feature point detection algorithms such as SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF) or SIFT (Scale Invariant Feature Transform); the local area around each first feature point is numerically described by the descriptor algorithm to generate multiple representative first descriptors; the local area around each second feature point is numerically described to generate multiple corresponding second descriptors. For example, a 128-dimensional first descriptor can be formed by calculating the gradient direction histogram around the first feature point, or a binary first descriptor can be generated by calculating the gray value comparison result near the first feature point. Among them, the feature point can be a robust and unique key point located at a corner, edge or texture-rich area in the image, and the descriptor can be represented in the form of a vector or a matrix.

Next, the similarity between each first descriptor and each second descriptor can be calculated by cosine similarity or other similarity measurement methods, and the reference image whose similarity with the captured image meets the preset matching condition is used as the target reference image. The distance between each first descriptor and each second descriptor can also be calculated by distance measurement such as Euclidean distance, Hamming distance or angular distance, and the reference image corresponding to the second descriptor whose distance with the first descriptor meets the preset matching condition is used as the target reference image. The preset matching condition can be a similarity value or distance value pre-set according to the scene requirements.

Optionally, the captured image and each reference image may be matched by a feature matching method such as a nearest neighbor matching algorithm, a best candidate matching algorithm, or RANSAC (Random Sample Consensus) to determine a target reference image.

In addition, the pixel matching value between the screenshot image and each reference image can also be calculated through a deep learning model. First, the features of the screenshot image and each reference image can be extracted through a pre-trained convolutional neural network model, and then the matching value between the screenshot image and each reference image can be calculated through methods such as cosine similarity or Euclidean distance. When the matching value with the screenshot image reaches a preset value, the corresponding reference image can be used as the target reference image, or the matching values with the screenshot image can be sorted according to size, and the reference image corresponding to the matching value with a higher sorting can be selected as the target reference image.

Optionally, the pixel matching value between the screen capture image and each reference image can also be determined by pixel-level comparison. Specifically, the screen capture image and each reference image can be scaled to the same resolution and color mode, and then the screen capture image and each reference image can be compared pixel by pixel, and then the matching value can be determined by calculating the same pixel ratio or average difference value between the screen capture image and each reference image.

In this example, each reference image is obtained by enhancing the contrast and detail information of each original image, thereby improving the matching accuracy between the screenshot image and the target reference image. The target reference image is determined by calculating the matching value, thereby improving the precision of the comparison between images, thereby more accurately determining the target reference image and making the generated game auxiliary information more accurate.

In step S230, the behavior data of the virtual object within a preset time period is determined according to the current time point of the target reference image in the game auxiliary video, and corresponding game auxiliary information is generated according to the behavior data.

In an example embodiment of the present invention, the virtual object in the game assistance video may be a game character in the game scene shown in the game strategy video; the behavior data of the virtual object may include the displacement vector of the virtual object, and the game assistance information may include the movement route of the virtual object.

For example, the current time point corresponding to the target reference image in the game auxiliary video can be determined first, and then multiple frames of intermediate reference images corresponding to the preset time period in the game auxiliary video can be obtained according to the current time point; the feature points of the virtual object in each intermediate reference image can be extracted, and the position information of the feature points can be determined; the optical flow field between two adjacent intermediate reference images can be calculated, and the displacement vector of the virtual object between two adjacent intermediate reference images can be determined according to the position information of the feature points and the optical flow field; and the movement route of the virtual object can be generated according to each displacement vector.

Specifically, when determining the current time point corresponding to the target reference image in the game auxiliary video, the number of frames played per second in the game auxiliary video can be determined first and the frame rate of the game auxiliary video can be calculated; the position information corresponding to the target reference image in the game auxiliary video can be obtained, where the position information of the target reference image can be represented by a frame number; and then the current time point corresponding to the target reference image in the game auxiliary video can be determined by the frame number of the target reference image and the frame rate of the game auxiliary video. The current time point can also be determined by the time point corresponding to the playback progress below the game auxiliary video.

After determining the current time point corresponding to the target reference image, the preset time period can be determined through the behavior data of the virtual game scene and the virtual object in the game auxiliary video. For example, the time it takes for the virtual object to complete a certain action in the virtual game scene can be used as the preset time period, or a fixed duration can be set as the preset time period according to actual needs. After obtaining multiple frames of intermediate reference images corresponding to the preset time period in the game auxiliary video, the virtual object can be determined by a target detection algorithm, an image segmentation algorithm, or other algorithms for identifying virtual objects, and feature points of the virtual object can be extracted by feature extraction algorithms such as SIFT, SURF, or FAST (Features from accelerated segment test), and then the position information of the feature points of the virtual object can be obtained, where the position information can be represented by coordinates.

After obtaining the position information of the feature points of the virtual object, the optical flow field between two adjacent reference frames can be calculated by feature extraction and optical flow calculation methods, and the displacement vector of the virtual object between adjacent frames can be determined based on the position information of the feature points of the virtual object and the optical flow field. Finally, multiple continuous displacement vectors of the virtual object within a preset time period are smoothed by smoothing methods such as Gaussian smoothing to obtain the movement route of the virtual object. The movement route of the virtual object can be displayed in the form of a curve.

In an exemplary embodiment of the present invention, a screenshot of the current user interface may be re-acquired after a preset period of time, so as to obtain a new movement route by comparing the new screenshot with the game auxiliary video.

In this example, the movement route of the virtual object is generated according to the displacement vector of each adjacent frame of the virtual object in the game auxiliary video, so that the user can move the virtual object in the current user interface with reference to the movement route of the virtual object in the game auxiliary video, thereby improving the game winning rate and enhancing the user's gaming experience.

In step S240, the game auxiliary information is displayed in the current user interface to provide operation prompts to the user.

In an example implementation of the present invention, a display window may be provided in the current user interface, and game auxiliary information may be displayed in the display window. The display window may be a full-screen window with high transparency that occupies the entire current user interface, or a small semi-transparent window that is smaller than the current user interface; when displaying the movement route of a virtual object in the display window, the starting point and end point of the movement route may be indicated by arrows or color differences; when there are multiple virtual objects and corresponding movement routes, text or avatars may be added above the movement route to help users identify the virtual objects corresponding to the movement route.

Optionally, the estimated travel time can be calculated based on the moving route and moving speed of the virtual object in the current user interface, the estimated travel time of the moving route of the virtual object can be displayed in the current user interface, and the estimated travel time of the moving route of the virtual object can be updated in real time based on the movement of the virtual object. Other prompt information can also be displayed in the current user interface according to actual needs, and the present invention does not specifically limit this.

Optionally, you can also add pause, resume, and close icons to the corners of the display window. When the user does not need to operate through the game auxiliary information, you can click the pause icon to pause the display of the game auxiliary information, and the display window will be temporarily minimized, so that only the continue and close icons can be seen on the user interface. The user can click the continue icon to continue displaying the display window, or click the close icon to completely close the display window. The setting of the pause, open, and close icons can prevent the game auxiliary information from interfering with the user's operation when the user does not need to use the game auxiliary information, further enhancing the user's gaming experience.

As shown in Figure 3, Figure 3 shows a schematic diagram of a display interface of game auxiliary information designed by the method of an embodiment of the present invention. Among them, the display interface of the game auxiliary information includes a moving route 310 of a virtual object, an estimated travel time display module 320, an operation icon module 330, and a real-time position 340 of a virtual object. The operation icon module 330 may include a pause icon 331, a resume icon 332, and a close icon 333. The moving route 310 of the virtual object can be displayed in the form of a curve, the estimated travel time display module 320 can also be used to display other game auxiliary information, and the operation icon module 330 can also increase or decrease the operation icons according to different needs to achieve different operation needs.

In this example, game auxiliary information is displayed in the current user interface, so that the user can directly refer to the game auxiliary information to play the game without opening the game auxiliary video, which reduces the operation steps when the user refers to the game auxiliary video to play the game and improves the user's gaming experience.

In this example implementation, a display device for game auxiliary information is also provided. Referring to FIG. 4 , the display device 400 for game auxiliary information may include an image acquisition module 410, an image matching module 420, an auxiliary information generation module 430, and an auxiliary information display module 440, wherein:

An image acquisition module 410 is used to acquire a screen capture image of the current user interface;

An image matching module 420 is used to match the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image;

An auxiliary information generation module 430 is used to determine the behavior data of the virtual object within a preset time period according to the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information according to the behavior data;

The auxiliary information display module 440 is used to display the game auxiliary information in the current user interface to provide operation prompts to the user.

In an optional implementation, the behavior data includes a displacement vector of the virtual object, and the game auxiliary information includes a movement route of the virtual object; the auxiliary information generation module 430 includes:

A reference image acquisition module, used for acquiring, according to the current time point, a plurality of intermediate reference images corresponding to the preset time period in the game auxiliary video;

A position information determination module, used to extract feature points of the virtual object in each of the intermediate reference images and determine position information of the feature points;

A displacement vector determination module, used to calculate the optical flow field between two adjacent intermediate reference images, and determine the displacement vector of the virtual object between the two adjacent intermediate reference images according to the position information of the feature points and the optical flow field;

A movement route generating module is used to generate a movement route of the virtual object according to the displacement vector.

In an optional implementation, the moving route generation module includes:

The displacement vector processing module is used to perform smoothing on a plurality of continuous displacement vectors to obtain a moving route of the virtual object.

In an optional implementation, the image matching module 420 includes:

A feature point extraction module, used to extract a plurality of first feature points from the virtual game scene area in the screenshot image, and to extract a plurality of second feature points from the virtual game scene area in each of the reference images;

A descriptor generation module, used to describe the local area around each of the first feature points and the local area around each of the second feature points, respectively, to obtain a corresponding plurality of first descriptors and a plurality of second descriptors;

The target image determination module is used to calculate the similarity between each of the first descriptors and each of the second descriptors corresponding to the local area, and use the reference image whose similarity meets the preset similarity condition as the target reference image.

In an optional implementation, the auxiliary information display module 440 includes:

A display window display module is used to provide a display window in the current user interface and display the game auxiliary information in the display window.

In an optional implementation, the game auxiliary information further includes an estimated travel time of the moving route of the virtual object; the auxiliary information display module 440 further includes:

A travel time determination module, configured to calculate the estimated travel time according to the moving route and moving speed of the virtual object in the current user interface;

A travel time display module, used to display the estimated travel time of the moving route of the virtual object in the current user interface;

The travel time updating module is used to update the estimated travel time of the moving route of the virtual object in real time according to the movement of the virtual object.

In an optional implementation, the image matching module 420 further includes:

A video decomposition module, used for decomposing the game auxiliary video into multiple frames of original images;

A cumulative histogram determination module, used for calculating the grayscale histogram of each of the original images, and determining the cumulative histogram according to the frequency of each grayscale level in the grayscale histogram;

The reference image generation module is used to determine a mapping function according to the cumulative histogram, and to change the gray level of each pixel in each of the original images according to the mapping function to obtain the corresponding reference images.

The specific details of each module in the above-mentioned game auxiliary information display device have been described in detail in the corresponding game auxiliary information display method, so they will not be repeated here.

The exemplary embodiments of the present invention also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible embodiments, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on an electronic device, the program code is used to cause the electronic device to perform the steps of various exemplary embodiments of the present invention described in the above-mentioned "Exemplary Method" section of this specification. The program product can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

The program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C#, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

The exemplary embodiment of the present invention also provides an electronic device capable of implementing the above method. The electronic device 400 according to this exemplary embodiment of the present invention is described below with reference to FIG5. The electronic device 500 shown in FIG5 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

As shown in Fig. 5, electronic device 500 may be in the form of a general-purpose computing device. Components of electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), and a display unit 540.

The storage unit 520 stores program codes, which can be executed by the processing unit 510, so that the processing unit 510 performs the steps of various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification. For example, the processing unit 510 can perform the method steps in Figure 2. The storage unit 520 can be used to store a screen capture image of the current user interface, a game auxiliary video, and the game auxiliary information finally obtained, etc., which is not limited by the present invention.

The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 521 and/or a cache memory unit (Cache) 522 , and may further include a read-only memory unit (ROM) 523 .

The storage unit 520 may also include a program/utility 524 having a set (at least one) of program modules 525, such program modules 525 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

Bus 530 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 500 may also communicate with one or more external devices 600 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 500, and/or communicate with any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input/output (I/O) interface 550. In addition, the electronic device 500 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and/or public networks, such as the Internet) through a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 500 through a bus 530. It should be understood that, although not shown in the figure, other hardware and/or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc. Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein may be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation mode of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary implementation mode of the present invention.

In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

Other embodiments of the invention will readily occur to those skilled in the art after considering and practicing the inventions invented herein. The invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not invented by the invention. The invention and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims (10)

1. A game auxiliary information display method is characterized in that: providing, by a terminal device, a user interface comprising a virtual game scene and a virtual object, the method comprising:

acquiring a screen capturing image aiming at a current user interface;

matching the screen capturing image with a multi-frame reference image in a game auxiliary video, and determining a target reference image matched with a virtual game scene in the screen capturing image;

determining behavior data of the virtual object in a preset time period according to the current time point of the target reference image in the game auxiliary video, and generating corresponding game auxiliary information according to the behavior data;

And displaying the game auxiliary information in the current user interface so as to prompt the user for operation.

2. The game support information presentation method of claim 1, wherein: the behavior data comprises a displacement vector of the virtual object, and the game auxiliary information comprises a moving route of the virtual object;

Determining behavior data of the virtual object in a preset time period according to a current time point of the target reference image in the game auxiliary video, and generating corresponding game auxiliary information according to the behavior data, wherein the game auxiliary information comprises:

Acquiring multi-frame intermediate reference images corresponding to the preset time period in the game auxiliary video according to the current time point;

Extracting characteristic points of the virtual object in each intermediate reference image, and determining position information of the characteristic points;

Calculating an optical flow field between two adjacent frames of intermediate reference images, and determining a displacement vector of the virtual object between the two adjacent frames of intermediate reference images according to the position information of the characteristic points and the optical flow field;

And generating a moving route of the virtual object according to the displacement vector.

3. The game support information presentation method of claim 2, wherein: the generating the moving route of the virtual object according to the displacement vector includes:

And carrying out smoothing processing on the continuous displacement vectors to obtain a moving route of the virtual object.

4. The game support information presentation method of claim 1, wherein: the step of matching the screen capturing image with a plurality of frame reference images in a game auxiliary video, and determining a target reference image matched with a virtual game scene in the screen capturing image comprises the following steps:

Extracting a plurality of first feature points from the virtual game scene area in the screen capturing image, and extracting a plurality of second feature points from the virtual game scene area in each reference image;

describing the local area around each first characteristic point and the local area around each second characteristic point respectively to obtain a plurality of corresponding first descriptors and a plurality of second descriptors;

And calculating the similarity between each first descriptor and each second descriptor corresponding to the local area, and taking the reference image with the similarity meeting the preset similarity condition as the target reference image.

5. The game support information presentation method of claim 1, wherein: the displaying the game auxiliary information in the current user interface comprises:

Providing a display window in the current user interface, and displaying the game auxiliary information in the display window.

6. The game support information presentation method of claim 2, wherein: the game auxiliary information further includes a predicted passage time of a moving route of the virtual object;

When the game auxiliary information is presented in the current user interface, the method further comprises:

calculating the estimated transit time according to the moving route and the moving speed of the virtual object in the current user interface;

Displaying the predicted passing time of the moving route of the virtual object in the current user interface;

And updating the expected transit time of the moving route of the virtual object in real time according to the movement of the virtual object.

7. The game support information presentation method of claim 1, wherein: before matching the screen capture image with the plurality of reference images in the game auxiliary video, the method further comprises:

Decomposing the game auxiliary video into a plurality of frames of original images;

Calculating a gray level histogram of each original image, and determining a cumulative histogram according to the frequency of each gray level in the gray level histogram;

And determining a mapping function according to the cumulative histogram, and changing the gray level of each pixel in each original image according to the mapping function to obtain each corresponding reference image.

8. The utility model provides a display device of recreation auxiliary information which characterized in that: providing, by a terminal device, a user interface comprising a virtual game scene and a virtual object, comprising:

the image acquisition module is used for acquiring a screen capturing image aiming at the current user interface;

The image matching module is used for matching the screen capturing image with a plurality of frame reference images in the game auxiliary video and determining a target reference image matched with a virtual game scene in the screen capturing image;

The auxiliary information generation module is used for determining behavior data of the virtual object in a preset time period according to the current time point of the target reference image in the game auxiliary video and generating corresponding game auxiliary information according to the behavior data;

And the auxiliary information display module is used for displaying the game auxiliary information in the current user interface so as to prompt the user to operate.

9. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processing unit, implements the method of presenting game auxiliary information according to any one of claims 1 to 7.

10. An electronic device, characterized in that: comprising the following steps:

A processing unit; and

A storage unit configured to store executable instructions of the processing unit;

wherein the processing unit is configured to perform the method of presenting game auxiliary information of any one of claims 1-7 via execution of the executable instructions.