CN114422697B - Virtual shooting method, system and storage medium based on optical capture - Google Patents

Abstract

The present invention provides a virtual shooting method, system and storage medium based on optical capture, and relates to the technical field of virtual production. The second location information is used for virtual shooting, and finally the picture captured by the virtual camera is pushed to the smart terminal, which solves the problem that the transformation of the virtual camera position and parameters in the prior art is carried out through the animation curve manually made by the production staff, which does not have real-time sexual technical issues. Compared with the prior art in which multiple links are separated and finally stitched together, the virtual shooting method based on optical capture in the present invention has the advantage of being able to see the shooting picture in real time as in traditional shooting, not only allowing directors and photographers to see real-time The pictures taken by the camera simulate the operation and interaction of the real camera.

Description

Virtual shooting method, system and storage medium based on optical capture

technical field

The present invention relates to the technical field of virtual production, in particular to a virtual shooting method, system and storage medium based on optical capture.

Background technique

Virtual shooting refers to that in film shooting, according to the shooting action required by the director, all the shots are carried out in the virtual scene in the computer. All the elements needed to shoot this shot, including scenes, characters, lighting, etc., are all integrated into the computer, and then the director can "direct" the performance and actions of the characters on the computer according to his own intentions, from any angle Sport his lens.

However, in the existing virtual shooting methods, there are still the following problems: on the one hand, since the position and parameter transformation of the virtual camera are mostly animation curves manually produced by the production staff, they are not the result of real-time shooting during the shooting process, and are not It is real-time, so many virtual shootings are not real-time shootings. They are separated from various links and finally stitched together. It is impossible to see the shooting pictures in real time like traditional shootings. On the other hand, many data are mostly produced manually, which seriously affects The pace and realistic accuracy of the virtual shoot.

Contents of the invention

The purpose of the present invention is to solve the problems mentioned in the background technology, and propose a virtual shooting method, system and storage medium based on optical capture.

In order to achieve the above object, the present invention first proposes a virtual shooting method based on optical capture, including the following steps: acquiring the first position information determined by the optical capture smart terminal; generating second position information according to the first position information; Carrying out virtual shooting according to the second location information; pushing the picture captured by the virtual camera to the smart terminal.

Optionally, the following steps are also included: obtaining the first face data; generating second face data by matching the first face data; using the second face data as the preset character model Face target volume data to generate facial expressions corresponding to preset character models.

Optionally, said generating the second face data by matching the first face data includes the following steps: using the first face data in the expressionless state as an initial value, and the first person in each other frame The initial value is subtracted from the face data to perform face initialization; the first face data after face initialization is multiplied by a preset coefficient to perform overall expression scaling.

Optionally, the following steps are further included: forming skeleton data according to the acquired key point position data; binding the skeleton data to the skeleton model; driving the avatar in real time according to the bound skeleton data.

Optionally, the smart terminal is a tablet computer.

The present invention also proposes a virtual shooting system based on optical capture, including: a first position information processing module configured to acquire the first position information determined by the optical capture smart terminal; a second position information processing module configured to Generate second position information according to the first position information; the virtual shooting module is configured to use the second position information as the position of the virtual camera to perform virtual shooting; the screen push module is configured to capture the virtual camera The screen is pushed to the smart terminal.

Optionally, it also includes: a first face data processing module configured to acquire first face data; a second face data processing module configured to generate a face data by matching the first face data The second human face data; a facial expression generation module configured to use the second human face data as the human face object data of the preset character model to generate facial expressions corresponding to the preset character model.

Optionally, the second human face data processing module also includes: a human face initialization module configured to use the first human face data in the expressionless state as an initial value, and subtract the first human face data of each other frame The initial value is removed to initialize the face; the overall expression scaling module is configured to multiply the first human face data initialized by the face by a preset coefficient to perform overall expression scaling

The present invention also proposes a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned virtual shooting method based on optical capture is realized.

Beneficial effects of the present invention:

In a virtual shooting method based on optical capture in an embodiment of the present invention, the first position information determined by the optical capture smart terminal is obtained first, and then the virtual shooting is performed according to the second position information generated by the first position information, and finally the The picture captured by the virtual camera is pushed to the smart terminal, which solves the technical problem in the prior art that the transformation of the position and parameters of the virtual camera is carried out through the animation curve manually made by the production staff and does not have real-time performance. Compared with the existing technology in which various links are separated and finally stitched together, a virtual shooting method based on optical capture in the embodiment of the present invention has the same advantage as traditional shooting that can see the shooting picture in real time, not only allowing the director and The videographer sees the images captured by the camera in real time, and can simulate the operation and interaction of the real camera.

The features and advantages of the present invention will be described in detail with reference to the accompanying drawings.

Description of drawings

FIG. 1 is one of the schematic flow diagrams of a virtual shooting method based on optical capture according to an embodiment of the present invention;

FIG. 2 is the second schematic flow diagram of a virtual shooting method based on optical capture according to an embodiment of the present invention;

Fig. 3 is the third schematic flow diagram of a virtual shooting method based on optical capture according to an embodiment of the present invention;

4 is one of the structural block diagrams of a virtual shooting system based on optical capture according to an embodiment of the present invention;

5 is the second structural block diagram of a virtual shooting system based on optical capture according to an embodiment of the present invention;

FIG. 6 is the third structural block diagram of a virtual shooting system based on optical capture according to an embodiment of the present invention.

Detailed ways

In order to facilitate the understanding of those skilled in the art, the present invention will be further described in detail below in conjunction with specific embodiments.

Fig. 1 schematically shows a schematic flowchart of a virtual shooting method based on optical capture according to an embodiment of the present invention.

As shown in Figure 1, the virtual shooting method based on optical capture includes step S10 to step S40:

Step S10, acquiring the first position information determined by the optical capture smart terminal;

Step S20, generating second location information according to the first location information;

Step S30, performing virtual shooting according to the second location information;

Step S40, pushing the picture captured by the virtual camera to the smart terminal.

In a virtual shooting method based on optical capture in an embodiment of the present invention, the first position information determined by the optical capture smart terminal is obtained first, and then the virtual shooting is performed according to the second position information generated by the first position information, and finally Pushing the picture captured by the virtual camera to the smart terminal solves the technical problem in the prior art that the transformation of the position and parameters of the virtual camera is carried out through the animation curve manually made for the production staff and does not have real-time performance. Compared with the existing technology in which various links are separated and finally stitched together, a virtual shooting method based on optical capture in the embodiment of the present invention has the same advantage as traditional shooting that can see the shooting picture in real time, not only allowing the director and The videographer sees the images captured by the camera in real time, and simulates the operation and interaction of the real camera.

Next, each step of the optical capture-based virtual shooting method in the embodiment of the present invention will be described in more detail with reference to the accompanying drawings and embodiments.

Step S10, acquiring the first location information determined by the optical capture smart terminal.

It should be noted that, as a device that represents a virtual camera in the real world, a smart terminal with a shooting function has the following two conditions: one is that it can receive the push stream of the shooting screen in the virtual camera, and the other is that it can interact with the virtual camera accordingly Function. In a preferred embodiment, a tablet computer is selected as the smart terminal, and in other embodiments, other devices such as mobile phones may also be selected as the smart terminal. In addition, the process of optically capturing the smart terminal can be performed by using the existing optical capturing technology, which will not be repeated here.

Step S20, generating second location information according to the first location information.

The purpose of this step is to unify the consistency of the real coordinates and the virtual coordinates, so as to ensure the consistency between the movement of the smart terminal representing the real camera in the real world and the movement of the virtual camera in the virtual world.

Specifically, after receiving the unprocessed first position information representing the position of the smart terminal in the real world, the first position information is mapped to the virtual light capture field coordinate system to generate the second position information.

It should be noted that in the process of optically capturing smart terminals, the effective range area of optical capture in the real world is called the optical capture field, and this area limits the movement range of objects that need to be optically captured, such as virtual cameras. Since the positive directions of the X-axis and Y-axis in the light-trapping field are not consistent with the positive directions of the X-axis and Y-axis in the virtual environment, the corresponding position information needs to be converted when it is transmitted in the light-trapping field and the virtual environment to ensure the movement consistency. In this embodiment, the second position information is generated by mapping the first position information to the coordinate system of the virtual light trapping field, thereby saving the complexity of multiple coordinate system transformations.

Step S30, virtual shooting is performed according to the second location information.

Since the second position information changes synchronously with the first position information, the virtual camera can change its position or orientation in the virtual scene according to the position change of the real camera to perform real-time virtual shooting.

Step S40, pushing the picture captured by the virtual camera to the smart terminal.

By pushing the picture captured by the virtual camera to the smart terminal, the virtual shooting picture can be displayed in real time on the smart terminal. Compared with the prior art in which multiple links are separated and finally stitched together, this embodiment has the advantage of being able to see the shooting picture in real time just like traditional shooting.

Please refer to Fig. 2, a kind of virtual shooting method based on optical capture in the embodiment of the present invention also includes the following steps:

Step S11, acquiring the first face data; collecting the target face with an optical camera and/or a depth camera to obtain the first face data.

Step S21, generating second face data by performing matching processing on the first face data, in order to reduce the difference between the same virtual character generated by different people.

Step S31, using the second face data as the face object data of the preset character model to generate facial expressions corresponding to the preset character model.

Please refer to Fig. 3, by matching the first face data, generating the second face data specifically includes the following steps:

Step S2110, using the first face data in the expressionless state as an initial value, and subtracting the initial value from the first face data of each other frame to perform face initialization;

Step S2120: Multiply the first human face data initialized by the human face by a preset coefficient to perform overall expression scaling.

According to the results of the actual test, according to the obtained first face data of different actors, for the same facial expression, such as opening the mouth, some actors may have a numerical change of 0.3 to 0.7, and some may have a numerical change of 0.2 to 0.9 Variety. Through the above steps, when the obtained first face data of different actors is overall too large or too small, the similarity of the same virtual character performed by different actors is adjusted and the expression details of the virtual character are supplemented. , so as to achieve the technical effect that the generated virtual characters will not have too much difference when the actors are replaced.

In addition, this embodiment also has the function of dynamically capturing the human body so that the virtual character moves synchronously with the real human body. Specifically, the above functions are realized through the following steps: forming skeleton data according to the acquired key point position data; binding the skeleton data to the skeleton model; driving the avatar in real time according to the bound skeleton data.

Based on the above-mentioned virtual shooting method based on optical capture, an embodiment of the present invention also provides a virtual shooting system based on optical capture, as shown in Figure 4, the system includes the following modules:

The first location information processing module 100 is configured to acquire the first location information determined by the optical capture smart terminal;

The second location information processing module 200 is configured to generate second location information according to the first location information;

The virtual shooting module 300 is configured to use the second position information as the position of the virtual camera to perform virtual shooting;

The screen push module 400 is configured to push the screen captured by the virtual camera to the smart terminal.

As shown in Figure 5, in one embodiment, a kind of virtual shooting system based on optical capture also includes:

The first face data processing module 110 is configured to acquire the first face data;

The second face data processing module 210 is configured to generate second face data by performing matching processing on the first face data;

The facial expression generating module 310 is configured to use the second human face data as the human face object data of the preset character model, so as to generate a facial expression corresponding to the preset character model.

As shown in Figure 6, in one embodiment, the second face data processing module also includes:

The face initialization module 21100 is configured to use the first face data in the expressionless state as an initial value, and subtract the initial value from the first face data of each other frame to perform face initialization;

The overall expression scaling module 21200 is configured to multiply the first human face data initialized by the face by a preset coefficient to perform overall expression scaling.

To sum up, the optical capture-based virtual shooting system of the embodiment of the present invention can be realized as a program and run on a computer device. The memory of the computer device can store various program modules that make up the virtual shooting system based on optical capture, for example, the first position information processing module 100 shown in FIG. 4 , the second position information processing module 200, the virtual shooting module 300, the picture Push module 400. The program constituted by the various program modules enables the processor to execute the steps in the optical capture-based virtual shooting method of the various embodiments of the present application described in this specification.

An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in a virtual shooting method based on optical capture in various embodiments of the present application are implemented.

The various technical features of the above-mentioned embodiments can be combined arbitrarily. For the sake of concise description, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiment is an illustration of the present invention, not a limitation of the present invention, and any solution after a simple transformation of the present invention belongs to the protection scope of the present invention. The above descriptions are only preferred implementations of the present invention, and the scope of protection of the present invention is not limited to the above examples, and all technical solutions that fall under the idea of the present invention belong to the scope of protection of the present invention. It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principles of the present invention should also be regarded as the protection scope of the present invention.

Claims (5)

1. The virtual shooting method based on optical capturing is characterized by comprising the following steps of:

acquiring first position information determined by an optical capturing intelligent terminal;

generating second position information according to the first position information;

performing virtual photographing according to the second position information;

pushing the picture shot by the virtual camera to the intelligent terminal;

further comprises: acquiring first face data;

generating second face data by carrying out matching processing on the first face data;

taking the second face data as face target data of a preset character model, and generating a facial expression corresponding to the preset character model;

wherein the generating the second face data by performing the matching processing on the first face data includes:

taking the first face data in the state of no expression as an initial value, subtracting the initial value from the first face data of each other frame to initialize the face;

and multiplying the first face data initialized by the face by a preset coefficient to perform expression overall scaling, wherein the preset coefficient represents the degree of different actors when making the same expression.

2. The virtual photographing method based on optical capturing as claimed in claim 1, further comprising the steps of:

forming skeleton data according to the acquired key point position data;

binding the bone data with a bone model;

the virtual character is driven in real time according to the bound bone data.

3. The virtual shooting method based on optical capturing of claim 1, wherein the intelligent terminal is a tablet computer.

4. A virtual photographing system based on optical capturing, comprising:

the first position information processing module is configured to acquire first position information determined by the optical capturing intelligent terminal;

a second location information processing module configured to generate second location information from the first location information;

a virtual photographing module configured to perform virtual photographing using the second position information as a position of a virtual camera;

the picture pushing module is configured to push pictures shot by the virtual camera to the intelligent terminal;

further comprises: the first face data processing module is configured to acquire first face data;

the second face data processing module is configured to generate second face data by carrying out matching processing on the first face data;

a facial expression generating module configured to take the second facial data as facial target volume data of a preset character model to generate a facial expression corresponding to the preset character model;

the second face data processing module further includes:

the face initialization module is configured to take the first face data in the non-expression state as an initial value, and the initial value is subtracted from the first face data of each other frame to perform face initialization;

the expression overall scaling module is configured to multiply the face initialized first face data by a preset coefficient to perform expression overall scaling, wherein the preset coefficient represents the degree of different actors doing the same expression.

5. A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, implements the optical capture-based virtual photographing method of any one of claims 1 to 3.