CN115035306A - Image material determination method and related device - Google Patents

Abstract

The application provides an image material determining method and a related device, and firstly, target characteristic data of a target image is determined; then, highlight feature data and standard feature data are determined according to the target feature data, wherein the highlight feature data are used for indicating image features without the influence of the saturated pixels, and the standard feature data are used for indicating image features without the influence of the saturated pixels; then, determining fusion characteristic data according to the highlight characteristic data and the standard characteristic data; and finally, determining the material data of the target image according to the fusion characteristic data, wherein the material data comprises at least one of a diffuse reflection material map, a roughness material map, a mirror surface material map and a normal line material map. The method has the advantages that excessive acquisition of the input target image is not needed, accurate material data can be obtained only by a single target image, and the efficiency and the accuracy of material estimation are greatly improved.

Description

Image material determination method and related device

technical field

The present application relates to the technical field of image analysis, in particular to an image material determination method and related devices.

Background technique

With the development of technology, in many fields, it is necessary to determine the real material information of the object according to the image. However, due to the inherent complexity of the material in the real world, additional cameras, cameras and other acquisition equipment are required to perform detailed spatial and angle analysis of the object. Sampling to get the input image, which greatly affects the efficiency of the material analysis of the image.

SUMMARY OF THE INVENTION

In view of this, the present application provides a method that can determine accurate material data according to a single image without the need for other parameters, which can greatly improve the efficiency and accuracy of material estimation.

In a first aspect, an embodiment of the present application provides a method for determining an image material, the method comprising:

Determine the target feature data of the target image;

Determine highlight feature data and standard feature data according to the target feature data, where the highlight feature data is used to indicate an image feature that eliminates the influence of saturated pixels, and the standard feature data is used to indicate an image feature that does not eliminate the influence of the saturated pixel;

Determine fusion feature data according to the highlight feature data and the standard feature data;

The material data of the target image is determined according to the fusion feature data, and the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map.

In a second aspect, an embodiment of the present application provides an image material determination device, the device comprising:

A feature determination unit, which determines the target feature data of the target image in English;

A branch extraction unit, configured to determine highlight feature data and standard feature data according to the target feature data, where the highlight feature data is used to indicate the image features that eliminate the influence of saturated pixels, and the standard feature data is used to indicate that the saturation is not eliminated Pixel-influenced image features;

a fusion extraction unit, configured to determine fusion characteristic data according to the highlight characteristic data and the standard characteristic data;

A material estimation unit, configured to determine material data of the target image according to the fusion feature data, where the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory, and are configured by the above Executed by the processor, the above program includes instructions for executing steps in any method in the first aspect of the embodiments of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the computer program as described in the first embodiment of the present application. In one aspect some or all of the steps described in any method.

In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute as implemented in the present application. Examples include some or all of the steps described in any method of the first aspect. The computer program product may be a software installation package.

It can be seen that, through the above-mentioned image material determination method and related device, first, the target feature data of the target image is determined; then, according to the target feature data, highlight feature data and standard feature data are determined, and the highlight feature data is used to indicate the elimination of saturated pixels. Influenced image features, the standard feature data is used to indicate the image features that are not affected by the saturated pixels; then, fusion feature data is determined according to the highlight feature data and the standard feature data; finally, according to the fusion feature The data determines material data of the target image, and the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map, and a normal material map. There is no need to collect too many input target images, and only a single target image is needed to obtain accurate material data, which greatly improves the efficiency and accuracy of material estimation.

Description of drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention, which are of great significance to the art For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a system architecture diagram of a method for determining an image material provided by an embodiment of the present application;

2 is a schematic flowchart of a method for determining an image material according to an embodiment of the present application;

3A is a schematic structural diagram of a material estimation model provided by an embodiment of the present application;

3B is a schematic structural diagram of another material estimation model provided by an embodiment of the present application;

3C is a schematic structural diagram of another material estimation model provided by an embodiment of the present application;

4 is a schematic flowchart of another image material determination method provided by an embodiment of the present application;

5 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 6 is a block diagram of functional units of an image material determination device provided by an embodiment of the present application;

FIG. 7 is a block diagram of functional units of another image material determination apparatus provided by an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

It should be understood that the term "and/or" in this document is only an association relationship to describe associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, and A and B exist at the same time , there are three cases of B alone. In addition, the character "/" in this text indicates that the related objects are an "or" relationship. The "plurality" in the embodiments of the present application refers to two or more.

The "connection" in the embodiments of the present application refers to various connection modes such as direct connection or indirect connection, so as to realize communication between devices, which is not limited in the embodiments of the present application.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The background technology and related terms of the present application are described below.

Background technology related:

basic concept:

Physically Based Rendering (PBR) refers to the technology of rendering real pictures by establishing rendering equations that simulate real phenomena based on basic real physical laws and mathematical derivation. Compared with the previous traditional basic simulation rendering, PBR began to abide by the laws of physics, making the rendering more realistic. However, due to the limitations of the current hardware level, it is not completely calculated according to the laws of the real world. It is between pure empirical algorithm simulation rendering and real rendering. Rendering techniques between physical rendering. In PBR rendering, the consideration of physical laws such as energy conservation, Fresnel reflection law, light absorption phenomenon, etc. is added to better represent the details and roughness of the object surface, anisotropy, distinguish between metal and non-metal materials, and translucent materials and other complex material properties.

It can be seen that in the field of graphics, the analysis of object materials is becoming more and more important, and the rendering technology is becoming more and more refined. In addition, the collected images may have some noise, and the existing methods cannot remove the influence of noise and cannot guarantee the final Determines the accuracy of the material information.

In order to solve the above problems, the present application provides an image material determination method and related device, which can determine the material data of an object in an image based on a single image, without the need for an additional camera to collect multiple images or other parameters as additional input, and adopts dual The branched neural network model eliminates the influence of noise, which can greatly improve the efficiency and accuracy of determining material data.

First, the system architecture of an image material determination method provided by an embodiment of the present application will be described with reference to FIG. 1 . FIG. 1 is a system architecture diagram of an image material determination method provided by an embodiment of the present application. The system architecture 100 includes a collection device. 110 and a material determination device 120, wherein the above-mentioned acquisition device 110 can be a device that collects images, such as a camera, a laser device, etc., used to collect images of objects, and the above-mentioned acquisition device 110 can be wired or wirelessly connected to the material determination device 120, to the above-mentioned material. The determination device 120 sends the captured image, or the aforementioned capture device 110 may upload the image to the network without connecting to the material determination device 120 . The above-mentioned material determination device 120 may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, and various forms of user equipment (User Equipment, UE), Mobile station (Mobile Station, MS), terminal device (terminal device), server and so on.

In a possible embodiment, the above-mentioned material determination device 120 may be equipped with a material estimation model or a related material estimation algorithm to acquire images from the acquisition device 110 or other channels (such as network platforms, etc.), and perform the processing through the material estimation model. Corresponding processing, the material data is obtained.

It can be understood that no matter how many images are collected by the collection device 110, the material determination device 120 will perform the flow of the material determination method based on a single image.

It can be seen that through the above system architecture, the material data can be obtained only from a single image without the need for detailed sampling through a large number of cameras, which greatly improves the efficiency and accuracy of material estimation.

After understanding the system architecture of the present application, a method for determining an image material in the implementation of the present application is described below with reference to FIG. 2 . FIG. 2 is a schematic flowchart of a method for determining an image material provided in an embodiment of the present application, which specifically includes the following step:

Step 201: Determine target feature data of the target image.

The target image may be an image containing a target object, and the target feature data may be a feature vector reflecting the target image.

In a possible embodiment, preprocessing before feature extraction may be performed on the target image first, and the preprocessing steps may include grayscale, color space standardization, etc., and then preliminary feature extraction may be performed. The steps of feature extraction processing may include determining the gradient of each pixel, dividing the image into multiple unit areas, counting the gradient histogram of each unit area, obtaining target feature data according to the histogram, etc. The preprocessing and feature extraction processing are not specifically limited here.

In a possible embodiment, the target image can be input into the feature extraction module of the material estimation model, and the target feature data can be obtained according to the output of the feature extraction module. It can be understood that the above-mentioned feature extraction module can first perform regularization processing on the target image. , for example, a color image can be parsed into three channels of R (red) G (green) B (blue), where each value is between 0 and 255. After the target image is regularized, it can be processed by convolution The module performs the convolution operation and obtains the target feature data. The number of convolution layers in the convolution module is proportional to the complexity of the target feature data. For example, the target feature data can be a 128-dimensional feature vector or a 64-dimensional feature vector, which is not specifically limited here, and the convolution module in the feature extraction module can be flexibly adjusted according to requirements to adjust the final target. The dimension of the feature data.

It can be seen that by determining the target feature data of the target image, the target feature number can be extracted based on the document target image to provide data reference for subsequent material estimation.

Step 202: Determine highlight feature data and standard feature data according to the target feature data.

Wherein, the highlight feature data is used to indicate the image features whose influence of saturated pixels is eliminated, and the standard feature data is used to indicate the image features whose influences of saturated pixels are not eliminated.

Wherein, the target feature data can be input into the highlight branch module of the material estimation model, and the highlight feature data can be determined according to the output of the highlight branch module; and, the target feature data can be input into the material estimation model. The standard branch module determines the standard feature data according to the output of the standard branch module.

In a possible embodiment, the highlight branch module includes a first highlight convolution module, a first hole convolution module and a first standard convolution module, and the target feature data can be input into the first highlight convolution module , obtain the first highlight convolution data, the second highlight convolution data and the third highlight convolution data according to the output of the first highlight convolution module; combine the second highlight convolution data and the third highlight volume The product data is input into the first hole convolution module, and the fourth highlight convolution data is obtained according to the output of the first hole convolution module; the first highlight convolution data and the fourth highlight convolution data are input The first standard convolution module obtains the highlight feature data according to the output of the first standard convolution module.

In a possible embodiment, the standard branch module includes a second standard convolution module, a second hole convolution module and a third standard convolution module, and the target feature data can be input into the second standard convolution module , obtain the first standard convolution data, the second standard convolution data and the third standard convolution data according to the output of the second standard convolution module; combine the second standard convolution data and the third standard convolution data The product data is input into the second hole convolution module, and the fourth standard convolution data is obtained according to the output of the second hole convolution module; the first standard convolution data and the fourth standard convolution data are input The third standard convolution module obtains the standard feature data according to the output of the third standard convolution module.

It can be seen that determining the highlight feature data and the standard feature data according to the target feature data can exclude the influence of saturated pixels in the image, avoid extracting unnecessary features, and improve the accuracy of material estimation.

Step 203: Determine fusion feature data according to the highlight feature data and the standard feature data.

The highlight feature data and the standard feature data can be input into a feature fusion module of a material estimation model, and the fusion feature data can be obtained according to the output of the feature fusion module.

In a possible embodiment, the feature fusion module includes a feature connection module, a self-attention feature selection module, and a fusion convolution module. The highlight feature data and the standard feature data can be input into the feature connection module, and feature connection data can be obtained according to the output of the feature connection module; the feature connection data can be input into the self-attention feature selection module, according to The output of the self-attention feature selection module obtains feature weight data; the feature weight data is input into the fusion convolution module, and the fusion feature data is obtained according to the output of the fusion convolution module.

It can be seen that the fusion feature data is determined according to the highlight feature data and the standard feature data, and the highlight feature data and the standard feature data can be fused, so that the features describing the target image are increased, and the information under each feature does not increase. Improve the accuracy of material estimation while eliminating the interference of saturated pixels.

Step 204: Determine material data of the target image according to the fusion feature data.

The material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map, and a normal material map.

It can be seen that, through the above-mentioned image material determination method, first, first, the target feature data of the target image is determined; then, according to the target feature data, highlight feature data and standard feature data are determined, and the highlight feature data is used to indicate the elimination of the influence of saturated pixels. image features, the standard feature data is used to indicate the image features that do not eliminate the influence of the saturated pixels; then, the fusion feature data is determined according to the highlight feature data and the standard feature data; finally, according to the fusion feature data Determine the material data of the target image, the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map. There is no need to collect too many input target images, and only a single target image is needed to obtain accurate material data, which greatly improves the efficiency and accuracy of material estimation.

For ease of understanding, the material estimation model in the embodiment of the present application is described below. As shown in FIG. 3A , FIG. 3A is a schematic diagram of the architecture of a material estimation model provided by the embodiment of the present application. The material estimation model 300 includes a feature extraction module. 310, the highlight branch module 320, the standard branch module 330 and the feature fusion module 340, wherein, the target image can be used as input data, firstly input the feature extraction module 310, then, the target feature data output by the feature extraction module 310 are respectively input into the highlight The branch module 320 and the standard branch module 330, the highlight feature data output by the highlight branch module 320 and the standard feature data output by the standard branch module 330 are input into the feature fusion module 340 together, and the target is determined according to the output of the feature fusion module 340 The material data of the image, can finally determine the diffuse material map, the roughness material map, the specular material map and the normal material map respectively from four independent prediction branches.

Specifically, as shown in FIG. 3B , which is a schematic diagram of the architecture of another material estimation model provided by an embodiment of the present application, the above-mentioned feature extraction module 310 may include at least one convolution module 311 for feature extraction, and the above-mentioned highlight branch The module 320 may include a first highlight convolution module 321, a first hole convolution module 322 and a first standard convolution module 323, and the above-mentioned standard branch module 330 may include a second standard convolution module 331, a second hole convolution Module 332 and a third standard convolution module 333 , the above-mentioned feature fusion module 340 may include a feature connection module 341 , a self-attention feature selection module 342 and a fusion convolution module 343 . The first highlight convolution module 321 may include at least one convolution unit for highlight feature extraction, the first hole convolution module 322 may include at least one hole convolution unit, and the first standard convolution module 323 may include at least one hole convolution unit. a convolution unit for standard feature extraction, the above-mentioned second standard convolution module 331 may include at least one convolution unit for standard feature extraction, the above-mentioned second hole convolution module 332 may include at least one hole convolution unit, The above-mentioned third standard convolution module 333 may include at least one convolution unit for standard feature extraction.

Further, in a possible embodiment, the material estimation model may be as shown in FIG. 3C , which is a schematic diagram of the architecture of another material estimation model provided by this embodiment of the present application, including a convolution used for target feature data extraction. Unit a, highlight convolution unit h1, highlight convolution unit h2, highlight convolution unit h3, highlight convolution unit h4, highlight convolution unit h5, highlight convolution unit h6, and hole convolution unit for highlight feature data extraction h7, standard convolution unit h8, standard convolution unit h9, standard convolution unit s1, standard convolution unit s2, standard convolution unit s3, standard convolution unit s4, standard convolution unit s5 for standard feature data extraction , standard convolution unit s6, hole convolution unit s7, standard convolution unit s8, standard convolution unit s9, used to determine the feature connection unit of the fusion feature data, the self-attention feature selection module (Attention-based Feature Selection Module, AFS), fusion convolution unit f1, fusion convolution unit f2, fusion convolution unit f3, fusion convolution unit f4.

Among them, the highlight convolution unit h1 to the highlight convolution unit h6 can be understood as the first highlight convolution module, the hole convolution unit h7 can be understood as the first hole convolution module, the standard convolution unit h8 and the standard convolution unit h9 can be It is understood as the first standard convolution module; the standard convolution unit s1 to the standard convolution unit s6 can be understood as the second standard convolution module, the hole convolution unit s7 can be understood as the second hole convolution module, and the standard convolution unit s8 And the standard convolution unit s9 can be understood as the second standard convolution module; the fusion convolution unit f1 to the fusion convolution unit f4 can be understood as the fusion convolution module.

Specifically, the target image is input to the convolutional layer a, the convolutional layer a outputs the target feature data, and the target feature data is transferred to the highlight convolution unit h1 and the standard convolution unit s1 respectively.

Among them, the highlight convolution unit h1 to the highlight convolution unit h4 perform convolution operations in sequence, and the highlight convolution unit h4 can transfer the output data from the highlight convolution unit h3 to the standard convolution unit h8. At the same time, the highlight convolution unit h4 The convolution operation is performed sequentially to the highlight convolution unit h6. The highlight convolution unit h6 can transfer the output data from the highlight convolution unit h5 and its own output data to the hole convolution unit h7, and the hole convolution unit h7 can be connected from the highlight convolution unit h7. The output data of the convolution unit h5 and the output data from the highlight convolution unit h6, the standard convolution unit h8 can connect the output data from the highlight convolution unit h3 and the output data of the hole convolution unit h7, and the standard convolution unit h9 can Perform convolution operation and output highlight feature data.

Among them, the standard convolution unit s1 to the standard convolution unit s4 perform convolution operations in sequence, and the standard convolution unit s4 can transmit the output data of the AIDS standard convolution unit s3 to the standard convolution unit s8. At the same time, the standard convolution unit s4 To the standard convolution unit s4 to perform convolution operations in turn, the standard convolution unit s6 can transfer the output data from the standard convolution unit s5 and its own output data to the hole convolution unit s7, and the hole convolution unit s7 can be connected from the standard convolution unit s7. The output data of the convolution unit s5 and the output data from the standard convolution unit s6, the standard convolution unit s8 can connect the output data from the standard convolution unit s3 and the output data of the hole convolution unit s7, and the standard convolution unit s9 can Perform convolution operation and output standard feature data.

Among them, the highlight feature data and standard feature data are input into the feature connection unit for merging, and the feature connection unit outputs the merged data, and transmits the merged data to the self-attention feature selection AFS module, and then passes through the fusion convolution unit f1 in turn , fusion convolution unit f2, fusion convolution unit f3, fusion convolution unit f4 to obtain fusion feature data. Finally, the diffuse material map, the roughness material map, the specular material map and the normal material map are output from the four prediction branches based on the fusion feature data.

It should be noted that each highlight convolution unit can include multiple convolution branch operations, including normalization (IN, etc.) processing, activation function (sigmod, etc.) processing, advanced activation function (Leaky ReLU, etc.) processing, For the inception branch operation that preserves global and local information, the highlight area in the target image can be repaired through the highlight branch module, but it will produce blurred normals and biased specular components, which need to be combined with the standard branch module. Perfect fusion feature data, and each standard convolution unit can use the existing convolution operation method, which will not be repeated here. While the self-attention feature selection module can include pooling layers and multiple convolutional layers, the self-attention feature selection module can assign high weights to informative features and suppress the expression of useless features.

It can be understood that the feature connection unit is often used to combine features, fuse features extracted by multiple convolutional feature extraction frameworks, or fuse information from the output layer. That is to say, the features describing the target image itself are increased, but the information under each feature is not increased.

It can be seen that through the material estimation model of the above-mentioned dual-stream architecture, the influence of saturated pixels can be excluded, and there is no need to collect too many input target images, and only a single target image can be used to obtain accurate material data, which greatly improves the performance of materials. Estimated efficiency and accuracy.

Another image material determination method provided by an embodiment of the present application will be described below with reference to FIG. 4 . FIG. 4 is a schematic flowchart of another image material determination method provided by an embodiment of the present application, which specifically includes the following steps:

Step 401: Input the training image into a preset model, and obtain estimated material data according to the output of the preset model.

Wherein, the estimated material data includes an estimated diffuse reflection texture map, an estimated roughness texture map, an estimated specular texture map, and an estimated normal texture map. The above-mentioned training images can be from the training database, which are already labeled images. The above-mentioned preset model may be a model that has not completed training, and its architecture is consistent with the material model architecture in the embodiment of the present application.

Step 402: Process the estimated diffuse reflection texture map, the estimated roughness texture map, the estimated specular texture map, and the estimated normal texture map through a first loss function to obtain a first difference data.

Among them, the pixel loss of the first loss function can be used to punish the difference in the pixel space. For example, the map loss and the rendering loss can be used to estimate the roughness material map, the pre- The estimated specular texture map and the estimated normal texture map are processed to obtain the first difference data, which will not be repeated here.

Step 403: Process the estimated diffuse reflection texture map and the estimated normal texture map through a second loss function to obtain second difference data.

The second loss function may include global D and local D. Add global D and local D to the estimated diffuse material map and the estimated normal material map respectively to form an adversarial loss function and obtain second difference data.

It can be seen that because the first loss function usually produces blurred textures and lacks high-frequency details, especially the final estimated diffuse material map and estimated normal material map will be blurred. In order to prevent this situation , the estimated diffuse material map and the estimated normal material map can be specially processed through the second loss function to improve the accuracy of the diffuse material map and the normal material map finally determined by the material estimation model.

Step 404: Adjust the preset model according to the first difference data and the second difference data to obtain a material estimation model.

It can be seen that the material estimation model obtained by the above training method can eliminate the influence of saturated pixels, and there is no need to collect too many input target images, and only a single target image can be used to obtain accurate material data, which greatly improves the performance of materials. Estimated efficiency and accuracy.

Step 405: Determine target feature data of the target image.

Step 406: Determine highlight feature data and standard feature data according to the target feature data.

Step 407: Determine fusion feature data according to the highlight feature data and the standard feature data.

Step 408: Determine material data of the target image according to the fusion feature data.

It can be seen that, through the above-mentioned image material determination method, first, the target feature data of the target image is determined; then, according to the target feature data, highlight feature data and standard feature data are determined, and the highlight feature data is used to indicate an image that eliminates the influence of saturated pixels. feature, the standard feature data is used to indicate the image features that do not eliminate the influence of the saturated pixels; then, the fusion feature data is determined according to the highlight feature data and the standard feature data; finally, the fusion feature data is determined according to the fusion feature data. The material data of the target image includes at least one of a diffuse reflection material map, a roughness material map, a specular material map, and a normal material map. There is no need to collect too many input target images, and only a single target image is needed to obtain accurate material data, which greatly improves the efficiency and accuracy of material estimation.

An electronic device in an embodiment of the present application will be described below with reference to FIG. 5, which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 5, the electronic device 500 includes a processor 501, a communication The interface 502 and the memory 503, the processor, the communication interface and the memory are connected to each other, wherein the electronic device 500 may further include a bus 504, and the processor 501, the communication interface 502 and the memory 503 may be connected to each other through the bus 504, and the bus 504 It may be a Peripheral Component Interconnect (PCI for short) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA for short) bus or the like. The bus 504 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 5, but it does not mean that there is only one bus or one type of bus. The memory 503 is used to store a computer program, the computer program includes program instructions, and the processor is configured to invoke the program instructions to execute all or part of the methods described in FIG. 2 and FIG. 4 .

The foregoing mainly introduces the solutions of the embodiments of the present application from the perspective of the method-side execution process. It can be understood that, in order to realize the above-mentioned functions, the electronic device includes corresponding hardware structures and/or software modules for executing each function. Those skilled in the art should easily realize that the present application can be implemented in hardware or in the form of a combination of hardware and computer software, in combination with the units and algorithm steps of each example described in the embodiments provided herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In this embodiment of the present application, the electronic device may be divided into functional units according to the foregoing method examples. For example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical function division, and other division methods may be used in actual implementation.

In the case where each function module is divided according to each function, an image material determination apparatus in an embodiment of the present application will be described in detail below with reference to FIG. 6 , and FIG. 6 is a function of an image material determination apparatus provided by an embodiment of the application. Unit composition block diagram, the image material determination device 600 includes:

Feature determination unit 610, determines the target feature data of the target image in English;

The branch extraction unit 620 is configured to determine highlight feature data and standard feature data according to the target feature data, where the highlight feature data is used to indicate that the image features affected by saturated pixels are eliminated, and the standard feature data is used to indicate that the Image characteristics affected by saturated pixels;

a fusion extraction unit 630, configured to determine fusion characteristic data according to the highlight characteristic data and the standard characteristic data;

A material estimation unit 640, configured to determine material data of the target image according to the fusion feature data, where the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map .

It can be seen that, through the above-mentioned image material determination method and related device, first, the target feature data of the target image is determined; then, according to the target feature data, highlight feature data and standard feature data are determined, and the highlight feature data is used to indicate the elimination of saturated pixels. Influenced image features, the standard feature data is used to indicate the image features that are not affected by the saturated pixels; then, fusion feature data is determined according to the highlight feature data and the standard feature data; finally, according to the fusion feature The data determines material data of the target image, and the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map, and a normal material map. There is no need to collect too many input target images, and only a single target image is needed to obtain accurate material data, which greatly improves the efficiency and accuracy of material estimation.

In the case of using an integrated unit, another image material determination apparatus 700 in this embodiment of the present application will be described in detail below with reference to FIG. 7 . The image material determination apparatus 700 includes a processing unit 701 and a communication unit 702 , wherein the The processing unit 701 is configured to perform any step in the above method embodiments, and when performing data transmission such as sending, the communication unit 702 can be selectively invoked to complete corresponding operations.

Wherein, the image material determination apparatus 700 may further include a storage unit 703 for storing program codes and data. The processing unit 701 may be a processor, the communication unit 702 may be a wireless communication module, and the storage unit 703 may be a memory.

The processing unit 701 is specifically used for:

Determine the target feature data of the target image;

Determine highlight feature data and standard feature data according to the target feature data, where the highlight feature data is used to indicate an image feature that eliminates the influence of saturated pixels, and the standard feature data is used to indicate an image feature that does not eliminate the influence of the saturated pixel;

Determine fusion feature data according to the highlight feature data and the standard feature data;

The material data of the target image is determined according to the fusion feature data, and the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map.

It can be seen that, through the above-mentioned image material determination method and related device, first, the target feature data of the target image is determined; then, according to the target feature data, highlight feature data and standard feature data are determined, and the highlight feature data is used to indicate the elimination of saturated pixels. Influenced image features, the standard feature data is used to indicate the image features that are not affected by the saturated pixels; then, fusion feature data is determined according to the highlight feature data and the standard feature data; finally, according to the fusion feature The data determines material data of the target image, and the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map, and a normal material map. There is no need to collect too many input target images, and only a single target image is needed to obtain accurate material data, which greatly improves the efficiency and accuracy of material estimation.

Embodiments of the present application further provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes the computer to execute part or all of the steps of any method described in the above method embodiments .

Embodiments of the present application further provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of the method embodiments described above. some or all of the steps of the method. The computer program product may be a software installation package, and the computer includes an electronic device.

It should be noted that, for the sake of simple description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Because in accordance with the present application, certain steps may be performed in other orders or concurrently. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The above-mentioned integrated units, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art, or all or part of the technical solution, and the computer software product is stored in a memory, Several instructions are included to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods in the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash disk , Read-only memory (English: Read-Only Memory, referred to as: ROM), random access device (English: Random Access Memory, referred to as: RAM), magnetic disk or optical disk, etc.

The embodiments of the present application have been introduced in detail above, and the principles and implementations of the present application are described in this paper by using specific examples. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, based on the idea of the present application, will have changes in the specific implementation manner and application scope. In summary, the contents of this specification should not be construed as limitations on the present application.

Claims (10)

1. A method for determining an image material, wherein the method comprises: Determine the target feature data of the target image; Determine highlight feature data and standard feature data according to the target feature data, where the highlight feature data is used to indicate an image feature that eliminates the influence of saturated pixels, and the standard feature data is used to indicate an image feature that does not eliminate the influence of the saturated pixel; Determine fusion feature data according to the highlight feature data and the standard feature data; The material data of the target image is determined according to the fusion feature data, and the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map. 2. The method according to claim 1, wherein the determining of the highlight feature data and the standard feature data according to the target feature data comprises: Inputting the target feature data into the highlight branch module of the material estimation model, and determining the highlight feature data according to the output of the highlight branch module; and, The target feature data is input into the standard branch module of the material estimation model, and the standard feature data is determined according to the output of the standard branch module. 3. The method according to claim 2, wherein the highlight branch module comprises a first highlight convolution module, a first hole convolution module and a first standard convolution module; The data is input to the highlight branch module of the material estimation model, and the highlight feature data is determined according to the output of the highlight branch module, including: Inputting the target feature data into the first highlight convolution module, and obtaining the first highlight convolution data, the second highlight convolution data and the third highlight convolution data according to the output of the first highlight convolution module; Inputting the second highlight convolution data and the third highlight convolution data into the first hole convolution module, and obtaining fourth highlight convolution data according to the output of the first hole convolution module; The first highlight convolution data and the fourth highlight convolution data are input into the first standard convolution module, and the highlight feature data is obtained according to the output of the first standard convolution module. 4. The method according to claim 2, wherein the standard branch module comprises a second standard convolution module, a second hole convolution module and a third standard convolution module; The data is input into the standard branch module of the material estimation model, and the standard feature data is determined according to the output of the standard branch module, including: Inputting the target feature data into the second standard convolution module, and obtaining the first standard convolution data, the second standard convolution data and the third standard convolution data according to the output of the second standard convolution module; The second standard convolution data and the third standard convolution data are input into the second hole convolution module, and the fourth standard convolution data is obtained according to the output of the second hole convolution module; The first standard convolution data and the fourth standard convolution data are input into the third standard convolution module, and the standard feature data is obtained according to the output of the third standard convolution module. 5. The method according to claim 1, wherein the determining of fusion feature data according to the highlight feature data and the standard feature data comprises: Input the highlight feature data and the standard feature data into the feature fusion module of the material estimation model, and obtain the fusion feature data according to the output of the feature fusion module. 6. The method according to claim 5, wherein the feature fusion module comprises a feature connection module, a self-attention feature selection module and a fusion convolution module; The feature data is input into the feature fusion module of the material estimation model, and the fusion feature data is obtained according to the output of the feature fusion module, including: Input the highlight feature data and the standard feature data into the feature connection module, and obtain the feature connection data according to the output of the feature connection module; Input the feature connection data into the self-attention feature selection module, and obtain feature weight data according to the output of the self-attention feature selection module; The feature weight data is input into the fusion convolution module, and the fusion feature data is obtained according to the output of the fusion convolution module. 7. The method according to claim 1, characterized in that, before said determining the target feature data of the target image, the method further comprises: The training image is input into a preset model, and estimated material data is obtained according to the output of the preset model, and the estimated material data includes an estimated diffuse reflection texture map, an estimated roughness texture map, an estimated specular texture map, and a predicted texture map. Estimate normal material map; The estimated diffuse reflection texture map, the estimated roughness texture map, the estimated specular texture map, and the estimated normal texture map are processed by the first loss function to obtain first difference data; The estimated diffuse reflection texture map and the estimated normal texture map are processed through a second loss function to obtain second difference data; The preset model is adjusted according to the first difference data and the second difference data to obtain a material estimation model. 8. An image material determination device, wherein the device comprises: A feature determination unit, which determines the target feature data of the target image in English; A branch extraction unit, configured to determine highlight feature data and standard feature data according to the target feature data, where the highlight feature data is used to indicate the image features that eliminate the influence of saturated pixels, and the standard feature data is used to indicate that the saturation is not eliminated Pixel-influenced image features; a fusion extraction unit, configured to determine fusion characteristic data according to the highlight characteristic data and the standard characteristic data; A material estimation unit, configured to determine material data of the target image according to the fusion feature data, where the material data includes at least one of a diffuse reflection material map, a roughness material map, a specular material map and a normal material map. 9. An electronic device, characterized in that it comprises a baseband chip, the baseband chip comprising a digital signal processor and an accumulator, and the baseband chip is used to execute the process as claimed in the claims through the digital signal processor and the accumulator Instructions for steps in the method of any one of 1-8. 10. A computer storage medium, wherein the computer storage medium stores a computer program, and the computer program includes program instructions, the program instructions, when executed by a baseband chip, cause the baseband chip to execute as claimed in claim 1 The method of any one of -7.

Images