CN114926593A - SVBRDF material modeling method and system based on single highlight image - Google Patents

Abstract

The present disclosure provides an SVBRDF material modeling method and system based on a single highlight image, the solution belongs to the technical field of three-dimensional rendering materials, and the solution includes: acquiring a highlight image of the surface of an object material; Perform highlight removal based on dense feature fusion connection and highlight multi-level recognition to obtain a highlight-free image; input the highlight image and highlight-free image into a pre-trained generator network to obtain the spatially-varying bidirectional reflectance distribution function of the object surface , and then obtain the corresponding material map; wherein, the generator network includes a shared encoder and several decoders respectively connected to the shared encoder, the decoders are respectively corresponding to the diffuse reflection map, the normal map, the roughness map Processing of degree maps and reflection maps.

Description

SVBRDF material modeling method and system based on single specular image

technical field

The present disclosure belongs to the technical field of three-dimensional rendering materials, and in particular, relates to a SVBRDF material modeling method and system based on a single highlight image.

Background technique

The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

Estimating the reflective properties of materials from images has been an extensive research topic in the fields of computer graphics and vision. In the real world, the appearance of materials depends on viewing and lighting directions, which makes appearance estimation a challenging task. The surface reflectance property is usually represented by a bi-directional reflectance distribution function (BRDF). Since most surfaces are not completely homogeneous, the BRDF is usually given as a function of surface position, which is called spatial variation Bi-directional reflectance distribution function (spatially-varying bi-directional reflectance distribution function, SVBRDF), including diffuse albedo, specular albedo, surface normal, and roughness, corresponding to diffuse maps, reflection maps, normal maps and roughness Degree map Four kinds of material maps.

Different reflectances can produce the same observed image, so it is very difficult to reconstruct a high-quality spatially varying bidirectional reflectance distribution function from input images taken under different illumination and viewing directions. Usually, the apparent material modeling of an object requires the use of professional instruments to measure the reflection characteristics of the material surface of a specific object in the scene, or the use of professional tools to mark the material map with a lot of manual intervention. The inventor found that recently, the deep learning-based method has realized the reconstruction of various material maps from the captured material images in the material modeling work, which greatly simplifies the process of material modeling work, but due to the dynamic range of many consumer cameras. Limited, so some image areas will be polluted by specular highlights, which will cause obvious speckle artifacts in the generated feature map, resulting in the generated material map effect not meeting the actual needs.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present disclosure provides an SVBRDF material modeling method and system based on a single specular image. The solution automatically generates a material map close to the appearance of the real material based on a single specular image captured in the real world. .

According to a first aspect of the embodiments of the present disclosure, an SVBRDF material modeling method based on a single highlight image is provided, including:

Get the specular image of the material surface of the object;

Performing highlight removal on the highlight image by means of dense feature fusion connection and highlight multi-level recognition to obtain a highlight-free image;

Input the high-light image and the non-high-light image into the pre-trained generator network to obtain the spatially-varying bidirectional reflectance distribution function of the object surface, and then obtain the corresponding material map; wherein, the generator network includes a shared encoder and a separate A plurality of decoders connected with the shared encoder, the decoders respectively correspond to the processing of diffuse reflection maps, normal maps, roughness maps and reflection maps.

Further, the highlight removal is performed by a method based on dense feature fusion connection and highlight multi-level recognition, specifically: using an encoder-decoder group with dense feature fusion connection, using a multi-level highlight recognition strategy, to shoot images. Perform a highlight removal operation to obtain a highlight-free image of the captured image.

Further, the encoder comprises several convolutional layers, instance normalization layers, linear rectification units and maximum pooling layers connected in sequence; the decoder comprises several deconvolutional layers, instance normalization layers connected in sequence; It is composed of a uniform layer and a linear rectifier unit.

Further, based on the dense feature fusion connection, the encoder performs a cascade operation on the features extracted by the convolution block and the features extracted by the previous layer of convolution blocks, then performs an upsampling operation, and finally performs a convolution operation.

Further, the encoder of the generator network includes eight convolutional layers for downsampling, the first seven convolutional layers are connected to an instantiated normalization layer and a linear rectification unit, and the last convolutional layer is connected to an instantiated layer. down to one level.

Further, the generator network and the multi-discriminator network are co-trained, specifically:

Acquire a public data set, and realize the construction of a training data set by performing deduplication, random cropping and random screening on the public data set;

Based on a pre-built minimization loss function, the generator network and the multi-discriminator network are co-trained through the constructed training data set, wherein, through the multi-discriminator network and the pre-built loss function, the generated The material map generated by the filter network is used to judge the quality.

According to a second aspect of the embodiments of the present disclosure, an SVBRDF material modeling system based on a single specular image is provided, including:

A data acquisition unit, which is used to acquire a specular image of the surface of the object material;

a highlight removal unit, which is used to remove the highlights from the highlight image by means of dense feature fusion connection and highlight multi-level recognition to obtain a highlight-free image;

an object surface material estimation unit, which is used to input the high-light image and the non-high-light image into a pre-trained generator network to obtain the spatially-varying bidirectional reflectance distribution function of the object surface, and then obtain the corresponding material map; wherein, the The generator network includes a shared encoder and a plurality of decoders respectively connected to the shared encoder, and the decoders respectively correspond to the processing of diffuse maps, normal maps, roughness maps and reflection maps.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the memory, the processor implementing the single highlight-based highlighting when executing the program SVBRDF material modeling method for images.

According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, realizes the SVBRDF material based on a single highlight image modeling method.

Compared with the prior art, the beneficial effects of the present disclosure are:

(1) The solution described in this disclosure provides an SVBRDF material modeling method based on a single highlight image. The proposed generator is an encoder-decoder network with a shared encoder and four decoders. Four The decoder processes the diffuse map, normal map, roughness map and reflection map respectively, uses the generator network to generate the material map, uses the rendering module to render the generated map into a rendered image, and uses the multi-discriminator network to Generate textures and real textures, discriminate between rendered images and real images, and generate high-quality material appearance textures.

(2) The scheme uses a highlight removal module based on dense feature fusion connection and highlight multi-level recognition to reduce the impact of overexposed areas on material estimation in the SVBRDF estimation process, and can remove the highlight artifacts existing in the existing SVBRDF estimation technology. Phenomenon.

(3) The scheme proposes loss functions based on texture loss, rendering loss, and adversarial loss, and introduces feature matching loss to stabilize training.

Advantages of additional aspects of the disclosure will be set forth in part in the description that follows, and in part will become apparent from the description below, or will be learned by practice of the disclosure.

Description of drawings

The accompanying drawings that constitute a part of the present disclosure are used to provide further understanding of the present disclosure, and the exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure.

1 is a network structure diagram adopted by the SVBRDF material modeling method based on a single highlight image described in the embodiment of the present disclosure;

2 is a network structure diagram of a highlight removal module described in an embodiment of the present disclosure;

FIG. 3(a) is a structural diagram of the encoder of the highlight removal module described in the embodiment of the present disclosure;

FIG. 3(b) is a structural diagram of the decoder of the highlight removal module described in the embodiment of the present disclosure;

4 is a schematic diagram of the result of the highlight removal module described in the embodiment of the present disclosure;

Figure 5(a) is a structural diagram of the generator network encoder described in the embodiment of the present disclosure;

FIG. 5(b) is a structural diagram of the generator network decoder described in the embodiment of the present disclosure;

Fig. 6 is the network structure diagram of the discriminator described in the embodiment of the present disclosure;

FIG. 7(a) and FIG. 7(b) are comparison diagrams of the results of the SVBRDF material modeling method based on a single highlight image described in the embodiment of the present disclosure.

Detailed ways

The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, furthermore, it is to be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.

The embodiments of this disclosure and features of the embodiments may be combined with each other without conflict.

Example 1:

The purpose of this embodiment is to provide an SVBRDF material modeling method based on a single highlight image.

An SVBRDF material modeling method based on a single specular image, including:

Get the specular image of the material surface of the object;

Performing highlight removal on the highlight image by means of dense feature fusion connection and highlight multi-level recognition to obtain a highlight-free image;

Input the high-light image and the non-high-light image into the pre-trained generator network to obtain the spatially-varying bidirectional reflectance distribution function of the object surface, and then obtain the corresponding material map; wherein, the generator network includes a shared encoder and a separate A plurality of decoders connected with the shared encoder, the decoders respectively correspond to the processing of diffuse reflection maps, normal maps, roughness maps and reflection maps.

Further, the highlight removal is performed by a method based on dense feature fusion connection and highlight multi-level recognition, specifically: using an encoder-decoder group with dense feature fusion connection, using a multi-level highlight recognition strategy, to shoot images. Perform a highlight removal operation to obtain a highlight-free image of the captured image.

Further, the encoder comprises several convolutional layers, instance normalization layers, linear rectification units and maximum pooling layers connected in sequence; the decoder comprises several deconvolutional layers, instance normalization layers connected in sequence; It is composed of a uniform layer and a linear rectifier unit.

Further, based on the dense feature fusion connection, the encoder performs a cascade operation on the features extracted by the convolution block and the features extracted by the previous layer of convolution blocks, then performs an upsampling operation, and finally performs a convolution operation.

Further, the encoder of the generator network includes eight convolutional layers for downsampling, the first seven convolutional layers are connected to an instantiated normalization layer and a linear rectification unit, and the last convolutional layer is connected to an instantiated layer. down to one level.

Further, the generator network and the multi-discriminator network are co-trained, specifically:

Acquire a public data set, and realize the construction of a training data set by performing deduplication, random cropping and random screening on the public data set;

Based on a pre-built minimization loss function, the generator network and the multi-discriminator network are co-trained through the constructed training data set, wherein, through the multi-discriminator network and the pre-built loss function, the generated The material map generated by the filter network is used to judge the quality.

Further, the multi-discriminator network specifically includes a discriminator for processing diffuse reflection maps, a discriminator for processing normal maps, a discriminator for processing roughness maps, a discriminator for processing reflection maps, and a discriminator for processing rendering results.

Specifically, in order to facilitate understanding, the solution described in this embodiment is described in detail below with reference to the accompanying drawings:

This embodiment provides a SVBRDF material modeling method based on a single highlight image. The solution can be divided into four main stages: training data set construction stage, generator network and multi-discriminator network co-training stage, highlight removal Stage, object surface material estimation stage, the flow chart is shown in Figure 1, which includes the following steps:

Step 1. Deduplication, random cropping, and random selection of large-scale data sets to construct training data sets. By minimizing the loss function, the generator network and the multi-discriminator network are co-trained using the training data set, and the multi-discriminator is used to discriminate the material map generated by the generator, and the loss function is calculated according to the formula. The loss functions include texture loss, rendering loss and adversarial loss, where adversarial loss includes feature matching loss and standard adversarial loss.

S101: Collect public datasets and use the large-scale dataset proposed by Deschaintre et al. De-duplication is performed on the public dataset, and only 1 set of data is retained for training data generated from the same SVBRDF but with slightly different viewing or lighting directions, and finally 195,284 instances are collected.

S102: Crop the pictures in the data set, and randomly crop all the pictures in the training data set to 256x256.

S103: Randomly select 18K instances from the dataset as training datasets, and the remaining instances as test datasets.

S104: Calculate the adversarial loss.

特征匹配损失通过最小化估计值和真实值之间高维特征的距离来稳定训练。Calculate the feature matching loss using formula (1)

The feature matching loss stabilizes training by minimizing the distance of high-dimensional features between estimated and true values.

Among them, G represents the generator, I represents the input image, and M represents the material map generated by the generator. D _i refers to any discriminator in D _maps = (D _d , D _n , D _r , D _s ), each discriminator has the same structure, wherein D _d , D _n , D _r , D _s are used to deal with diffuse reflection respectively Maps, normal maps, roughness maps and reflection maps.

通过最小化标准对抗损失，生成器试图生成与参考贴图无法区分的贴图，而辨别器则试图有效区分生成贴图与参考贴图。Use Equation (2) to calculate the standard adversarial loss

By minimizing the standard adversarial loss, the generator tries to generate a texture that is indistinguishable from the reference texture, while the discriminator tries to effectively distinguish the generated texture from the reference texture.

Among them, D _render represents the rendering discriminator;

Using formula (3), the adversarial loss is calculated by combining the feature matching loss and the standard adversarial loss

贴图损失反映了生成贴图和参考贴图的逐像素差异。S105: Calculate the texture loss using formula (4)

The texture loss reflects the pixel-by-pixel difference between the generated texture and the reference texture.

表示输入图像I对应的参考材质贴图。in,

Indicates the reference texture map corresponding to the input image I.

渲染损失反映了真实图像和渲染图像的逐像素差异。S106: Draw the diffuse reflection map, normal map, roughness map and reflection map generated by the generator into a rendered image, and use formula (5) to calculate the rendering loss

The rendering loss reflects the pixel-by-pixel difference between the real image and the rendered image.

Among them, R( ) represents the rendering module. The rendering module uses the Cook-Torrance BRDF model, the GGX microsurface normal distribution function.

S107: Combine the adversarial loss, texture loss, rendering loss, and weighted summation to calculate the total loss function:

Among them, λ _map , λ _render , and λ _adv are weights for balancing each loss item. In this embodiment, λ _map =10, λ _render =5, and λ _adv =1.

S108: Using the training data set, perform co-training on the generator network and the multi-discriminator network according to formula (7).

Among them, N is the number of samples in the training set, θ is the generator parameter set, and the network structure is shown in Figure (1).

In this example, the PyTorch framework, the Adam optimizer, is used, and the initial learning rate is 0.00002. Train 1,000,000 times. The graphics processor is NVIDIA TITAN RTX, and the video memory is 12GB.

Step 2. Use the captured image of the material surface of the object, and input it into the highlight removal module for multi-level highlight recognition and highlight removal to obtain a non-highlight image, and input the captured image and the non-highlight image into the trained generator network obtained in step 1. In , the surface material of the object is estimated, and the diffuse map, normal map, roughness map and reflection map of the material are estimated.

S201: Acquire a photographed image of the material surface of the object.

S202: Based on the highlight elimination module, perform multi-level highlight identification and elimination processing on the captured image to obtain a highlight-free image.

The network structure of the highlight removal module is shown in Figure 2. Using an encoder-decoder group with dense feature fusion connections, a multi-level highlight recognition strategy is used to perform highlight removal operations on the captured image to obtain a non-highlight image of the captured image. The recognition and highlight removal results are shown in Figure 4.

Using the captured image I as input, the highlight feature F is extracted using the highlight removal module encoder and the highlight removal module decoder.

Based on the highlight feature F, predict the highlight mask M, where M represents the visible highlight position

The specular density S is predicted based on the specular feature F and the specular mask M.

According to formula (8), based on M, S and I, the image without highlight D is calculated

表示对应位置元素相乘。in,

Indicates that the corresponding position elements are multiplied.

Figure 3(a) shows the encoder structure of the highlight removal module. The encoder of the highlight removal module consists of five convolutional blocks consisting of convolutional layers, instance normalization layers, linear rectification units, and max-pooling layers. The convolutional layer is used to extract image features, the instance normalization layer is used to receive the output of the convolutional layer and normalize it, the linear rectification unit is used to map the output of the instance normalization layer, and the max pooling layer is used to reduce the volume The parameter error of the stacked layer causes the error of the offset of the estimated mean, retains more texture information, and the maximum pooling window size is 2.

Figure 3(b) shows the decoder structure of the highlight removal module. The decoder of the highlight removal module consists of five deconvolution blocks consisting of a deconvolution layer, an instance normalization layer, and a linear rectification unit. The deconvolution layer is used to expand the dimension of the feature vector, the instance normalization layer is used to receive the output of the deconvolution layer and normalize it, and the linear rectification unit is used to map the output of the instance normalization layer.

In the encoder of the highlight removal module, the dense feature fusion connection is used, and the dense feature fusion strategy is used to express the features extracted by the convolution block and the features extracted by the previous convolution block. The cascade operation is performed, then the upsampling operation is performed, and finally the convolution operation is performed. Normally, the number of channels increases with the number of layers. In order to avoid high-level information dominating the fused features, a convolution operation is introduced to reduce the number of channels to a fixed value, so that each feature will be used in the fusion. The latter features contribute the same number of channels.

表示卷积块i抽取的特征，up表示上采样操作，cat表示级联操作，conv表示卷积操作。in,

Represents the features extracted by the convolution block i, up represents the upsampling operation, cat represents the cascade operation, and conv represents the convolution operation.

S203: Using the trained generator network, process the illuminated image and the non-highlight image to generate an SVBRDF map of the material corresponding to the input image.

Figure 5(a) shows the encoder structure of the generator network. The encoder of the generator network has 8 convolutional layers for downsampling, the first 7 convolutional layers are followed by an instantiated normalization layer and a linear rectification unit, and the last convolutional layer is followed by an instantiated normalization layer. Figure 5(b) shows the decoder structure of the generator network. The four decoders of the generator network have the same structure, each decoder network has 8 deconvolution layers for upsampling, the first 7 deconvolution layers are followed by an instantiated normalization layer and a linear rectification unit, and the last The deconvolution layer follows an instantiated normalization layer. The non-highlight image generated by the highlight removal module is sent to the encoder to extract features, and the extracted diffuse reflection features are sent to the decoder De _d to generate the diffuse reflection map. The captured image is sent to the encoder to extract features, and the extracted features are input to the decoders _Den , _Der and _Des to generate normal map, roughness map and reflection map respectively.

The discriminators D _d , D _n , D _r , D _s , and D _render have the same structure. Figure 6 shows the structure of the discriminator network. The discriminator network consists of cascade operations and three convolutional blocks consisting of convolutional layers, instance normalization layers, linear rectification units, and max-pooling layers. The cascade operation splices the generated texture with the real texture, the convolutional layer is used to extract image features, the instance normalization layer is used to receive the output of the convolutional layer and normalize it, and the linear rectification unit is used to map the instance normalization layer. output.

Compared with the prior art, in this embodiment, the quality of the generated material map and the re-rendering result are significantly improved in terms of synthetic data and real data. In this embodiment, the re-rendering root mean square error RMSE is reduced to 0.079, the normal map error is reduced to 0.042, the normal map error is reduced to 0.062, the roughness map error is reduced to 0.102, and the reflection map error is reduced to 0.062. There are technical issues with specular artifacts.

The present disclosure proposes a method for single image SVBRDF estimation. The method proposes a highlight removal module with multi-level highlight recognition and dense feature fusion connections. An encoder-decoder network with a shared encoder and four decoders is used as the generator network, and a set of discriminators are used to train the generator network to distinguish generated maps from real maps, and rendered images from real images. Extensive experiments on synthetic and real image datasets show that compared to the state-of-the-art, high-quality SVBRDFs can be generated and more realistic and believable rendering results can be produced for input images with a large number of highlight regions.

Embodiment 2:

The purpose of this embodiment is to provide an SVBRDF material modeling system based on a single highlight image.

An SVBRDF material modeling system based on a single specular image, including:

A data acquisition unit, which is used to acquire a specular image of the surface of the object material;

a highlight removal unit, which is used to remove the highlights from the highlight image by means of dense feature fusion connection and highlight multi-level recognition to obtain a highlight-free image;

an object surface material estimation unit, which is used to input the high-light image and the non-high-light image into a pre-trained generator network to obtain the spatially-varying bidirectional reflectance distribution function of the object surface, and then obtain the corresponding material map; wherein, the The generator network includes a shared encoder and a plurality of decoders respectively connected to the shared encoder, and the decoders respectively correspond to the processing of diffuse maps, normal maps, roughness maps and reflection maps.

It should be noted here that each module in this embodiment corresponds to each step in Embodiment 1 one-to-one, and the technical details of the system in this embodiment have been described in detail in Embodiment 1, so they are not repeated here. Repeat.

In further embodiments, there is also provided:

An electronic device includes a memory, a processor, and computer instructions stored on the memory and executed on the processor, and when the computer instructions are executed by the processor, the method described in the first embodiment is completed. For brevity, details are not repeated here.

It should be understood that, in this embodiment, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors, DSPs, application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices. , discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory may include read-only memory and random access memory and provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in the first embodiment is completed.

The method in the first embodiment can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware. To avoid repetition, detailed description is omitted here.

Those of ordinary skill in the art can realize that the unit, that is, the algorithm step of each example described in conjunction with this embodiment, can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this disclosure.

The SVBRDF material modeling method and system based on a single highlight image provided by the above embodiments can be implemented and have broad application prospects.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims (10)

1. A single highlight image-based SVBRDF material modeling method is characterized by comprising the following steps:

acquiring a highlight image of the surface of the object material;

highlight elimination is carried out on the highlight image in a mode based on dense feature fusion connection and highlight multilevel identification, and a highlight-free image is obtained;

inputting the highlight images and the non-highlight images into a generator network trained in advance to obtain a spatial variation bidirectional reflectivity distribution function of the surface of the object so as to obtain corresponding material maps; the generator network comprises a shared encoder and a plurality of decoders respectively connected with the shared encoder, wherein the decoders respectively correspond to the processing of the diffuse reflection map, the normal map, the roughness map and the reflection map.

2. The single highlight image-based SVBRDF material modeling method of claim 1, wherein said highlight elimination is performed by a highlight-based dense feature fusion connection and highlight multilevel identification method, specifically: and performing highlight removing operation on the shot image by using an encoder-decoder group with dense feature fusion connection and adopting a multi-level highlight identification strategy to obtain a highlight-free image of the shot image.

3. The single highlight image-based SVBRDF material modeling method of claim 2, wherein said encoder comprises a plurality of convolutional layers, instance normalization layers, linear rectification units and max-pooling layers connected in sequence; the decoder comprises a plurality of deconvolution layers, an example normalization layer and a linear rectification unit which are sequentially connected.

4. The single highlight image-based SVBRDF material modeling method of claim 2, wherein said encoder performs a cascade operation of the extracted features of the convolution block and the extracted features of the previous layer of convolution block based on dense feature fusion connection, then performs an upsampling operation, and finally performs a convolution operation.

5. The single highlight image-based SVBRDF material modeling method of claim 1, wherein said generator network encoder comprises eight convolutional layers for downsampling, the first seven convolutional layers are followed by an instantiated one-layer and linear rectification unit, and the last convolutional layer is followed by an instantiated one-layer.

6. The single highlight image-based SVBRDF material modeling method of claim 1, wherein said generator network and multi-discriminator network are trained in coordination, specifically:

acquiring a public data set, and implementing construction of a training data set by carrying out duplicate removal, random cutting and random screening on the public data set;

and performing collaborative training on the generator network and the multi-discriminator network through a constructed training data set based on a pre-constructed minimum loss function, wherein quality judgment is performed on the texture map generated by the generator network through the multi-discriminator network and the pre-constructed loss function.

7. The single highlight image-based SVBRDF texture modeling method of claim 1, wherein said multi-discriminator network specifically comprises a discriminator for processing diffuse reflection maps, a discriminator for processing normal maps, a discriminator for processing roughness maps, a discriminator for processing reflection maps and a discriminator for processing rendering results.

8. The utility model provides a SVBRDF material modeling system based on single highlight image which characterized in that includes:

the data acquisition unit is used for acquiring a highlight image of the surface of the object material;

the highlight elimination unit is used for eliminating highlights of the highlight image in a mode based on dense feature fusion connection and highlight multilevel identification to obtain a highlight-free image;

the object surface material estimation unit is used for inputting the highlight images and the non-highlight images into a generator network trained in advance to obtain a spatial variation bidirectional reflectivity distribution function of the object surface so as to obtain a corresponding material chartlet; the generator network comprises a shared encoder and a plurality of decoders respectively connected with the shared encoder, wherein the decoders respectively correspond to the processing of the diffuse reflection map, the normal map, the roughness map and the reflection map.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory for execution, wherein the processor when executing the program implements the single highlight image-based SVBRDF material modeling method of any of claims 1-7.

10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the program when executed by a processor implements the single highlight image based SVBRDF material modeling method of any one of claims 1-7.

Images