CN108564104A - Product defects detection method, device, system, server and storage medium - Google Patents

Abstract

The present invention proposes a product defect detection method, device, system, server and computer-readable storage medium, wherein the product defect detection method includes: obtaining image data of the product; converting the image data into a classification request, according to the Deployment of the classification prediction model on the execution server is determined, and the classification request is sent to the execution server, so as to give the prediction result of the defect category through the classification prediction model on the execution server. The embodiment provided by the present invention can iterate the model along with business development, so that the model can adapt to the latest needs of the production environment, and bring significant improvements to industrial production lines in terms of classification accuracy, scalability, standardization, etc., and parallel processing further improves efficiency .

Description

Product defect detection method, device, system, server and storage medium

technical field

The present invention relates to the field of information technology, in particular to a product defect detection method, device, system, server and computer-readable storage medium.

Background technique

At present, the quality inspection link in many production industries mainly uses visual methods to detect defects on product surface images. Taking the paper industry as an example, the detection of paper quality is mainly based on the pictures on the surface of the paper. In the production line of the paper industry, the quality inspection is mostly manual inspection or semi-automatic optical instrument-assisted quality inspection, which is not only inefficient, but also prone to misjudgment; in addition, the industrial data generated in this way is not easy to store, manage and secondary mining Reuse. In the case of manual inspections, business experts are required to conduct inspections on the production site, and manually record defects after they are found for subsequent processing. This method is not only inefficient, it is easy to miss and misjudgment, and it is difficult to mine the data for secondary use. Moreover, the production environment is often harsh, which will adversely affect the health and safety of personnel. The latter quality inspection method is mostly a quality inspection system based on traditional expert systems or feature engineering. The features and judgment rules are solidified into the machine based on experience, and it is difficult to iterate with the development of the business. As a result, with the development of the production process, the system The detection accuracy is getting lower and lower, and even reduced to a completely unusable state. In addition, the features of the traditional quality inspection system are all pre-fixed in the hardware by third-party suppliers. The upgrade not only requires major modification of the production line, but also is expensive. The traditional quality inspection system has obvious shortcomings in terms of security, standardization, and scalability, which is not conducive to the optimization and upgrading of the production line.

Contents of the invention

Embodiments of the present invention provide a product defect detection method, device, system, server, and computer-readable storage medium, so as to at least solve one or more technical problems in the prior art.

In the first aspect, the embodiment of the present invention provides a product defect detection method, including: obtaining image data of the product; converting the image data into a classification request, and determining the execution according to the deployment of classification prediction models on multiple servers server, and send the classification request to the execution server, so as to give the prediction result of the defect category through the classification prediction model on the execution server.

With reference to the first aspect, in the first implementation manner of the first aspect of the present invention, determining the execution server according to the deployment of the classification prediction model on multiple servers includes: querying a preset server resource configuration management table, the The server resource configuration management table is used to record the load status of each server deployed with the classification prediction model; compare the load status of each server deployed with the classification prediction model, and determine the server with the lowest load as the execution server.

In combination with the first aspect and the first implementation manner of the first aspect, in the second implementation manner of the first aspect of the present invention, the prediction result of the defect category returned by the execution server is received; according to the prediction result of the defect category Perform corresponding defect handling operations, where the defect handling operations include: alarming, labeling, storing logs and/or shutting down.

With reference to the first implementation manner of the first aspect, performing a corresponding defect processing operation according to the prediction result of the defect category includes: according to a preset correspondence between the prediction result of the defect category and the defect processing operation, Perform corresponding defect processing operations; or, perform corresponding defect processing according to the preset level of the predicted result of the defect category and the corresponding relationship between the level of the predicted result of the defect category and the defect processing operation operate.

In the second aspect, an embodiment of the present invention provides a product defect detection method, including: receiving a classification request of product image data; performing classification calculation on the classification request through a pre-trained classification prediction model, and giving a prediction of the defect category Result; the classification prediction model includes a feature extraction model and a defect location classification model; the feature extraction model is used to extract the features of the image data in the classification request; the defect location classification model is used to extract the image according to the extracted The characteristics of the data give the prediction result of the defect category, and the prediction result includes: whether there is a defect in the image in the classification request, and the defect category and position coordinates.

With reference to the second aspect, in the first implementation manner of the second aspect of the present invention, before performing classification calculation on the classification request, it further includes: preprocessing the image data in the classification request, the preprocessing Includes image denoising, background removal, image compression and/or format conversion.

With reference to the second aspect and the first implementation manner of the second aspect, in the second implementation manner of the second aspect of the present invention, the present invention further includes: obtaining the classification prediction model through pre-training according to historical annotation data of product image data.

In conjunction with the third implementation of the second aspect, in the third implementation of the second aspect of the present invention, the feature extraction model includes a deep convolutional neural network; the defect location classification model includes RCNN, SSD or Mask RCNN .

In combination with the second aspect and the first implementation manner of the second aspect, in the fourth implementation manner of the second aspect of the present invention, after giving the prediction result of the defect category, it further includes: according to the prediction result of the defect category Perform corresponding defect handling operations, where the defect handling operations include: alarming, labeling, storing logs and/or shutting down.

With reference to the fourth implementation manner of the second aspect, performing a corresponding defect processing operation according to the defect category prediction result includes: according to a preset correspondence between the defect category prediction result and the defect processing operation, Perform corresponding defect processing operations; or, perform corresponding defect processing according to the preset level of the predicted result of the defect category and the corresponding relationship between the level of the predicted result of the defect category and the defect processing operation operate.

In the third aspect, the embodiment of the present invention provides a product defect detection device, including: a data collection module, used to obtain image data of products; a load balancing module, used to convert the image data into classification requests, according to the The deployment of the classification prediction model on each server determines the execution server, and sends the classification request to the execution server, so as to give the prediction result of the defect category through the classification prediction model on the execution server.

In conjunction with the third aspect, in the first implementation manner of the third aspect of the present invention, the load balancing module is further configured to: query a preset server resource configuration management table, and the server resource configuration management table is used to record each deployment The load status of the server with the classification prediction model; comparing the load status of each server deployed with the classification prediction model, and determining the server with the lowest load as the execution server.

In a fourth aspect, an embodiment of the present invention provides a product defect detection device, including: a data receiving module, configured to receive a classification request for image data of a product; a classification prediction model, configured to classify the product through a pre-trained classification prediction model The classification request performs classification calculation, and gives the prediction result of the defect category; the classification prediction model includes a feature extraction model and a defect location classification model; the feature extraction model is used to extract the features of the image data in the classification request; the The defect location and classification model is used to give a defect category prediction result according to the extracted features of the image data, and the prediction result includes: whether there is a defect in the image in the classification request, and the defect category and position coordinates.

With reference to the fourth aspect, in the first implementation manner of the fourth aspect, the present invention further includes a control module, configured to: perform a corresponding defect processing operation according to the prediction result of the defect category, and the defect processing operation includes : Alarms, tagging, logging and/or shutdown.

In a possible design, the structure of the product defect detection device includes a processor and a memory, and the memory is used to store a program that supports the product defect detection device to execute the product defect detection method in the first aspect or the second aspect, the The processor is configured to execute programs stored in the memory.

In the fifth aspect, the embodiment of the present invention provides a product defect detection system, including the device described in any one of the above third or fourth aspects, and a production database for storing image data of products, and The prediction result of the defect category corresponding to the image data of the product and the defect processing operation corresponding to the prediction result of the defect category; the training database is used to store the historical annotation data of the image data of the product, and the historical annotation data is used for training Classification prediction model.

In a sixth aspect, an embodiment of the present invention provides a server, including: one or more processors; a storage device for storing one or more programs; when the one or more programs are used by the one or more When the processors are executed, the one or more processors are made to implement the method as described in any one of the first aspect or the second aspect above.

In a seventh aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, implements the method described in any one of the first aspect or the second aspect above.

One of the above technical solutions has the following advantages or beneficial effects: the embodiment provided by the present invention is applicable to any scene where human eyes, photos or machine vision are used to classify defects, and the model can be iterated with business development, so that the model can adapt to production The latest requirements of the environment have brought significant improvements to industrial production lines in terms of classification accuracy, scalability, and standardization.

Another technical solution in the above technical solutions has the following advantages or beneficial effects: through load balancing and scheduling, parallel processing further improves work efficiency.

The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the drawings and the following detailed description.

Description of drawings

In the drawings, unless otherwise specified, the same reference numerals designate the same or similar parts or elements throughout the several drawings. The drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in accordance with the present invention and should not be taken as limiting the scope of the present invention.

Fig. 1 is the overall frame diagram of the product defect detection method of the embodiment of the present invention;

Fig. 2 is a flow chart of the steps of a preferred embodiment of the product defect detection method provided by the present invention;

Fig. 3 is a work flow diagram of a preferred embodiment of the execution server side of the product defect detection method provided by the present invention;

Fig. 4 is a schematic workflow diagram of a preferred embodiment of the product defect detection method provided by the present invention;

5 is a schematic diagram of a preferred embodiment of the product defect detection method provided by the present invention;

6 is an overall frame diagram of a product defect detection device according to an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a preferred embodiment of the execution server side of the product defect detection device provided by the present invention;

Fig. 8 is a schematic structural diagram of another preferred embodiment of the execution server side of the product defect detection device provided by the present invention.

Detailed ways

In the following, only some exemplary embodiments are briefly described. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and descriptions are to be regarded as illustrative in nature and not restrictive.

An embodiment of the present invention provides a product defect detection method. FIG. 1 is an overall framework diagram of a product defect detection method according to an embodiment of the present invention. As shown in Figure 1, the product defect detection method of the embodiment of the present invention includes: step S110, obtaining image data of the product; step S120, converting the image data into a classification request, according to the classification prediction model on multiple servers The deployment situation determines the execution server, and sends the classification request to the execution server, so as to give a prediction result of the defect category through the classification prediction model on the execution server.

There are mainly two methods in the application of defect classification in the existing quality inspection system. The first is a purely manual quality inspection method, which relies on industry experts to visually observe photos in the production environment to give judgment; the second is a machine-assisted manual quality inspection method, which mainly filters out defects without defects by a quality inspection system with a certain ability to judge , and industry experts will inspect and judge the suspected defective photos. Among them, the second method is mostly developed from expert systems and feature engineering systems, and experts solidify their experience in the quality inspection system. Among the above two methods, the defect detection and location method relies heavily on expert knowledge, which has disadvantages such as low accuracy, poor real-time performance, difficulty in expansion, difficulty in evolving with business, and weak standardization.

The embodiment of the present invention is based on the application of artificial intelligence technology in machine vision, and uses the image collected by the image acquisition device in real time on the production line to detect and judge the surface quality of the product through a pre-trained machine learning model. If there is a quality problem on the product of the collection device, then determine the category corresponding to the quality problem. The embodiment of the present invention is applicable to all scenarios where the product quality is detected based on the image data on the product surface. Compared with the detection method of the prior art that relies on manual labor and expert experience, it has a high degree of automation, high detection accuracy, and can iterate the model with business development. Scalability, standardization, etc. have been significantly improved, and through load balancing, scheduling, and parallel processing, work efficiency has been further improved.

Fig. 2 is a flow chart of the steps of a preferred embodiment of the product defect detection method provided by the present invention. As shown in Figure 2, according to an embodiment of the product defect detection method of the present invention, the execution server is determined according to the deployment of the classification prediction model on multiple servers, including: step S210, querying the preset server resource configuration management table , the server resource configuration management table is used to record the load status of each server deployed with the classification prediction model; step S220, comparing the load status of each server deployed with the classification prediction model, and determining the server with the lowest load as Execute the server.

In this embodiment, the image data of the product is obtained, that is, the pictures generated in real time on the production line are obtained, and then the image data of the product is converted into a classification request (query), and the classification request is sent to the execution server. Among them, the execution server is determined according to the deployment of the classification prediction model on multiple servers. A version of the classification prediction model is copied to multiple machines in advance for parallel processing, and it is executed in real time according to the deployment of the online classification prediction model. Load balancing and scheduling, sending classification requests to the best servers equipped with classification prediction models. The machine resource configuration management table can be set to record the load status of each machine in real time, select a server with the least load, that is, the fastest response, and send the classification request to the server.

According to an embodiment of the product defect detection method of the present invention, after the prediction result of the defect category is given, it further includes: performing a corresponding defect processing operation according to the prediction result of the defect type, and the defect processing operation includes: Alarms, tagging, logging and/or shutdown.

According to an embodiment of the product defect detection method of the present invention, performing a corresponding defect processing operation according to the prediction result of the defect type includes: according to the preset correspondence between the prediction result of the defect type and the defect processing operation relationship, and make corresponding defect processing operations; or, according to the preset level of the predicted result of the defect category, and the corresponding relationship between the level of the predicted result of the defect category and the defect processing operation, make a corresponding Defect handling operations.

On the other hand, the embodiment of the present invention provides a product defect detection method. Fig. 3 is a work flow diagram of a preferred embodiment of the execution server side of the product defect detection method provided by the present invention. As shown in FIG. 3 , the implementation of the server-side product defect detection method in the embodiment of the present invention includes: step S310, receiving a classification request for image data of a product; step S320, performing classification calculation on the classification request through a pre-trained classification prediction model , giving the prediction result of the defect category.

According to an embodiment of the product defect detection method of the present invention, before performing classification calculation on the classification request, it further includes: preprocessing the image data in the classification request, the preprocessing includes image denoising, removing Background, image compression and/or format conversion. The image data is preprocessed, and the effective part is reserved for subsequent processing.

According to an embodiment of the product defect detection method of the present invention, the method further includes: obtaining the classification prediction model through pre-training according to historical annotation data of product image data. Fig. 4 is a schematic workflow diagram of a preferred embodiment of the product defect detection method provided by the present invention. As shown in Figure 4, the classification prediction model is trained by the training engine based on historical labeled data, which is stored in the training database, the training engine sends data requests to the training database, and the training database returns the training data to the training database in response to the data request. engine. In addition, the production database stores data including the image data of recent products and the prediction results of defect categories corresponding to the image data of the products. The production database can provide data updates for the training database at any time. If the production process is updated Therefore, the training data in the training database can be iterated with the development of the business, so that the model can adapt to the latest needs of the production environment.

According to an embodiment of the product defect detection method of the present invention, the classification prediction model includes a feature extraction model and a defect location classification model; the feature extraction model is used to extract the features of the image data in the classification request; the defect The location and classification model is used to give a defect category prediction result according to the extracted features of the image data, and the prediction result includes: whether there is a defect in the image in the classification request, and the defect category and position coordinates.

According to an embodiment of the product defect detection method of the present invention, the feature extraction model includes a deep convolutional neural network; the defect location classification model includes RCNN (Region Based CNN, regional convolutional neural network), SSD (Single Shot MultiBoxDetector ) or Mask RCNN.

Fig. 5 is a principle schematic diagram of a preferred embodiment of the product defect detection method provided by the present invention. As shown in Figure 5, in one embodiment, a deep convolutional neural network is used as the feature extraction model, and object detection and image segmentation technologies are used as the defect location and classification network. The original pictures on the production line are used as the input of the model, and the input After the image data is preprocessed, the effective part is input into the neural network model, and the deep convolutional neural network extracts the features in the original picture, and the features are input into the defect location classification network. The defect location classification network can use RCNN, SSD, Mask RCNN and other object detection and other image segmentation models as the defect location classification model, and judge whether there is a defect in a certain part of the picture according to the features extracted by the deep convolutional neural network. Defects, determine the category to which the defect belongs. The final output of the model is the category of defects in the picture and their relative position coordinates in the picture. If there are multiple defects in the picture, the model will give the category of each defect and its relative coordinates. Taking the paper production industry as an example, the categories of defects include wrinkles, holes, tears, impurities, and color differences.

For each trained model, the old model that is running online can be gradually replaced by going online with a small amount of traffic, so as to achieve the purpose of dynamic expansion and generalization of the model with the business.

In another embodiment, the feature extraction network can also be made based on machine learning methods other than deep convolutional neural network CNN (Convolutional Neural Network, convolutional neural network), and models other than object detection and image segmentation can be used as Defect localization classification network model.

According to an embodiment of the product defect detection method of the present invention, after the prediction result of the defect category is given, it further includes: performing a corresponding defect processing operation according to the prediction result of the defect type, and the defect processing operation includes: Alarms, tagging, logging and/or shutdown.

See Figure 4. After the prediction result representing the defect category is given, the prediction result is sent to the control module. The control module is designed in combination with the business scenario. According to the business requirements, the prediction result given by the model can be made in line with the production environment scenario. Required responses, such as alarming, labeling, storing logs, and/or shutting down, etc., the control module stores the prediction results and response processing behaviors as online production logs in the production database.

According to an embodiment of the product defect detection method of the present invention, performing a corresponding defect processing operation according to the prediction result of the defect type includes: according to the preset correspondence between the prediction result of the defect type and the defect processing operation relationship, and make corresponding defect processing operations; or, according to the preset level of the predicted result of the defect category, and the corresponding relationship between the level of the predicted result of the defect category and the defect processing operation, make a corresponding Defect handling operations.

Among them, the corresponding relationship between the predicted result of the defect category and the defect processing operation, the grade of the predicted result of the defect category, and the corresponding relationship between the grade of the predicted result of the defect category and the defect processing operation can be set by default by the system or customized by the manufacturer set up. For example, the grades of the prediction results of defect categories can be divided into serious, general or not serious; the manufacturer can set the shutdown operation when a serious defect is discovered, the alarm operation when a general defect is discovered, and the only Do labeling operations, etc.

On the other hand, an embodiment of the present invention provides a product defect detection device. Fig. 6 is an overall frame diagram of a product defect detection device according to an embodiment of the present invention. As shown in FIG. 6 , the product defect detection device in the embodiment of the present invention includes: a data collection module 100 for acquiring image data of a product; a load balancing module 200 for converting the image data into classification requests, according to multiple The deployment of the classification prediction model on each server determines the execution server, and sends the classification request to the execution server, so as to give the prediction result of the defect category through the classification prediction model on the execution server.

According to an embodiment of the product defect detection device of the present invention, the load balancing module 200 is also used to: query the preset server resource configuration management table, and the server resource configuration management table is used to record that each deployment has the classification prediction The load status of the server of the model; comparing the load status of each server deployed with the classification prediction model, and determining the server with the lowest load as the execution server.

Referring again to FIG. 4 , the product defect detection device according to the embodiment of the present invention includes a prediction engine Predictor (ie, a load balancing module), a classification prediction model Classifier, and a training engine Trainer. Among them, the prediction engine converts the pictures generated in real time on the production line into classification requests (query), and performs load balancing and scheduling in real time according to the deployment of the online prediction model, and sends the classification requests to the best server equipped with the prediction model . The real-time classification prediction model is running on the server, and the classification prediction model is trained by the training engine based on historical labeled data.

According to an embodiment of the product defect detection device of the present invention, it further includes: receiving the defect category prediction result returned by the execution server; performing corresponding defect processing operations according to the defect category prediction result, the defect processing Actions include: alarm, tag, store log and/or shut down.

According to an embodiment of the product defect detection device of the present invention, performing a corresponding defect processing operation according to the prediction result of the defect type includes: according to the preset correspondence between the prediction result of the defect type and the defect processing operation relationship, and make corresponding defect processing operations; or, according to the preset level of the predicted result of the defect category, and the corresponding relationship between the level of the predicted result of the defect category and the defect processing operation, make a corresponding Defect handling operations.

In yet another aspect, an embodiment of the present invention provides a product defect detection device. Fig. 7 is a schematic structural diagram of a preferred embodiment of the execution server side of the product defect detection device provided by the present invention. As shown in FIG. 7 , the device for detecting product defects on the server side of the embodiment of the present invention includes: a data receiving module 300, used to receive a classification request for image data of a product; a classification prediction model 400, used to pass a pre-trained classification prediction model Classification calculation is performed on the classification request, and the prediction result of the defect category is given.

According to an embodiment of the product defect detection device of the present invention, the classification prediction model 400 is also used for: before performing classification calculation on the classification request, preprocessing the image data in the classification request, the preprocessing Processing includes image denoising, background removal, image compression, and/or format conversion.

According to an embodiment of the product defect detection device of the present invention, it further includes a training engine for pre-training the classification prediction model according to the historical annotation data of the image data of the product.

According to an embodiment of the product defect detection device of the present invention, the classification prediction model 400 includes a feature extraction model and a defect location classification model; the feature extraction model is used to extract the features of the image data in the classification request; the The defect location and classification model is used to give a defect category prediction result according to the extracted features of the image data, and the prediction result includes: whether there is a defect in the image in the classification request, and the defect category and position coordinates.

According to an embodiment of the product defect detection device of the present invention, the feature extraction model includes a deep convolutional neural network; and the defect location classification model includes RCNN, SSD or Mask RCNN.

Fig. 8 is a schematic structural diagram of another preferred embodiment of the execution server side of the product defect detection device provided by the present invention. As shown in FIG. 8, according to an embodiment of the product defect detection device of the present invention, it also includes a control module 600, configured to: perform corresponding defect processing operations according to the prediction results of the defect categories, and the defect processing operations Includes: alarms, tagging, storage logs and/or shutdowns.

According to an embodiment of the product defect detection device of the present invention, the control module 600 is further configured to: perform corresponding defect processing operations according to the preset correspondence between the defect category prediction results and the defect processing operations or, according to the preset level of the predicted result of the defect category, and the corresponding relationship between the level of the predicted result of the defect category and the defect processing operation, make a corresponding defect processing operation.

Referring again to FIG. 4 , the product defect detection device according to the embodiment of the present invention mainly includes a prediction engine Predictor (ie, a load balancing module), a classification prediction model Classifier, a training engine Trainer, a control module Controller, and a database Database.

Among them, the prediction engine converts the pictures generated in real time on the production line into classification requests (query), and performs load balancing and scheduling in real time according to the deployment of the online prediction model, and sends the classification requests to the best server equipped with the prediction model . The server runs a real-time classification prediction model that has been trained by the training engine. After the model pre-processes the image data in the incoming classification request, it performs classification calculations, and gives a prediction result representing the defect category, and sends the result to the control module. The control module is designed in combination with business scenarios, and can respond to the prediction results given by the model in line with the requirements of the production environment scenario according to business needs, such as alarming and storing logs. The control module will store the prediction results and response processing behaviors in the production database as online production logs. The classification prediction model is trained by the training engine based on historical labeled data.

In a possible design, the structure of the product defect detection device includes a processor and a memory, and the memory is used to store a program that supports the product defect detection device to execute the product defect detection method in the first aspect or the second aspect, the The processor is configured to execute programs stored in the memory.

In yet another aspect, an embodiment of the present invention provides a product defect detection system, including the device described in any one of the above-mentioned third or fourth aspects, and a production database for storing product image data, and the The prediction result of the defect category corresponding to the image data of the product and the defect processing operation corresponding to the prediction result of the defect category; the training database is used to store the historical annotation data of the image data of the product, and the historical annotation data is used for training Classification prediction model. Referring to Fig. 4, the historical annotation data is stored in the training database, the training engine sends a data request to the training database, and the training database returns the training data to the training engine in response to the data request. In addition, the image data of recent products and the prediction results of defect categories corresponding to the image data of the products are stored in the production database. The production database can provide data updates for the training database at any time. If the production process is updated, the training database The training data can be iterated with the development of the business, so that the model can adapt to the latest needs of the production environment.

In yet another aspect, an embodiment of the present invention provides a server, including: one or more processors; a storage device for storing one or more programs; when the one or more programs are used by the one or more When the processors are executed, the one or more processors are made to implement the method as described in any one of the first aspect or the second aspect above.

In another aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, implements the method described in any one of the first aspect or the second aspect above.

One of the above technical solutions has the following advantages or beneficial effects: the embodiment provided by the present invention is applicable to any scene where human eyes, photos or machine vision are used to classify defects, and the model can be iterated with business development, so that the model can adapt to production The latest requirements of the environment have brought significant improvements to industrial production lines in terms of classification accuracy, scalability, and standardization.

Another technical solution in the above technical solutions has the following advantages or beneficial effects: through load balancing and scheduling, parallel processing further improves work efficiency.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium, For use with instruction execution systems, devices, or devices (such as computer-based systems, systems including processors, or other systems that can fetch instructions from instruction execution systems, devices, or devices and execute instructions), or in conjunction with these instruction execution systems, devices or equipment used. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since the program can be read, for example, by optically scanning the paper or other medium, followed by editing, interpretation or other suitable processing if necessary. The program is processed electronically and stored in computer memory.

It should be understood that various parts of the present invention can be realized by hardware, software, firmware or their combination. In the embodiments described above, various steps or methods may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, Programmable Gate Arrays (PGAs), Field Programmable Gate Arrays (FPGAs), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. During execution, one or a combination of the steps of the method embodiments is included. The implementation of the device corresponds to the implementation of the method, so the description of the implementation of the device is relatively brief, and relevant descriptions can refer to the description of the implementation of the method.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of its various changes or modifications within the technical scope disclosed in the present invention. Replacement, these should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (17)

1. A method for detecting product defects, comprising:

acquiring image data of a product;

and converting the image data into a classification request, determining an execution server according to the deployment condition of classification prediction models on a plurality of servers, and sending the classification request to the execution server so as to give a prediction result of the defect category through the classification prediction models on the execution server.

2. The method of claim 1, wherein determining the execution server based on a deployment of the classification predictive model across a plurality of servers comprises:

inquiring a preset server resource configuration management table, wherein the server resource configuration management table is used for recording the load state of each server with the classification prediction model;

and comparing the load states of the servers with the classification prediction models, and determining the server with the lowest load as an execution server.

3. The method of claim 1 or 2, further comprising:

receiving a prediction result of the defect type returned by the execution server;

making corresponding defect processing operation according to the prediction result of the defect category, wherein the defect processing operation comprises the following steps: alarm, tag, log, and/or shutdown.

4. The method of claim 3, wherein the performing the corresponding defect handling operation according to the prediction result of the defect category comprises:

making corresponding defect processing operation according to the preset corresponding relation between the prediction result of the defect type and the defect processing operation; or,

and making corresponding defect processing operation according to the preset grade of the prediction result of the defect type and the corresponding relation between the grade of the prediction result of the defect type and the defect processing operation.

5. A method for detecting product defects, comprising:

receiving a classification request for image data of a product;

classifying and calculating the classification request through a pre-trained classification prediction model to give a prediction result of the defect classification; the classification prediction model comprises a feature extraction model and a defect positioning classification model; the feature extraction model is used for extracting features of the image data in the classification request; the defect positioning classification model is used for providing a prediction result of a defect category according to the extracted features of the image data, and the prediction result comprises: whether the image in the classification request has a defect or not and the category and the position coordinates of the defect.

6. The method of claim 5, further comprising, prior to performing a classification calculation on the classification request: and preprocessing the image data in the classification request, wherein the preprocessing comprises image denoising, background removing, image compression and/or format conversion.

7. The method of claim 5 or 6, further comprising: and pre-training according to historical labeling data of image data of the product to obtain the classification prediction model.

8. The method of claim 5 or 6, wherein the feature extraction model comprises a deep convolutional neural network; the defect localization classification model comprises RCNN, SSD or Mask RCNN.

9. The method according to claim 5 or 6, further comprising, after giving the prediction result of the defect class:

making corresponding defect processing operation according to the prediction result of the defect category, wherein the defect processing operation comprises the following steps: alarm, tag, log, and/or shutdown.

10. The method of claim 9, wherein performing the corresponding defect handling operation according to the prediction result of the defect category comprises:

making corresponding defect processing operation according to the preset corresponding relation between the prediction result of the defect type and the defect processing operation; or,

and making corresponding defect processing operation according to the preset grade of the prediction result of the defect type and the corresponding relation between the grade of the prediction result of the defect type and the defect processing operation.

11. A product defect detecting apparatus, comprising:

the data acquisition module is used for acquiring image data of a product;

and the load balancing module is used for converting the image data into a classification request, determining an execution server according to the deployment condition of the classification prediction models on the plurality of servers, and sending the classification request to the execution server so as to give a prediction result of the defect category through the classification prediction models on the execution server.

12. The apparatus of claim 11, wherein the load balancing module is further configured to:

inquiring a preset server resource configuration management table, wherein the server resource configuration management table is used for recording the load state of each server with the classification prediction model;

and comparing the load states of the servers with the classification prediction models, and determining the server with the lowest load as an execution server.

13. A product defect detecting apparatus, comprising:

the data receiving module is used for receiving a classification request of image data of a product;

the classification prediction model is used for performing classification calculation on the classification request through a pre-trained classification prediction model to give a prediction result of the defect type; the classification prediction model comprises a feature extraction model and a defect positioning classification model; the feature extraction model is used for extracting features of the image data in the classification request; the defect positioning classification model is used for providing a prediction result of a defect category according to the extracted features of the image data, and the prediction result comprises: whether the image in the classification request has a defect or not and the category and the position coordinates of the defect.

14. The apparatus of claim 13, further comprising a control module to: making corresponding defect processing operation according to the prediction result of the defect category, wherein the defect processing operation comprises the following steps: alarm, tag, log, and/or shutdown.

15. A product defect detection system, characterized in that it comprises a device according to any of claims 11 or 12 and a device according to any of claims 13 or 14, and

the production database is used for storing image data of a product, a prediction result of a defect type corresponding to the image data of the product and a defect processing operation corresponding to the prediction result of the defect type;

and the training database is used for storing historical marking data of the image data of the product, and the historical marking data is used for training the classification prediction model.

16. A server, comprising:

one or more processors;

storage means for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-4 or 5-10.

17. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-4 or 5-10.