KR102687766B1 - Sorting devices, screening methods and screening programs, and computer-readable recording media or storage devices - Google Patents

Abstract

Allows users to easily set and change the sorting accuracy.
(Solution) Using the line sensor camera 11 and a learning data creation unit 122 that creates learning data (LD) from the data of each type of object acquired by the line sensor camera 11, and the learning data (LD). A learning unit 123 that learns how to classify a mixture (MO) by type and make it into an object by type, and creates a learning model (GM) that converts the knowledge and experience obtained through the learning into data, and a selection object (SO) ) A recognition rate (RR) is calculated from the selection target selection unit 124 that selects the type, and the data of the mixture (MO) acquired by the line sensor camera 11 based on the learning model (GM), and the recognition rate (RR) is calculated. Based on (RR), a judgment unit 125 that determines the presence and location of the selection object (SO), and a judgment result of the judgment unit 125 and a threshold formed for the recognition rate (RR), the mixture ( It is provided with an air injection nozzle (14) for sorting the sorting object (SO) from the MO).

Description

Sorting devices, screening methods and screening programs, and computer-readable recording media or storage devices

The present invention relates to a sorting device, a sorting method and a sorting program, and a computer-readable recording medium or storage device for sorting out a sorting object from a mixture consisting of a plurality of types of objects.

Recently, recycling of waste materials, etc. into resources and use as raw materials for new products has been implemented by many companies from the viewpoint of environmental protection and for purposes such as improving corporate image.

The recycling field covers many fields. For example, in the field of recycling old paper to produce recycled paper, there is a problem of impurities that the purity of the paper decreases when plastics such as laminate are mixed into the waste paper. There is. Additionally, if harmful substances are mixed, these harmful substances will spread widely. For this reason, before recycling, a process of selecting the objects used as raw materials and impurities is necessary. In addition, it is required to be able to freely sort the sorting object according to the purpose of recycling, for example, sorting white paper and colored paper.

In addition, regardless of recycling, since it is necessary to sort good products from defective products when manufacturing products, the technology for sorting objects into two or more can be said to be one of the essential technologies in the manufacturing industry. Techniques for sorting this type of object into two or more types are disclosed in, for example, Patent Document 1 and Patent Document 2.

Patent Document 1 discloses a technology related to a sorting device that includes a detection means consisting of a light source and an optical sensor and sorts objects based on the brightness of reflected light.

Additionally, Patent Document 2 discloses technology regarding a sorting device that automatically selects objects using artificial intelligence, including a gravity sensor and an RGB camera, an X-ray camera, a near-infrared camera, and a 3D camera as an imaging device.

Japanese Patent Publication No. 2018-017639 Japanese Patent Publication No. 2017-109197

However, the sorting device disclosed in Patent Document 1 requires pre-setting standards and algorithms for sorting objects based on the brightness of reflected light, and requires specialized knowledge and experience to set these, so the user can easily use them. Settings or changes to settings could not be made.

In addition, the sorting device disclosed in Patent Document 2 using artificial intelligence does not require the settings as described above, but requires a process of teaching the artificial intelligence the criteria and method for sorting in advance, so that the user can easily set it. It was not meant to be.

In this way, in the conventional sorting device or sorting method, it is not easy to set the sorting device to sort the sorting object, so the user is provided with a sorting device with settings according to the sorting object in advance and operates it. was doing For this reason, when the mixture (waste, etc.) or the object to be sorted is changed, there is a problem that the user cannot easily change the settings even if he or she wants to change them.

The present invention was made in consideration of such problems in the prior art. One object of the present invention is to provide a sorting device, a sorting method, a sorting program, and a computer-readable recording medium or It is about providing a device that remembers.

A sorting device related to the first aspect of the present invention is a sorting device that selects an object to be sorted from a mixture consisting of a plurality of types of objects, and acquires data based on the type object or the mixture, which is the object classified by type. An acquisition unit, a learning data creation unit for creating learning data from the data of the object of the type acquired by the data acquisition unit, classifying the mixture into types using the learning data created by the learning data creation unit, and classifying the mixture into types. A learning unit that learns how to use an object and creates a learning model that converts the knowledge and experience obtained through the learning into data, a selection target selection unit that selects the type of the selection object from the types of objects, and the learning unit. Based on the created learning model, a determination unit that determines the presence or absence and location of the type of selection object selected in the selection object selection unit from the imaging data of the mixture acquired by the data acquisition unit, and the mixture based on the judgment result of the determination unit It may be provided with a selection unit that selects the selection object from among the selection units, and an operation unit that receives operations from the user and gives instructions to each unit.

According to the above configuration, since the presence or absence and location of the object to be sorted can be determined from the imaging data of the mixture using artificial intelligence, there is no need to set standards or algorithms for sorting the object. Additionally, since each member is provided with an operation unit that receives operations from the user and gives instructions, the user can easily create a learning model, and the process of having artificial intelligence learn it can also be easily performed.

Therefore, according to the present invention, it can be simply operated using the control panel and most of the complicated setting work can be performed by artificial intelligence, so even if the user does not have specialized skills or knowledge, the sorting object can be selected. Settings for selection can be easily performed.

In addition, the selection device related to the second aspect of the present invention includes a learning unit where the operation unit includes a data acquisition instruction unit instructing the data acquisition unit to acquire data, and a learning data creation unit instructing the learning data creation unit to start creating the learning data. A data creation instruction unit, a learning start instruction unit instructing the learning unit to create the learning model, a selection object selection instruction unit instructing the selection object selection unit to select a type of the selection object, and the selection unit in the determination unit. The operation start instruction unit may be provided to determine the presence and location of an object, and to select the selection object from the mixture based on the determination result in the selection unit.

Furthermore, the selection device according to the third aspect of the present invention has a learning mode in which the operation unit displays at least the data acquisition instruction section, the learning data creation instruction section, and the learning start instruction section, and a driving mode in which the operating section displays at least the driving start instruction section. It may be provided with a mode switching instruction unit that instructs a mode switching operation including. According to the above configuration, the user can perform work while determining which operating situation the selection device is in, namely the learning mode and the operating mode, and the setting work in the learning mode includes instructions related to the setting. Because the information is concentrated, it is easy to prevent erroneous operation.

Furthermore, in the selection device according to the fourth aspect of the present invention, the operation unit can display at least the data acquisition instruction unit, the learning data creation instruction unit, the learning start instruction section, the selection target selection instruction section, and the driving start instruction section on one screen. . According to the above configuration, there is no distinction between modes such as learning mode and driving mode, and both the setting-related indicator and the driving-related indicator are displayed on one screen, so there is no need to switch between the learning mode and the driving mode. can do.

Furthermore, the selection device according to the fifth aspect of the present invention can be configured so that the operation unit is a touch panel. According to the above configuration, the user can operate it easily.

Furthermore, the screening device according to the sixth aspect of the present invention can be configured so that the data acquisition unit includes a visible camera, and the data acquired by the data acquisition unit is image data. According to the above configuration, the data acquisition unit is provided with a visible camera and can acquire data as image data, so that the selection target can be selected based on the shape, position, size, and range of the selection target. Additionally, for example, if the data acquisition unit is a camera equipped with a spectroscope, data can be acquired as spectral distribution data.

Furthermore, the selection device according to the seventh aspect of the present invention has a storage unit that stores image data of the type object in association with information specifying the type of the type object, and the learning data creation unit acquires the data. an image extraction unit that creates extracted image data by removing the background from the image data of the object of the type acquired by the unit and extracting the object of the type, and the extracted image of all types of objects included in the mixture created by the image extraction unit; an image synthesis unit that randomly selects one or a plurality of extracted image data from among the data and creates training image data by synthesizing background image data captured by the data acquisition unit and the extracted image data, and the image synthesis unit; It may be configured to have an answer creation unit that creates the learning data by associating the learning image data created in the unit with the information on the type and position of the type object included in the learning image data specified based on the information stored in the storage unit. You can. According to the above configuration, the number of learning data to be learned by artificial intelligence can be controlled by user instructions, and thus the precision of selection can be improved by increasing the number of learning.

Furthermore, in the sorting device according to the eighth aspect of the present invention, the sorting unit can select the sorting object from the mixture by blowing compressed air on the sorting object based on the judgment result.

Furthermore, the selection device according to the ninth aspect of the present invention is such that the determination unit selects each object in the mixture from the data of the mixture acquired by the data acquisition unit based on the learning model created in the learning unit to be the selection target. Calculate a first recognition rate indicating the probability that the selection object is selected by the selection unit, determine the presence or absence and location of the selection object based on the first recognition rate, and the selection unit determines the determination result of the determination unit and the first recognition rate. The selection object can be selected from the mixture based on a threshold value formed for . According to the above configuration, during the operation of selecting a selection object from a mixture, the first recognition rate indicating the probability that each object in the mixture is a selection object is calculated by artificial intelligence, and the recognition rate and a threshold value that can be set by the user are mutually related. Because the selection target is judged by connection, the user can control the selection precision while using artificial intelligence. In other words, the purpose of sorting can be assumed to be various, from cases where it is sufficient to roughly classify to cases where it is desired to extract only desired objects with high precision, etc., so that sorting according to the user's request for sorting precision becomes possible.

Furthermore, in the sorting device according to the tenth aspect of the present invention, the sorting unit can select a sorting object whose first recognition rate is equal to or greater than the threshold. According to the above configuration, by setting the threshold high, it is possible to select with high precision, and by setting the threshold low, it is possible to roughly select.

Furthermore, the selection device according to the eleventh aspect of the present invention is such that the judgment unit determines that each object in the mixture is an object of the type from the data of the mixture acquired by the data acquisition unit based on the learning model created by the learning unit. Calculating a second recognition rate for each object indicating the probability that it is an object of that type, specifying the type of each object in the mixture based on the second recognition rate, and a second recognition rate when the type matches the type of the selection object. By considering this as the first recognition rate, the presence and location of the selection object can be determined. According to the above configuration, since the second recognition rate is calculated for each object of each type for each object in the mixture, the type of the object can be determined as the type for which the second recognition rate is the highest, and the type is the same as the selection object. Since the identified objects are sorted by linking them to a threshold value that can be set by the user, the user can select the sorted object without having to re-create a learning model specialized for the sorted object even when changing the sorted object. It can be easily changed.

Furthermore, the selection device according to the twelfth aspect of the present invention includes a threshold setting unit that sets a desired threshold value for the first recognition rate, and the operation unit sets the threshold value in the threshold setting unit. It may have a threshold setting indicator that indicates. According to the above configuration, the user can easily set and change the sorting precision.

Furthermore, the selection method related to the thirteenth aspect of the present invention is a selection method for selecting a selection object from a mixture consisting of a plurality of types of objects, and receives an operation from a data acquisition instruction unit to select the object classified by type. A data acquisition step of acquiring data based on the object or the mixture, a learning data creation step of receiving an operation from a learning data creation instruction unit and creating learning data from the data of the type object acquired in the data acquisition step, and learning By receiving an operation from the start instruction unit, the learning data created in the learning data creation process is used to learn how to classify the mixture into types and create objects by type, and create a learning model that converts the knowledge and experience obtained through the learning into data. A learning process to create, a selection target selection process for receiving an operation from a selection target selection instruction unit to select the type of the selection object from the type objects, and a learning model created in the learning process by receiving an operation from an operation start instruction unit. Based on this, an operation process of determining the presence or absence and location of the type of selection object selected in the selection object selection process from the data of the mixture acquired in the data acquisition process, and selecting the selection object from the mixture based on the judgment result. may include.

Furthermore, the selection method according to the fourteenth aspect of the present invention includes a learning mode that receives an operation from a mode switching instruction unit and displays at least the data acquisition instruction section, the learning data creation instruction section, and the learning start instruction section, and at least the operation start instruction section. A mode switching operation including an operation mode displaying can be performed.

Furthermore, the selection method according to the fifteenth aspect of the present invention can display at least the data acquisition instruction section, learning data creation instruction section, learning start instruction section, selection object selection instruction section, and driving start instruction section on one screen.

In addition, the selection method related to the sixteenth aspect of the present invention provides, in the operation step, an operation from an operation start instruction unit, and based on the learning model created in the learning step, data of the mixture acquired in the data acquisition step. From this, a first recognition rate indicating the probability that each object in the mixture is the selection object selected in the selection object selection unit is calculated, and based on the first recognition rate, the presence or absence and location of the selection object are determined, and the judgment result is And the selection object may be selected from the mixture based on a threshold value formed for the first recognition rate.

Furthermore, the selection method according to the seventeenth aspect of the present invention can select, in the operation process, a selection object whose first recognition rate is equal to or greater than the threshold value.

Furthermore, the selection method related to the 18th aspect of the present invention is such that, in the operation process, each object in the mixture is selected from the mixture data acquired in the data acquisition process, based on the learning model created in the learning process. For each type of object, a second recognition rate indicating the probability that it is an object of that type is calculated, and based on the second recognition rate, the type of each object in the mixture is specified, and the type matches the type of the selection object. By considering the 2 recognition rate as the first recognition rate, the presence and location of the selection object can be determined.

Furthermore, the selection method related to the 19th aspect of the present invention is a selection program for selecting a selection object from a mixture consisting of a plurality of types of objects, and receives an operation from a data acquisition instruction unit to select the objects classified by type. A function to acquire data based on the type object or the mixture, an operation from a learning data creation instruction section, a function to create learning data from the acquired imaging data of the type object, and an operation from a learning start instruction section. , using the learning data created above, to learn how to classify mixtures by type and create objects by type, and to create a learning model that converts the knowledge and experience obtained through the learning into data, and operation from the selection target selection instruction unit. a function to select the type of the sorting object from the type objects, and receiving an operation from the operation start instruction unit, based on the created learning model, the presence or absence of the sorting object of the selected type from the data of the acquired mixture, and The function of determining the position and selecting the sorting object from the mixture based on the judgment result can be implemented in a computer.

Furthermore, the selection method according to the twentieth aspect of the present invention receives an operation from a driving start instruction unit and, based on the created learning model, selects each object in the mixture from the data of the mixture to be selected by the selection target selection unit. Calculate a first recognition rate indicating the probability that the selection object is selected, determine the presence or absence and location of the selection object based on the first recognition rate, and based on the judgment result and a threshold formed for the first recognition rate. The selection target can be selected from the mixture.

Moreover, the selection method according to the 21st aspect of the present invention enables a computer to realize the function of selecting a selection object whose first recognition rate is equal to or higher than the threshold value.

Furthermore, the selection method related to the 22nd aspect of the present invention is a second recognition rate that indicates the probability that each object in the mixture is an object of that type for each object of the type, from the data of the obtained mixture, based on the learning model created above. Calculate, based on the second recognition rate, specify the type of each object in the mixture, consider the second recognition rate when the type matches the type of the selection object as the first recognition rate, and perform the selection. The function of determining the presence and location of an object can be implemented on a computer.

Furthermore, the recording medium or storage device related to the twenty-third aspect of the present invention stores the above program. Recording media include CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD+R, DVD-RW, DVD+RW, Blu- Includes magnetic disks such as ray (registered trademark), BD-R, BD-RE, HD DVD (AOD), optical disks, magneto-optical disks, semiconductor memories, and other media capable of storing programs. In addition, programs include those distributed by being stored on the above-mentioned recording medium, as well as those distributed by downloading through network lines such as the Internet. Additionally, recording media include devices capable of recording programs, for example, general-purpose or dedicated devices in which the program is executable in the form of software or firmware. Additionally, each process or function included in the program may be executed by computer-executable program software, and the processing of each part may be performed using hardware such as a predetermined gate array (FPGA, ASIC) or some elements of program software and hardware. It may be realized in a format where partial hardware modules are mixed.

1 is a schematic diagram of a sorting device related to an embodiment of the present invention.
Fig. 2 is an explanatory diagram of the positional relationship between a line sensor camera and a conveyor according to an embodiment of the present invention.
Fig. 3 is an explanatory diagram of image data created by a line sensor camera according to an embodiment of the present invention.
Fig. 4 is an explanatory diagram of a method of extracting a portion where an object is reflected from image data and creating extracted image data.
Fig. 5 is an explanatory diagram of a method of extracting a portion where an object is reflected from image data and creating extracted image data.
Fig. 6 is an explanatory diagram of a method for creating image data of an artificial mixture.
Fig. 7 is an explanatory diagram of a method for setting the injection timing for each injection area and air injection nozzle.
Fig. 8 is a functional block diagram of a sorting device related to an embodiment of the present invention.
Fig. 9 is a flow chart showing the flow of the method of operating the selection device in the learning mode.
Fig. 10 is an explanatory diagram of the screen displayed on the controller.
Fig. 11 is an explanatory diagram of the screen displayed on the controller.
Fig. 12 is an explanatory diagram of the screen displayed on the controller.
Figure 13 is an explanatory diagram of the screen displayed on the controller.
Fig. 14 is an explanatory diagram of the screen displayed on the controller.
Fig. 15 is an explanatory diagram of the screen displayed on the controller.
Figure 16 is an explanatory diagram of the screen displayed on the controller.
Figure 17 is an explanatory diagram of the screen displayed on the controller.
Fig. 18 is a flow chart showing the flow of the method of operating the sorting device in the operation mode.
Figure 19 is an explanatory diagram of the screen displayed on the controller.
Figure 20 is an explanatory diagram of the screen displayed on the controller.
Figure 21 is an explanatory diagram of the screen displayed on the controller.
Figure 22 is an explanatory diagram of the screen displayed on the controller.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention will be described based on the drawings. However, the embodiments shown below exemplify a sorting device for embodying the technical idea of the present invention, and the present invention does not specify them as follows. In addition, this specification in no way specifies members appearing in the claims as members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, etc. of the component parts described in the embodiments are not intended to limit the scope of the present invention only thereto, unless specifically stated, and are merely illustrative examples. Additionally, the sizes and positional relationships of members shown in each drawing may be exaggerated for clarity of explanation. In addition, in the following description, the same names and symbols indicate identical or homogeneous members, and detailed descriptions are appropriately omitted. In addition, each element constituting the present invention may be configured by forming a plurality of elements into the same member, so that a single member can serve as a plurality of elements, or conversely, the function of one member may be divided into a plurality of members. there is.

(Selection device (1))

Regarding the sorting device 1 according to the embodiment of the present invention, based on FIG. 1 which is a schematic diagram, FIG. 8 which is a functional block diagram, and FIG. 2 which is an explanatory diagram of the positional relationship between the line sensor camera 11 and the conveyor 13. This explains.

As shown in FIG. 1, the sorting device 1 according to the present embodiment selects the sorting target object (SO) from the mixture (MO) composed of a plurality of types of objects supplied from the supply device 2 and flowing on the conveyor 13. ) as a device for sorting out the compressed air using an air injection nozzle 14 that discharges compressed air, mainly using a line sensor camera 11 (corresponding to an example of the “data acquisition unit” in the scope of the patent claims), 1 It is composed of a control unit 12, a controller 15 (corresponds to an example of the “operation unit” in the scope of the patent claims), a conveyor 13, and an air injection nozzle 14. The feeding device 2 includes, for example, an input hopper 21, a transfer conveyor 22, and an input feeder 23. The input hopper 21 is configured to receive the mixture MO. The transfer conveyor 22 supplies the mixture MO supplied from the input hopper 21 to the input feeder 23. The input feeder 23 is comprised of a vibrating feeder or an electromagnetic feeder, and supplies the mixture MO to the conveyor 13 by vibrating while preventing the mixture MO from overlapping with each other.

The sorting device 1 has two modes: learning mode (LM) and operating mode (OM). The learning mode (LM) is a mode in which preparations and settings for operating the sorting device 1 are performed. On the other hand, the operation mode OM is a mode that actually selects the sorting object SO from the mixture MO.

For the mixture (MO), individual objects such as metal, paper, plastic, etc. can be identified from image data acquired by the line sensor camera 11, and the path can be changed by spraying air by the air injection nozzle 14. It is composed of multiple types of objects. The types of objects included in the mixture (MO) are, for example, assumed to be metal, paper, plastic, etc., but are not limited to the large bundle called metal, and are classified into lower layers such as copper, aluminum, etc. Anything that can be identified from color and shape can be an object. In addition, the sorting device 1 according to the present embodiment is capable of identifying up to five types of objects at a time, such as aluminum, brass, gold, silver, and copper, and can identify mixtures composed of such objects ( Among the MO), as the sorting object (SO), it is configured so that one type, such as only copper, can be selected, or multiple types can be selected simultaneously, such as aluminum, brass, and gold, for example.

Hereinafter, each member will be described in detail. In addition, in the following description, for convenience, the mixture (MO) is composed of objects A to C (corresponding to an example of a “class object” in the scope of the patent claims), and object A is used as the object to be screened (SO). Make your choice.

(Line Sensor Camera (11))

As shown in FIG. 2, the sorting device 1 is formed with two line sensor cameras 11 aligned in the width direction of the conveyor 13. The line sensor camera 11 is a member that captures images every time it receives a pulse from the encoder 131 of the conveyor 13 and acquires image data (ID) from the results of this imaging.

The X direction of the line sensor camera 11 corresponds to the width direction of the conveyor 13, and the Y direction corresponds to the moving direction of the conveyor 13. As shown in FIG. 2, there is one line sensor camera 11, A predetermined X-direction imaging range 11a can be imaged. From this X-direction range (11a), the As shown in FIG. 3, image data (ID) is created by extracting (11e) in the Y direction from a predetermined Y direction range (11f). Among the created image data (ID), the desired overlapping range (11g) from one end of the Y direction is a range that overlaps with the image data (ID) created immediately before.

The line sensor camera 11 in the learning mode captures images of the objects included in the mixture MO for each object and creates image data ID for each object. Specifically, imaging is performed while a plurality of objects A are flowing on the conveyor 13, and image data (ID) of the objects A is created. Objects B and C are similarly processed to create image data (ID) for objects B and C. The created image data (ID) of each object is transmitted to the storage unit 121 shown in FIG. 8 and stored in a state in which it is associated with the name of the imaged object. Additionally, imaging is performed without the object flowing on the conveyor 13 to create a background image (BI), and the created background image (BI) is transmitted to the storage unit 121 and stored.

Additionally, the line sensor camera 11 in the operation mode OM captures the mixture MO while it is flowing on the conveyor 13, and creates image data ID of the mixture MO. Image data (ID) of the created mixture (MO) is transmitted to the judgment unit 125.

In addition, the line sensor camera 11 has been described as an example of the “data acquisition unit” in the scope of the patent claims, but the “data acquisition unit” is not limited to this, and may be an area sensor camera, and may be an area sensor camera, and may be an example of a “data acquisition unit” in the patent claims. Any of these may be used. When using X-rays, the X-ray light source can be placed on top of the object transported by the conveyor, the X-ray camera can be placed on the bottom of the conveyor belt, and vice versa.

In addition, the created image data (ID) of each object is, in addition to the name of the object, information that allows the user to know the type of the object when selecting the sorting object (SO) in the sorting object selection unit 124 described later. It just has to be related to.

In addition, it is not necessarily necessary to store the background image (BI) captured and created by the line sensor camera 11 in the storage unit 121. For example, the background image (BI) is stored in the storage unit 121. It may be prepared separately at this stage and stored in the storage unit 121.

Additionally, instead of the background image BI, information on the color of the conveyor 13 may be stored in the storage unit 121.

(First control unit (12))

The first control unit 12 includes a storage unit 121, a learning data creation unit 122, a learning unit 123, a selection target selection unit 124, a threshold setting unit 126, and a judgment unit 125. Equipped with In the operation mode OM, the first control unit 12 determines the presence and position of the selection object SO from the image data (ID) of the mixture MO acquired by the line sensor camera 11. Additionally, in the learning mode (LM), preparations and settings for the judgment are performed. Hereinafter, each member will be described in detail.

(Memory (121))

The storage unit 121 is a member that stores the image data (ID) of objects A to C created by the line sensor camera 11, the name of the object associated with the image data (ID), and the background image (BI). .

(Learning data creation department (122))

The learning data creation unit 122 creates and stores learning data (LD) from the image data (ID) and background image (BI) of the objects A to C acquired and imaged by the line sensor camera 11. do. The learning data creation unit 122 is comprised of three members: an image extraction unit 122a, an image synthesis unit 122b, and an answer creation unit 122c. The configuration of each member is as described later.

The created learning data (LD) is used for learning conducted by the learning unit 123. For one learning, one learning data (LD) is used, and the greater the number of times this learning is repeated, the better the selection accuracy in the operation mode (OM). In other words, the more learning data LD created by the learning data creation unit 122, the better the selection accuracy in the operation mode OM. Additionally, the selection device 1 according to the first embodiment of the present invention is in an aspect in which the user can freely set the number of repetitions of learning, with the upper limit set at 40,000 (details will be described later).

(Image extraction unit 122a)

The image extracting unit 122a retrieves the image data (ID) and background images (BI) of objects A to C from the storage unit 121 and, based on the background images (BI), retrieves the image data of objects A to C. The portion where the object is reflected is extracted from (ID), and extracted image data (SD) is created. For example, when extracting image data (ID) of object A, as shown in FIG. 4, the range excluding the overlapping range (11g) is compared with the background image (BI) for each pixel. As a result of the comparison, the part other than the part matching the background image (BI) is cut out as the part where the object A is reflected, and extracted image data (SD) of the object A is created. As mentioned above, basically, comparison is performed in a range excluding the overlapping range (11g), but as shown in Figure 5, if object A is located outside of this range, the range is expanded to the overlapping range (11g) and comparison is made. carry out. In the same manner, extracted image data (SD) of objects B and C are created from the image data (ID) of objects B and C.

Additionally, in addition to the portion that completely matches the background image (BI), a range considered to match the background image (BI) may be set, and areas other than these may be cut out to create extracted image data (SD). According to this, even if, for example, there are scratches or dirt on the conveyor 13 and it does not completely match the background image (BI), the object can be cut out appropriately and extracted image data (SD) of the object can be created. there is.

In addition, it is not necessary to strictly extract only the part where the object is reflected, and the object may be cut out so that the background image (BI) remains, and extracted image data (SD) of the object may be created, for example, including the part of the object. You can also cut it into a rectangular or circular shape. In this way, when cutting out an object so that the background image BI remains, the shape is not particularly limited, but a shape that reduces the area of the remaining background image BI is preferable.

(Image synthesis unit 122b)

As shown in FIG. 6, the image synthesis unit 122b randomly selects some data from the extracted image data (SD) of objects A to C created by the image extraction unit 122a and creates a background image (BI). Then, image data (ID) of an artificial mixture (MO) is created by compositing at random positions, angles, and sizes.

That is, the image synthesis unit 122b changes the position, angle, and size of the extracted image data (SD), from the image data (ID) of a small number of objects A to C, to the image data of a large number of artificial mixtures (MO). (ID) can be written. In addition, as explained in the section on the image extraction unit 122a, in the case where the object is cut out so that the background image BI remains, and the extracted image data SD is created, the image synthesis unit 122b generates the extracted image data. Extracted image data (SD) is synthesized at positions where (SD) overlap each other, so that image data (ID) of the mixture (MO) is not created. This is to prevent the shape of the object from changing by overlapping the part of the background image (BI) in which the extracted image data (SD) remains with the part of the object in other extracted image data (SD).

(Answer writing section (122c))

The answer creation unit 122c stores information recording which of the objects A to C is placed at which position in the image data (ID) of the artificial mixture (MO) created in the image combining unit 122b, as the artificial mixture ( Learning data (LD), which is data related to the image data (ID) of MO), is created.

(Study Department (123))

The learning unit 123 has artificial intelligence and uses the learning data (LD) created in the learning data creation unit 122 to learn how to discriminate objects A to C and creates a learning model (GM). .

Specifically, first, the probability that each object reflected in the image data (ID) of the artificial mixture (MO) in the learning data (LD) is object A is calculated. Similarly, the probability of object B and the probability of object C are calculated (the calculated probabilities are hereinafter referred to as recognition rate (RR). Additionally, recognition rate (RR) is the "second recognition rate" in the scope of the patent claims. corresponds to an example of ). Next, each object is predicted to be the type of object with the highest recognition rate (RR) among objects A to C, and it is checked whether the prediction was correct based on the related information in the answer creation unit 122c. By repeating this, a learning model (GM), which is a data conversion of the knowledge and experience obtained, is created and saved.

(Selection target selection section (124))

The selection object selection unit 124 creates and stores a recipe (RE), which is data that relates information on the selection object (SO) selected by the user from objects A to C to the learning model (GM). In the operation mode, the recipe (RE) selected by the user is read and output to the judgment unit 125.

As described above, the sorting device 1 is configured to teach the learning unit 123 how to discriminate between objects A to C, but does not teach it which sorting object SO is. As a result, even if, for example, you want to change the selection object (SO) from object A to object B, you only need to select object B as the selection object (SO) in the selection object selection unit 124, so that learning There is no need to re-learn part (123). Additionally, the selection object SO may be selected before being taught by the learning unit 123.

(Threshold setting unit 126)

The threshold setting unit 126 sets a threshold for the recognition rate (RR) of the selection object (SO). Information on the set threshold value is transmitted to the second control unit 141 and referred to when selecting the selection object SO (details will be described later). Additionally, the threshold does not necessarily need to be set.

(Judgment Department (125))

The judgment unit 125 has artificial intelligence, reads and outputs a recipe (RE) from the selection object selection unit 124 in the operation mode (OM), and based on this recipe (RE), selects a line sensor camera ( 11) Among the image data (ID) of the mixture (MO) created and transmitted, the presence or absence of object A is determined, and if object A is present, the pixel-unit position information is sent to the second control unit 141. Send.

To determine the presence or absence of object A, similarly to the learning unit 123, the recognition rates (RR) of objects A to C for each object are calculated, and the object for which the recognition rate (RR) of object A is the highest is determined to be object A. Also, similarly, the object for which object B's recognition rate (RR) was highest is determined to be object B, and the object for which object C's recognition rate (RR) was highest is determined to be object C.

(Conveyor (13))

The conveyor 13 is a member that flows and moves an object through the imaging range of the line sensor camera 11 to the position of the air injection nozzle 14. The conveyor 13 moves objects at a predetermined speed. In addition, an encoder 131 is formed on the conveyor 13, and the encoder 131 controls the line sensor camera 11, the first control unit 12, and the second control unit 141 every time the conveyor 13 moves a predetermined distance. Send a pulse to The line sensor camera 11 captures images each time it receives this pulse. That is, one pixel of image data (ID) captured by the line sensor camera 11 corresponds to a predetermined distance. Additionally, the first control unit 12 and the second control unit 141 specify the position of the object based on this pulse.

(Air blowing nozzle (14))

The air injection nozzle 14 emits compressed air to the sorting object (SO) whose recognition rate (RR) is equal to or higher than the threshold set in the threshold setting unit 126, and selects the sorting object (SO). It is an absence of selection. In the sorting device 1, a plurality of air injection nozzles 14 are arranged at minute intervals throughout the width direction of the conveyor 13. With the above configuration, the recognition rate (RR) and the threshold that can be set by the user are linked to determine the selection target, allowing the user to control the selection precision while using artificial intelligence, allowing the user to control the selection precision. Selection according to needs becomes possible. Specifically, if the threshold is set low, rough classification is possible, and if the threshold is set high, only the desired object can be extracted with high precision. In addition, the object to be sorted is not limited to the selection object SO whose recognition rate RR is equal to or greater than the threshold set in the threshold setting unit 126. For example, the recognition rate (RR) of the selection object SO may be greater than the threshold set in the threshold setting unit 126 for selecting the selection object SO. Alternatively, an upper limit and a lower limit threshold may be set to select selection objects (SO) with recognition rates (RR) between them, or all selection objects (SO) may be selected without setting a threshold. You can also do this. In addition, compressed air may be sent to items other than the sorting object SO to sort the sorting object SO.

Additionally, the air injection nozzle 14 receives instructions from the second control unit 141 regarding the injection timing, which is the timing for injecting compressed air. Specifically, the second control unit first sets the injection area IR for spraying compressed air based on the positional information of the object A transmitted from the determination unit 125, as shown in FIG. 7 . Next, the injection timing is set for each air injection nozzle 14 based on the injection area IR. The injection timing is formed at predetermined time intervals with respect to the moving direction of the conveyor 13. That is, considering the image data ID of the mixture MO shown in FIG. 7 as an example, d to h based on the time T0 when the upper end of the image data ID reaches the position of the air injection nozzle 14. The air injection nozzles 14 in the row are instructed to spray compressed air at the timing when the air injection nozzles 14 pass through the injection region IR.

The object A, to which compressed air has been blown by the air blowing nozzle 14, is placed at the lower part of the conveyor 13 and is recovered by the hopper 31 of the recovery hopper 3 provided for each type of material to be sorted. The objects B and C, on which compressed air is not blown by the air blowing nozzle 14, are collected by the hopper 32.

(Controller (15))

The controller 15 is a touch panel type controller, and the user can easily operate the sorting device 1 by using the controller 15. The controller 15 includes a mode switching button 15a (corresponding to an example of the “mode switching instruction section” in the patent claims) and an imaging button 15b (corresponding to an example of the “data acquisition instruction section” in the patent claims). Corresponds to an example), learning data creation button 15c (corresponds to an example of the “learning data creation instruction unit” in the patent claims), learning start button 15d (corresponds to the “learning data creation instruction unit” in the patent claims) (Corresponds to an example of the “starting instruction section”), selection target selection button 15e (corresponds to an example of the “selection target selection instruction section” in the patent claims), threshold value setting button 15h (scope of the patent claims) ), an operation start button 15f (corresponds to an example of an “operation start instruction unit” in the patent claims), and an operation end button 15g. Equipped with

(How to operate the sorting device (1))

Hereinafter, a method of operating the sorting device 1 using the controller 15 will be described.

(Operation method in learning mode (LM))

The operating method of the sorting device 1 in the learning mode (LM) is shown in the functional block diagram of FIG. 8, the flow chart of FIG. 9, and the explanatory drawings of the screen displayed on the controller 15 of FIGS. 10 to 17. Explain based on this.

First, in step ST101, the mode switching button 15a is used to switch the sorting device 1 to the learning mode (LM). When the selection device 1 is activated, the screen shown in Fig. 10 is displayed on the controller 15, so the selection device 1 is switched to the learning mode (LM) by pressing the learning mode button 151a, and the controller 15 At (15), the screen shown in FIG. 11 is displayed.

Next, in step ST102, the line sensor camera 11 is made to create image data (ID) and background images (BI) of objects A to C. When the user flows a plurality of objects A on the conveyor and presses the imaging button 15b on the screen shown in FIG. 11, the line sensor camera 11 starts imaging and creates image data (ID) of the objects A. do. When acquisition of image data (ID) of object A is completed, the screen shown in FIG. 12 is displayed on the controller 15, so the user inputs the name of object A into the name input unit 151b and enters the storage unit 121. ) is preserved in . When the storage of object A is completed, the screen in FIG. 11 is displayed again on the controller 15, so the user photographs objects B, C and the background image BI in the same order.

Next, in step ST103, the learning data (LD) is created in the learning data creation unit. When the user presses the learning data creation button 15c on the screen shown in FIG. 11, the screen shown in FIG. 13 is displayed on the controller 15. By pressing the object selection button 151c, the user selects an object (as described in this description) to be used for creating learning data LD from the list of names of objects stored in the storage unit 121 displayed as shown in FIG. 14. In this case, select “Object A”, “Object B”, “Object C”). Once the selection is completed, the screen shown in FIG. 13 is displayed again on the controller 15, so the number of learning data LD to be created is input into the data number input section 152c. When the input is completed, the controller 15 displays a standby screen, as shown in FIG. 15, showing the user the expected time until creation of the learning data LD is completed. When creation of the learning data LD is completed, the screen shown in FIG. 11 is displayed on the controller 15.

Finally, in step ST104, the learning unit 123 is trained using the learning data (LD) to create a learning model (GM). When the user presses the learning start button 15d on the screen shown in FIG. 11, the screen shown in FIG. 16 is displayed on the controller 15. The user learns from the list of learning data LD stored in the learning data creation unit 122 displayed as shown in FIG. 16 (the name of the object used to create the learning data LD is displayed). Learning data (LD) (“object A, object B, object C” in the case of this explanation) used for learning of unit 123 is selected. Upon completing the selection, the controller 15 displays a waiting screen showing the user the expected time until creation of the learning model (GM) is completed, as shown in FIG. 17. When creation of the learning model (GM) is completed, the screen shown in FIG. 11 is displayed on the controller 15.

(Operation method in operation mode (OM))

The operating method of the sorting device 1 in the operation mode (OM) is shown in the functional block diagram of Fig. 8, the flow chart of Fig. 18, and the explanatory drawings of the screen displayed on the controller 15 of Figs. 19 to 21. Explain based on this.

First, in step ST201, the mode switching button 15a is used to switch the sorting device 1 to the operation mode (OM). When the sorting device 1 is activated, the screen shown in FIG. 10 is displayed on the controller 15, so by pressing the operation mode button 152a, the sorting device 1 is switched to the operation mode (OM), and the controller 15 (15) The screen shown in FIG. 19 is displayed.

Next, in step ST202, object A is selected as the selection object SO, and a recipe RE is created in the selection object selection unit 124. When the user presses the selection target selection button 15e on the screen shown in FIG. 19, the screen shown in FIG. 20 is displayed on the controller 15. The user can use the list of learning models (GM) stored in the learning unit 123 displayed as shown in FIG. 20 (the name of the object used to create the learning model (GM) is displayed) to use for discrimination. A learning model (GM) (in this example, “object A, object B, object C”) is selected. When selection is completed, the screen shown in FIG. 21 is displayed on the controller 15. The user selects the selection object SO (“object A” in the case of this description) from the list of objects used to create the selected learning model GM displayed as shown in FIG. 21. When selection is completed, the selection object selection unit 124 creates a recipe RE, and the screen shown in FIG. 19 is displayed on the controller 15.

Next, in step ST203, a threshold value is set for the recognition rate (RR) of the selection object (SO). When the user presses the threshold value setting button 15h on the screen shown in FIG. 19, the screen shown in FIG. 22 is displayed on the controller 15, and a desired threshold value is input into the threshold value input unit 151h. When the input is completed, the threshold setting unit 126 transmits the threshold value information to the second control unit 141, and the screen shown in FIG. 19 is displayed on the controller 15.

In addition, when the desired threshold value is not input to the threshold input unit 151h, it is determined that the threshold value has not been set and the recognition rate (RR) of all selection objects (SO), that is, objects set to the selection objects (SO) ) Select all objects that were the highest. Additionally, the means by which the user sets the threshold value is not limited to the threshold value setting button 15h displayed on the touch panel of the controller 15. For example, instead of the threshold setting button 15h, a seek bar may be displayed on the touch panel, and the seek bar may be used to set the threshold. In addition, the means for setting the threshold value is not limited to using a touch panel. For example, buttons, rotary switches, etc. may be provided on the controller 15 and the threshold value can be set using these. Alternatively, the means for setting the above-mentioned threshold value may be used together. Additionally, the threshold value may be set not only in step ST203 but also in step ST204, which will be described later. According to this configuration, the user can check the actual selection result and fine-adjust the threshold value. At this time, if the means for setting the threshold value uses the above-described seek bar or rotary switch, it becomes possible to operate sensuously, which is desirable for fine adjustment.

Next, object A is selected in step ST204. When the user flows the mixture MO on the conveyor and presses the operation start button 15f on the screen shown in FIG. 19, the line sensor camera 11 starts imaging, and the judgment unit 125 detects object A. The presence or absence of and the pixel-unit position of the object A are determined, and based on this judgment, the air injection nozzle 14 selects the object A.

Finally, in step ST205, the operation end button 15g is pressed to end the selection.

In addition, the mode of the controller 15 and the screen display are not limited to those described above, and may be changed appropriately so that the user can easily operate the selection device 1. For example, the controller 15 may use push buttons, and in this case, the mode switching button 15a is unnecessary. Additionally, the mode switching button 15a may not be provided and all buttons may be displayed on one screen. Additionally, a display may be displayed on the controller 15 to instruct the user to perform the next operation.

Additionally, in the above-described embodiment, each button is provided with a different function, but each function may be linked, or a given button may serve multiple functions. For example, by pressing the learning data creation button 15c, learning data (LD) may be created and a learning model (GM) may be created based on the learning data. In addition, for example, the operation start button 15f also serves the function of instructing the end of operation, so that operation is started by pressing the operation start button 15f the first time, and operation is ended by pressing the operation start button 15f the second time. You can do it if possible. In addition, in the above-described embodiment, the object A is described as the selection target, but a plurality of objects may be selected as the selection target, and a plurality of air injection nozzles or hoppers may be formed accordingly.

As described above, the sorting device 1 to which the present invention is applied can determine the presence or absence and position of the sorting object (SO) from the imaging data of the mixture (MO) using artificial intelligence, so that it can be used as a standard for sorting objects. In addition to the fact that the setting of the algorithm is unnecessary, it can be easily operated, including the step of setting the threshold value, using various buttons displayed on the controller 15. In addition, the recognition rate indicating the probability that each object in the mixture is a selection object is calculated by artificial intelligence, and the recognition rate is linked to a threshold value that can be set by the user to determine the selection target, allowing the user to control the selection precision. there is.

Therefore, according to the present invention, in addition to having artificial intelligence perform most of the complicated setting work, it can be easily operated by the operation panel, so that the user can select the sorting target even if he or she does not have specialized skills or knowledge. Settings for selecting (SO) can be easily performed.

Industrial applicability

The sorting device, sorting method, sorting program, and computer-readable recording medium or storage device related to the present invention can be applied to the purpose of sorting objects into two or more types.

1: Sorting device
11: Line sensor camera
11a: X-direction imaging range; 11b, 11c: Exclusion range; 11d: Effective range in X direction; 11e: X direction range; 11f: Y direction range; 11g: Overlapping ranges
12: first control unit
121: memory unit
122: Learning data creation unit; 122a: image extraction unit; 122b: Image synthesis section; 122c: Answer writing section
123: Learning Department
124: Type selection unit
126: Threshold value setting unit
125: Judgment unit
13: Conveyor
131: encoder
14: Air injection nozzle
141: second control unit
15: controller
15a: Mode switching button; 151a: Learning mode button; 152a: Driving mode button
15b: imaging button; 151b: Name input unit
15c: Create learning data button; 151c: object selection button; 152c: Data number input unit
15d: Start learning button
15e: Screening target selection button
15h: Threshold setting button; 151h: Threshold input unit
15f: Operation start button
15g: End operation button
2: Supply device
21: Input hopper; 22: transfer conveyor; 23: Input feeder
3: Recovery hopper
31, 32: Hopper
MO: mixture
SO: Sorting object
ID: image data
LD: training data
SD: Extracted image data
BI: Background image
GM: learning model
RE: Recipe
RR: Recognition rate
IR: Injection area
LM: Learning mode
OM: driving mode

Claims (23)

A sorting device that selects a sorting object from a mixture consisting of a plurality of types of objects, comprising:
a data acquisition unit that acquires data based on the type object or the mixture, which is the object classified by type;
a learning data creation unit that creates learning data from data on the type of object acquired by the data acquisition unit;
A learning unit that uses the learning data created by the learning data creation unit to learn how to classify mixtures by type and make objects by type, and to create a learning model that converts the knowledge and experience obtained through the learning into data;
a sorting object selection unit that selects a type of the sorting object from among the types of objects;
A determination unit that determines the presence or absence and location of a selection object of the type selected in the selection object selection unit from the imaging data of the mixture acquired by the data acquisition unit, based on the learning model created in the learning unit;
a sorting unit that selects the sorting object from the mixture based on the judgment result of the judging part;
A sorting device comprising an operation unit that receives operations from a user and gives instructions to each of the above units. According to claim 1,
The control unit,
a data acquisition instruction unit that instructs the data acquisition unit to acquire data;
a learning data creation instruction unit instructing the learning data creation unit to start creation of the learning data;
a learning start instruction unit instructing the learning unit to create the learning model;
a selection object selection instruction unit that instructs the selection object selection unit to select a type of the selection object;
A sorting device characterized by comprising an operation start instruction unit that causes the determination unit to determine the presence or absence and location of the sorting object, and causes the sorting unit to select the sorting object from the mixture based on the determination result. According to claim 2,
The operation unit includes a learning mode that displays at least the data acquisition instruction section, the learning data creation instruction section, and the learning start instruction section, and a mode change instruction section that instructs a mode change operation including at least a driving mode that displays the driving start instruction section. A sorting device characterized in that. A screening method for selecting a screening object from a mixture consisting of a plurality of types of objects, comprising:
A data acquisition step of receiving an operation from a data acquisition instruction unit and acquiring data based on the type object or the mixture, which is the object classified by type;
A learning data creation step of receiving an operation from a learning data creation instruction unit and creating learning data from the data of the type object acquired in the data acquisition step;
A learning model that receives an operation from the learning start instruction unit, uses the learning data created in the learning data creation process above, learns how to classify mixtures into types and make objects by type, and converts the knowledge and experience obtained through the learning into data. A learning process to create,
A sorting object selection step of receiving an operation from a sorting object selection instruction unit and selecting a type of the sorting object from the type objects;
Upon receiving an operation from the operation start instruction unit, based on the learning model created in the learning process, the presence and position of a sorting object of the type selected in the sorting object selection process is determined from the mixture data acquired in the data acquisition process, A selection method comprising an operation process of selecting the selection object from the mixture based on the judgment result. A selection program stored in a recording medium for selecting a selection object from a mixture consisting of a plurality of types of objects, comprising:
A function to receive an operation from a data acquisition instruction unit and acquire data based on the type object or the mixture, which is the object classified by type;
A function to receive an operation from a learning data creation instruction unit and create learning data from the acquired imaging data of the object of the type;
A function to receive an operation from the learning start instruction unit, use the learning data created above to learn how to classify mixtures into types and make objects by type, and create a learning model that converts the knowledge and experience obtained through the learning into data; ,
A function to receive an operation from a selection object selection instruction unit and select a type of the object to be sorted from among the objects of each type;
Upon receiving an operation from the operation start instruction unit, based on the learning model created above, the presence and position of the selected type of selection object is judged from the data of the obtained mixture, and based on the judgment result, the selection object is selected from the mixture. A selection program stored in a recording medium that enables the computer to perform the selection function. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images