FR3123008A1 - Waste conformity control system - Google Patents

Abstract

The invention relates to a device making it possible to acquire an image of waste before it is dumped into a collection bin. The image of the waste is analyzed by a supervised learning artificial intelligence algorithm, so as to estimate the occurrence of non-compliance of the waste with a specification, in particular a recycling specification. Figure 1A.

Description

Waste conformity control system

The technical field of the invention is the conformity control of waste.

PRIOR ART

The sorting of household waste has become an important social issue, in order to limit the quantity of "ultimate" waste, destined to be eliminated, and to promote recycling possibilities. Formerly confined to a limited number of materials, such as glass, the possibilities of recycling household waste are developing. They now concern common materials, such as plastics, wood, cardboard, metals, paper.

However, an important element of the efficiency of recycling lies in the sorting of waste, during the collection of the latter. The presence of non-recyclable materials in waste intended for recycling is a pitfall that can compromise the quality or efficiency of recycling. Specifications have been put in place, for example by the agglomerations, in order to regulate the sorting of waste. Users have generally been made aware of the practice of sorting through displays, generally at the waste collection bins.

However, it is difficult for a user to ensure that his sorting practice is correct.

The invention described below solves this problem. It allows a user to become aware that his practice in terms of sorting recyclable waste is correct, or to improve his sorting practice.

A first object of the invention is a waste control device, the device being intended to be arranged overhanging a tank to carry out a control of waste dumped into the tank, the device comprising

• a structure, comprising two vertical uprights, between which extends an empty space intended to be occupied by the tray;
• an enclosure, supported by the structure, and arranged above the empty space, the enclosure comprising an inlet opening, configured to allow admission of waste into the enclosure;

the device being characterized in that the enclosure comprises:

• a tray, intended to receive the waste admitted through the admission opening;
• an evacuation opening, opening above the empty space, to allow evacuation of the waste, admitted into the enclosure, towards the bin;

the device also comprising:

• a camera, configured to acquire an image of the waste placed on the plate;
• a processing unit, programmed to classify the waste from the image acquired by the camera;

Preferably, the device comprises an identification unit, configured to detect an identifier of a user of the device. The processing unit can then be programmed to associate the classification carried out with the identifier of the user.

According to one embodiment, the enclosure includes a tilting mechanism allowing the tray to tilt between:

• a reception position, horizontal to within ±20°;
• an evacuation position, inclined with respect to the reception position, so that the plate is oriented towards the evacuation opening, so that each waste deposited on the plate slides towards the evacuation opening.

The device may include an intake hatch, configured to move from an open position, in which the intake opening is open, to a closed position, in which the intake opening is closed, the tilting mechanism being arranged so that:

• when the inlet hatch is open, the tray occupies the receiving position;
• when the inlet hatch is closed, and the camera has acquired the image of each piece of waste placed on the tray, the tray occupies the evacuation position.

The device may comprise a mechanism for opening the admission hatch, configured to maintain the admission hatch closed and authorize an opening of the admission hatch after the identification unit has detected the identifier of the 'user.

The tilting mechanism may comprise a motorization to effect the tilting of the plate between the reception position and the evacuation position and vice versa.

Preferably, the processing unit is programmed to implement a supervised artificial intelligence algorithm, so as to detect non-compliance of the waste with respect to an acceptance criterion, such that the waste is classified as :

• non-compliant when a non-compliance is detected;
• compliant in the absence of detection of non-compliance.

The processing unit can be configured to transmit a signal representative of the classification to a remote central unit, the signal transmitted comprising the identifier of the user detected by the identification unit.

The supervised artificial intelligence algorithm can implement a feature extraction module from the image acquired by the camera by convolutional neural network. The supervised artificial intelligence algorithm can be of the regionalized convolutional neural network type, and in particular regionalized convolutional mask neural network.

A second object of the invention is a method for classifying waste using a device according to the first object of the invention, the method comprising:

1. detection of an identifier of a user by the identification unit;
2. reception of the waste on the tray;
3. acquisition of an image of the waste by the camera;
4. evacuation of the waste through the evacuation opening;
5. from the image acquired by the camera, classification of the waste by the processing unit;
6. association of the result of the classification with the identifier of the user.

The process may include:

g) transmission of the classification result to the user.

Steps d) and e) can be simultaneous or successive, the order in which these steps are carried out being irrelevant.

The invention will be better understood on reading the description of the embodiments presented, in the remainder of the description, in connection with the figures listed below.

FIGURES

FIGS. 1A and 1D are different views of an example of a device according to the invention.

The illustrates the steps of a method of using the device described in connection with steps 1A to 1D.

Figures 3A to 3E illustrate configurations of the device in different stages described in the .

The schematizes a structure of a mask regional convolutional neural network algorithm.

Figures 5A, 5C and 5E are sample waste images

Figures 5B, 5D and 5F are waste classification results corresponding respectively to the images of Figures 5A, 5C and 5E

The illustrates classification performance on 9 different classes of waste.

DESCRIPTION OF PARTICULAR EMBODIMENTS

Figures 1A to 1D represent an example of device 1 according to the invention. The device is intended to check the conformity of waste with a waste acceptance specification. This involves, for example, verifying that the waste belongs to a category of recyclable waste. The device 1 is intended to be used by being associated with existing waste collection bins.

The device 1 is formed of an enclosure 10, intended for the admission and control of the waste, and of a frame, forming a structure 20. The structure 20 forms a support for the enclosure 10. The structure comprises two uprights verticals 21, 22. Each upright extends between a lower end and an upper end, the lower end being closer to the ground than the upper end. The upper ends of the vertical posts are connected by a support 23. Preferably, the support 23 is horizontal. The space 24 between the vertical uprights and the support 23 is empty. The device 1 is intended to be placed above a collection container 40, the collection container 40 then occupying the space 24, between the vertical uprights. In this example, the lower ends of the vertical uprights have rollers so as to facilitate the installation of the device 1 above a bin 40. The bin 40 can be a bin usually used for waste collection. Thus, the device can adapt to a wide variety of collection bins, provided that the dimensions of the latter are compatible with the dimensions of space 24.

We have represented, on the , a frame formed by three axes X, Y and Z. The Z axis is vertical. The X and Y axes are horizontal.

The enclosure 10 rests on the support 23. The enclosure 10 is intended to allow control of waste prior to their transfer into the collection bin 40. The enclosure 10 comprises, cf. Figures 1B and 1C:

• an inlet opening 12, which can be closed by an inlet hatch 11. The inlet opening 12 is intended for the admission of waste into the enclosure 10.
• a tray 13, intended to receive the waste to be checked;
• an evacuation opening 14, opening above the empty space 24 delimited by the structure 20.
• a camera 30, intended to acquire an image of waste placed on the tray.

In this example, the tray is movable. It is articulated by a tilting mechanism 15, configured so that the plate can tilt, between

• a reception position, preferably horizontal, or, more generally, horizontal to within ±20°, cf. ;
• an evacuation position, inclined, towards the vertical, with respect to the reception position, so that the plate 13 is oriented towards the evacuation opening 14, so that waste deposited on the plate slides towards the evacuation opening, so as to be discharged into the collection tray 40. The evacuation position is represented on the described next.

The tilting mechanism 15 can be manual or motorized. In this example, the mechanism is motorized. A motor makes it possible to actuate a jack, arranged on either side of the plate 13, so as to allow a switching of the plate 14 between the reception position and the evacuation position and vice versa. A cylinder is represented on the .

In the example shown, the inlet opening 12 is associated with an inlet hatch 11, which can pass from an open position, in which the inlet opening 12 is open, to a closed position, in which the inlet opening 12 is closed. The admission flap 11 is connected to an opening mechanism 16, comprising a hinge 16 ₁ and an opening control 16 ₂ . The joint 16 ₁ is manual or motorized. The tray tilting mechanism 15 can be arranged so that:

• when the admission door 11 is open, the plate 13 occupies the receiving position;
• when the admission door is closed, after an image of each piece of waste placed on the plate has been acquired by the camera 30, the plate 13 tilts towards the evacuation position, so that the waste falls into the collection 40, through the discharge opening 14.

The device 1 comprises an identification unit 31, configured to detect an identifier of a user depositing waste in the enclosure 10. The identifier is a code associated with the user. The identification unit may comprise a smart card reader or a short-range wireless communication module, for example of the RFID type. In this case, the user identifier is stored in the smart card. Alternatively, the identification unit 31 is configured to communicate with a telephone or other nomadic means used by the user and able to contain and transmit the user's identifier, for example by wireless link. When the user uses a nomadic means of communication, the latter can implement a dedicated use application, so as to facilitate the identification of the user by the identification unit 31 and/or receive information relating the classification of the waste deposited by the user. The using application is programmed to transmit the user's identifier and to receive classification data associated with the user's identifier.

In the example shown, the hatch opening control 16 ₂ is configured to keep the admission hatch 11 closed as long as a user has not been detected by the identification unit 31. After a user identifier has been detected by the identification unit, the opening control 16 ₂ authorizes the opening of the admission hatch 11.

The device may include a trap closure detector 17 . When a closure has been detected, the camera 30 is activated, to acquire an image of each piece of waste placed on the tray.

The device comprises a processing unit 32, programmed to carry out a classification of a piece of waste or of several pieces of waste arranged on the plate 13. The processing unit 32 receives as input the image acquired by the camera 30. From the image of the waste, the processing unit 32 implements a supervised artificial intelligence algorithm, so as to classify the waste. The objective is to classify the waste between a compliant category and a non-compliant category. In this example, the compliant category corresponds to the situation in which each piece of waste deposited on the plate 13 is formed from a recyclable material, of the cardboard, plastic, paper or metal type. When the processing unit detects non-recyclable waste, for example organic waste, non-compliance is detected.

The waste classification result is associated with the user ID. The processing unit receives the identifier of the user, detected by the identification unit 31. It generates a transmission signal and transmits it to a central unit 34, so as to transmit the result of the classification and the user ID. In this way, the central unit 34 can communicate to the user the compliance and/or the non-compliance of the deposited waste. As previously described, when the user uses a dedicated usage application on a nomadic communication means, the central unit 34 is configured to transmit the result of the classification to the user via the usage application. . This alerts the user to insufficient sorting of waste, inviting him to review his sorting practice. In the event of compliance, communication between the central unit and the user makes it possible to reinforce the latter in his sorting practice. The central unit 34 can establish a score associated with each registered user, so as to motivate the users to sort waste that complies with the specifications. The score can be transmitted to the user by the central unit, through the application of use. The central unit 34 can be virtual, the centralization of information and communications being managed in a cloud-type environment.

The central unit 34 can be connected to several devices 1. It allows centralization of the information resulting from the different devices.

The device includes a control unit 33, to control the various components forming the device. The control unit 33 performs the following functions in particular:

• control of the hatch opening mechanism 16 when an identifier has been detected by the identification unit 31;
• activation of the camera 30, so as to launch an image acquisition, after closing of the hatch has been detected;
• activation of the platform tilting mechanism 15 after an image has been acquired by the camera 30.

The identification unit 31, the classification unit 32, the control unit 33 and the central unit 34 are for example means implementing a processor, for example a microprocessor or a microcontroller.

The schematizes the main steps of a waste classification method using the device as previously described. Figures 3A-3E illustrate some of these steps.

Step 100 : identification.

During this step, the enclosure is closed by the hatch 11, as shown in the . The user identifies himself using the identification unit 31.

Step 110 : introduction of waste.

Following the identification of the user, the hatch opening mechanism 16 is activated, so that the user can introduce waste in the enclosure 10. In the example shown, the hatch 11 is configured to delimit a perimeter in which the user has each piece of waste. The hatch can switch between an open configuration, in which it extends horizontally, or, more generally, horizontally to within 20°, to a closed position, in which it closes the inlet opening 12. shows the hatch open: the user can then deposit one or more pieces of waste within the perimeter delimited by the hatch.

Step 120 : closing the hatch.

Figure 3C shows the hatch closing. Figure 3D shows the hatch 11 closed, the waste having slipped, during closing, on the plate 13.

Step 130 : acquisition of an image.

The detection of the closing of the hatch is detected by the closing detector 17. This leads to an acquisition of an image garbage by the camera 30.

Step 140 : evacuation of the waste.

After image acquisition , the tilting mechanism of the plate 15 is activated, so that the plate 13 tilts in the evacuation configuration. Each piece of waste is dumped into tray 40, through opening 14. Cf. .

Step 150 : Classification

After image acquisition , the processing unit 32 performs a classification of each waste, so as to detect any non-compliance.

Step 160 : User information.

The result of the classification is transmitted to the central unit 34, being associated with the identifier of the user. A notable aspect is that the purpose of the device is not to prevent the collection of non-compliant waste, which could raise a question of acceptability by potential users. The device aims to allow the collection of waste, even non-compliant, while informing the user, a posteriori, of the compliance or non-compliance of each waste deposited. Thus, the purpose of the device is essentially educational, aiming to support the user in his sorting practice.

According to an optional possibility, the image , on which the non-compliant waste is materialized, is transmitted to the user. This allows the user to view each waste considered non-compliant following the classification.

In addition to the user, the organization in charge of recycling the waste can also be informed of the detection of non-compliance in the bin. In this case, preferably, the user identifier is not transmitted for reasons of confidentiality.

The processing unit 32 comprises a microprocessor programmed to implement a supervised learning artificial intelligence algorithm to perform the classification of each waste deposited. It may in particular be a convolutional neural network type algorithm. More precisely, the algorithm is an R-CNN (Regional Convolutional Neural Network) type algorithm, and advantageously a fast Mask R CNN type algorithm. Such an algorithm is described in the publication He K. et al; “Mask R-CNN”, ArXiv:1703.06870v3, Jan 24, 2018. The objective is to segment the image, allowing to define bounding boxes around each object detected on the image , and assign a class to each object contained in a bounding box.

The architecture of the network is schematized in figure 4. The input data is the image captured by the camera. A convolutional neural network , formed of successive convolution layers, extracts the characteristics of an image, so as to obtain a map of the characteristics (Feature Map). The number of successive convolution layers is for example equal to 50. The map of the characteristics is used as the input data of a proposed network of regions ( – Region Proposal Network) to define regions of interest likely to contain an object. A region of interest alignment module , usually referred to as "ROIalign" makes it possible to establish, from the map of the characteristics , features for each region of interest. The characteristics associated with each region of interest are used:

• as input data of a fully connected perceptron , comprising three layers whose output data includes the coordinates of the bounding boxes as well as the object class assigned to each bounding box. For an object under consideration, a probability of belonging to each predefined class is determined. The class assigned to the object is the most likely class. The class assigned to each bounding box corresponds to a waste type among predefined waste types. For example, 9 classes have been considered: transparent bottle, opaque bottle, plastic jar, transparent plastic, colored plastic, cardboard packaging, paper, metal, glass. Glass is unwanted waste. Other classes can be added, for example relating to organic waste, or any other waste not classified in the previous classes.
• as input data of a convolution network so as to define a mask for each detected object inside a bounding box . A mask corresponds to the pixels, in a bounding box, belonging to the object: it is an estimation of the contour of each object. A mask identifies the object inside a bounding box. The image of the mask associated with an object considered, following the classification, as being undesirable, can be transmitted to the user.

The Mask-R-CNN algorithm has been trained in order to optimize the classification performance as well as the precision with which the edges of the object are evaluated. Mask definition precision can be quantified by an indicator of the IoU (Intersection over Union) type. This indicator corresponds to a ratio between the intersection and the union:

• of the outline of the bounding box encompassing the real object (Ground Truth – Ground Truth);
• the outline of the bounding box as defined by the algorithm.

The closer the IoU parameter is to 1, the higher the contour determination performance.

The training was performed by implementing 4500 training images. It has been tested using 500 validation images. Training was performed using the training image set 50 times in a row.

Figures 5A, 5C, and 5E show images of trash from which classifications were made to test the algorithm. Figures 5B, 5D and 5F show the respective images 5A, 5C and 5E, on which are materialized:

• the bounding boxes corresponding to each waste;
• the masks established in each bounding box, each mask materializing a piece of waste inside a bounding box;
• the class assigned to each waste, as well as the probability of belonging to said class (%).

In FIGS. 5B, 5D and 5F, the designations colored plastic, plastic pot, cardboard packaging, metal, opaque bottle, paper, transparent plastic, transparent bottle designate respectively colored plastic, plastic pot, cardboard packaging, metal, opaque bottle, paper , transparent plastic, transparent bottle.

The use of a neural network regionalized by Mask R-CNN masks makes it possible to precisely identify any non-compliant waste and to transmit the cause of the non-compliance to the user. Thus, in case of non-compliance, the user can receive an image representing the bounding box and/or the mask corresponding to the waste inside the bounding box, as well as the classification assigned to the waste. The user, because he has been identified, can thus receive relevant information about his waste sorting practice.

The classification performance of the algorithm was tested relative to the nine previously defined classes. The shows an AP indicator “Average Precision” (average precision – y-axis) established for each class (x-axis). The classes of the x-axis are: 1: opaque bottle – 2: colored plastic – 3: metal – 4: cardboard packaging – 5: paper – 6: transparent plastic – 7: transparent bottle – 8: glass – 9 -jar plastic. The indicator AP, known to those skilled in the art, is established by calculating an IoU parameter, as previously defined, for each classified object. A threshold IoU parameter is established. Objects for which the IoU parameter is greater than or equal to the threshold are considered true positives.

We used different images, in order to detect a rate of true positives (objects correctly detected – IoU value above the threshold), a false negative rate (undetected objects - IoU value below threshold - when they should have been detected) and a false positive rate (objects correctly detected when they should not have been).

By taking into account different IoU threshold parameters, for example between 0.5 and 0.95, we calculated:

• a precision , corresponding to the ratio
• a reminder , usually designated by the term “recall”, corresponding to the ratio

A curve corresponding to different values of precision P (ordinate axis) as a function of different recall values (RC) was established. The area of the curve corresponds to the average accuracy indicator AP.

Other types of R-CNN regional convolutional neural networks can also be used, for example networks usually referred to as "Faster-R-CNN", as described in the publication Ren. S “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”.

Claims (12)

Waste control device (1), the device being intended to be arranged overhanging a bin (40) to carry out a control of waste dumped into the bin, the device comprising

• a structure (20), comprising two vertical uprights, between which extends an empty space (24) intended to be occupied by the tray;
• an enclosure (10), supported by the structure, and arranged above the empty space, the enclosure comprising an admission opening (12), configured to allow an admission of a waste ( )inside the enclosure ;

the device being characterized in that the enclosure comprises:

• a tray (13), intended to receive the waste admitted through the admission opening;
• an evacuation opening (14), opening above the empty space, to allow evacuation of the waste, admitted into the enclosure, towards the container;

the device also comprising:

• a camera (30), configured to acquire an image ( ) of each waste disposed on the tray (13);
• an identification unit (31), configured to detect an identifier of a user of the device;
• a processing unit (32), programmed for:
  • classify the waste from the image ( ) acquired by the camera;
  • associate the classification made with the identifier of the user.

Device according to claim 1, in which the enclosure includes a tilting mechanism (15) allowing the tray to tilt between:

• a reception position, horizontal to within ±20°;
• an evacuation position, inclined with respect to the reception position, so that the plate (13) is oriented towards the evacuation opening (14), so that each piece of waste deposited on the plate slides towards the opening 'evacuation.

Device according to claim 2, comprising an inlet hatch (11), configured to pass from an open position, in which the inlet opening (12) is open, to a closed position, in which the opening of intake is closed, the tilting mechanism (15) being arranged such that:

• when the admission hatch is open, the tray occupies the receiving position;
• when the admission hatch is closed, and the camera (30) has acquired the image of each piece of waste placed on the plate, the plate occupies the evacuation position.

Device according to claim 3, comprising an inlet hatch opening mechanism (16), configured to keep the inlet hatch closed and to authorize an opening of the inlet hatch after the identification unit ( 31) detected the user ID. Device according to any one of Claims 2 to 4, in which the tilting mechanism (15) comprises a motorization to effect the tilting of the plate (13) between the reception position and the evacuation position and vice versa. Device according to any one of the preceding claims, in which the processing unit (32) is programmed to implement a supervised artificial intelligence algorithm, so as to detect non-compliance of the waste with respect to a criterion of acceptance, such that the waste is classified as:

• non-compliant when a non-compliance is detected;
• compliant in the absence of detection of non-compliance.

Device according to Claim 6, in which the processing unit (32) is configured to transmit a signal representative of the classification to a remote central unit (34), the signal transmitted comprising the identifier of the user detected by the identification unit. Device according to any one of claims 6 or 7, in which the supervised artificial intelligence algorithm implements a feature extraction module ( ) of the image acquired by the camera by convolutional neural network ( ). Device according to claim 8, in which the supervised artificial intelligence algorithm is of the regionalized convolutional neural network type. Device according to claim 9, in which the supervised artificial intelligence algorithm is of the mask regionalized convolutional neural network type. Method for classifying waste using a device (1) according to any one of the preceding claims, the method comprising:

1. detection of an identifier of a user, by the identification unit (31);
2. reception of the waste on the plate (13);
3. acquiring an image ( ) of the waste by the camera (30);
4. evacuation of the waste through the evacuation opening (14)
5. from the image acquired by the camera, classification of the waste by the processing unit (32);
6. association of the result of the classification with the identifier of the user.

Method according to claim 11, comprising:
g) transmission of the classification result to the user.