TWI839679B - Sorting of plastics - Google Patents

Abstract

Systems and methods for classifying and sorting of plastic materials utilizing a vision system and one or more sensor systems, which may implement a machine learning system in order to identify or classify each of the materials, which may then be sorted into separate groups based on such an identification or classification.

Description

Plastic Classification

The present invention relates generally to the sorting of solid waste and more particularly to the sorting of plastic chunks from municipal or industrial solid waste.

Plastic classification

This application claims priority to U.S. Provisional Patent Application No. 63/146,892 and U.S. Provisional Patent Application No. 63/173,301. This application is a continuation-in-part of U.S. Patent Application No. 17/495,291, which is a continuation-in-part of U.S. Patent Application No. 17/380,928, which is a continuation-in-part of U.S. Patent Application No. 17/227,245, which is a continuation-in-part of U.S. Patent Application No. 16/939,011, which is a continuation-in-part of U.S. Patent Application No. 16/375,675 (published as U.S. Patent No. 10,722,922), which is a continuation-in-part of U.S. Patent Application No. 15/963,755 (published as U.S. Patent No. 10,710,119), claiming priority to U.S. Provisional Patent Application No. 62/490,219, and which is a continuation-in-part of U.S. Patent Application No. 15/213,129 (published as U.S. Patent No. 10,207,296), claiming priority to U.S. Provisional Patent Application No. 62/193,332, all of which are hereby incorporated by reference. This application is also a continuation-in-part of U.S. Patent Application No. 17/491,415, which is a continuation-in-part of U.S. Patent Application No. 16/852,514, which is a divisional application of U.S. Patent Application No. 16/358,374 (published as U.S. Patent No. 10,625,304), which is a continuation-in-part of U.S. Patent Application No. 15/963,755 (published as U.S. Patent No. 10,710,119).

Government license

This invention was made with support from the U.S. Government under Grant No. DE-AR0000422 from the U.S. Department of Energy. The U.S. Government has certain rights in this invention.

This section is intended to introduce various aspects of the art that may be associated with exemplary embodiments of the present invention. It is believed that what is discussed here helps provide a framework to facilitate a better understanding of certain aspects of the present invention. Therefore, it should be understood that this section should be read from this perspective and not necessarily as an admission of prior art.

Recycling is the process of collecting and processing materials that would otherwise be thrown away as trash (e.g., from a waste stream) and transforming them into new products, or at least enabling more appropriate disposal. Recycling benefits communities and the environment by reducing the amount of waste going to landfills, conserving natural resources such as wood, water, and minerals, increasing economic security by utilizing domestic sources of materials, preventing pollution by reducing the need to collect new raw materials, and saving energy. After collection, recyclables may be sent to a material recovery facility (“MRF”) to be sorted, cleaned, and processed into materials that can be used in manufacturing. Therefore, a high-throughput automated sorting platform that can economically sort highly mixed waste streams would benefit a variety of industries. Therefore, there is a need for cost-effective sorting platforms that can identify, analyze and separate mixed industrial or municipal solid waste streams with high throughput to economically produce higher quality feedstock (which may also contain low levels of trace contaminants) for subsequent processing. Typically, MRFs are unable to distinguish between many materials, which limits the sorted materials to low-quality and lower-value markets, or are too slow, labor-intensive and inefficient, limiting the amount of material that can be economically recycled or recovered.

Municipal solid waste (“MSW”) is a broad term that covers waste streams from household, commercial, and industrial sources. Within each category, there are thousands of different materials and products. The U.S. Environmental Protection Agency reported that 267.8 million tons of MSW were generated in 2017. 35.37 million tons, or 13.2% of the total weight of that MSW, consisted of plastics. Of the 35.37 million tons of plastics, 2.96 million tons (8.4%) were recycled, 5.59 million tons (15.8%) were burned for energy recovery, and 26.82 million tons (75.8%) were landfilled. Clearly, more plastics need to be recycled.

Plastic recycling is the reprocessing of plastic waste into new, useful products. Recycling is necessary because almost all plastics are non-biodegradable and therefore accumulate in the environment. Currently, almost all recycling is done by remelting and reforming waste plastics into new products; so-called mechanical recycling. This causes the polymers to degrade at a chemical level and also requires sorting plastic waste by color and polymer type before reprocessing, which is both complex and expensive. Failure in this regard can result in inconsistent material properties, which is unattractive to industry. In an alternative, known as feedstock recycling, plastic waste is converted back to its starting chemicals, which can then be reprocessed into fresh plastic. This offers the promise of more recycling but is subject to higher energy and capital costs. Plastic waste can also replace fossil fuel burning as part of energy recovery.

Currently, only some plastics are recyclable. When plastics enter recycling, they are usually sorted into different types of plastics. Recycling rates also vary by plastic type. Several types are commonly used, each with different chemical and physical properties. This results in differences in how easily they can be sorted and reprocessed, which in turn affects the value and market size of the recycled material. Plastic packaging and products made from a single material, such as polyethylene terephthalate (“PET”), high-density polyethylene (“HDPE”), and polypropylene (“PP”), are more easily recycled. Plastics that are sometimes or almost never recycled include polyvinyl chloride (“PVC”), low-density polyethylene (“LDPE”), linear low-density polyethylene (“LLDPE”), and polystyrene (“PS”). In addition, plastics can only be recycled a limited number of times.

In modern single-stream MRFs and plastic recyclers, large volumes of feed material require handling equipment capable of moving and sorting material at high speeds. At the same time, obtaining the highest value from the purest, least contaminated stream. To achieve these somewhat conflicting goals, today’s single-stream MRFs and recyclers employ automated equipment to sort plastic packaging by near-infrared (“NIR”) signatures, either in transmission or reflection. These sensors rely on light reflections from an external light source and can only view the surface of the material. Furthermore, only polymer information is captured from this sensor. For example, NIR spectroscopy can identify type #1 clear plastics and light blue PET and #2 HDPE, while rejecting #1 colored PET, #3 PVC, #4 LDPE, #5 PP, #6 PS, and #7 other plastics such as multi-layer polymers, composite polymers, acrylics, and nylons. Additionally, NIR spectroscopy cannot accurately identify black or strongly colored plastics, as well as composite materials such as plastic-coated paper and multi-layer packaging (made of polymer multi-layer films), which can produce misleading readings. Most black plastics are colored with carbon. Black plastics are widely used in the automotive industry, electronics, food packaging, plastic bags, etc. But in addition to absorbing visible light, black plastics also absorb the near-infrared part of the spectrum, which unfortunately has the side effect of making the near-infrared spectrum invisible. Therefore, "invisible" black plastics go undetected into the "miscellaneous" waste bins at the end of the conveyor, which are burned for energy or dumped into landfills.

In closed-loop or primary recycling, waste plastics are recycled into new items of similar quality and kind (e.g., turning drinks bottles back into drinks bottles). However, the continuous mechanical recycling of plastics without loss of quality is very challenging due to the cumulative degradation of polymers and the risk of contaminant accumulation. Although closed-loop recycling of many polymers has been studied, the only industrial success to date has been PET bottle recycling.

In open-loop or secondary recycling (also known as downcycling), the quality of the plastic degrades each time it is recycled, so the material cannot be recycled indefinitely and eventually becomes waste. Recycling PET bottles into wool or other fibers is a common example and accounts for the majority of PET recycling. The reduction in polymer quality can be offset by blending recycled plastic with virgin material or compatible plastics when making new products.

Although thermoset polymers do not melt, technology has been developed for mechanical recycling. This generally involves breaking the material down into scraps, which can then be mixed with some kind of binder to form a new composite material.

In feedstock or tertiary recycling (also called chemical recycling), polymers are reduced to their chemical structural units (monomers) and can then be polymerized back into fresh plastic. Thermal depolymerization and chemical depolymerization are two types of feedstock recycling.

Energy recovery, also known as energy recovery or quad recycling, involves burning plastic waste to generate energy instead of fossil fuels.

A process has been developed whereby certain types of plastics can be used as a carbon source (instead of coke) in scrap steel recycling. In some applications, ground plastics can be used as building aggregate or fill material.

Plastic waste can be simply burned as refuse-derived fuel (“RDF”) in a waste-to-energy process, or it can be first chemically converted to synthetic fuels. In either approach, PVC must be excluded or compensated by installing dichlorination technology, as it produces large amounts of hydrogen chloride (HCl) when burned, which can corrode equipment and lead to undesirable chlorination of the fuel product.

Mixed plastic waste can be depolymerized to produce synthetic fuels. This has a higher calorific value than the starting plastic and can be burned more efficiently, although it is still less efficient than fossil fuels. Various conversion technologies have been investigated, of which pyrolysis is the most common. The use of catalysts in pyrolysis can result in more defined products with higher value. Plastic-to-fuel technology has historically been difficult to make economically viable due to the cost of collecting and sorting plastics and the relatively low value of the fuel produced compared to the widespread use of incineration.

For the reasons stated above, there is a need for improved methods for sorting all types of plastics, the ability to sort plastic types #3 to #7, the ability to sort PVC, and the ability to sort plastic mixtures into new classifications or fractions for more efficient recycling.

100:Classification system

101,201,401: Material block

102:Transmission system or funnel

103,203,403:Transmission system

104: Transmission system motor

105: Position detector

106: Drum/vibrator/sorter

107: Computer system

108:Automation control system

109,410:Camera

110: Visual system

111: Material block tracking device

112: Accompanying control system

120:Sensor system

121: Energy emission source

122: Power supply

124: Detector

125: Electronics

126,127,128,129: Classification device

136,137,138,139: Classification boxes

140: Box

300,800,3500:Program

301,302,303,304,305,801,802,803,804,805,806,807,3501, 3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,3512,3513: Programming Block

411:XRF system

412:NIR system

413:MWIR system

3400:Data processing system

3401: Graphics Processor Unit

3405: Local bus

3412: User interface adapter

3413:Keyboard

3414: Mouse

3415:Processor

3416:Display adapter

3420: Volatile memory

3425: Network (LAN) adapter

3430:I/O adapter

3431: Hard drive

3432:Tape drive

3435: Non-volatile memory

3440:Display

[Figure 1] shows a schematic diagram of a classification system configured according to some embodiments of the present invention.

[Figure 2] shows an exemplary representation of a control set of blocks of material used during the training phase in a machine learning system.

[Figure 3] shows a flow chart of some configurations according to the present invention.

[Figure 4] shows a simplified schematic diagram of the configuration of some embodiments of the present invention.

[Figures 5 and 6] show examples of chemical characterization.

[Figure 7] shows a flow chart of the configuration of some embodiments of the present invention.

[Figure 8] shows a flow chart of the configuration of some embodiments of the present invention.

[Figure 9] shows a block diagram of a data processing system configured according to some embodiments of the present invention.

[Content of invention] and [implementation method]

Various detailed embodiments of the present invention are disclosed herein. However, it should be understood that the disclosed embodiments are merely exemplary of the present invention, which may be implemented in various and alternative forms. The drawings are not necessarily drawn to scale; some features may be exaggerated or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching a person of ordinary skill in the art to employ various embodiments of the present invention.

As used herein, "material" may include any article or object, including but not limited to metals (ferrous and non-ferrous), metal alloys, plastics (including but not limited to any plastics known in the industry disclosed herein or newly created in the future), rubber, foam, glass (including but not limited to borosilicate or sodium calcium glass and various colored glasses), ceramics, paper, paperboard, Teflon, PE, bundled wires, insulated coated wires, rare earth elements, leaves, wood, plants, parts of plants, textiles, biowaste, packaging, electronic waste, batteries and accumulators, scrapped vehicles, mining, Construction and demolition waste, crop waste, forest residues, specialized grasses, woody energy crops, microalgae, municipal food waste, food waste, hazardous chemicals and biomedical waste, construction waste, farm waste, biological items, non-biological items, items with a specific carbon content, any other items that may be found in municipal solid waste, and any other items, items or materials disclosed herein, including further types or categories of any of the foregoing that can be distinguished from one another, including but not limited to by one or more sensor systems, including but not limited to any sensor technology disclosed herein.

"Material" may include any article or object composed of chemical elements, compounds or mixtures of chemical elements, or compounds or mixtures of compounds or mixtures of chemical elements, where the complexity of the compounds or mixtures can range from simple to complex. As used herein, "element" refers to a chemical element in the periodic table of elements, including elements that may be discovered after the filing date of this application. In the present invention, the terms "waste", "waste block", "material" and "block of material" may be used interchangeably.

As is well known in the industry, a "polymer" is a substance or material composed of very large molecules or macromolecules, composed of many repeated subunits. Polymers can be natural polymers found in nature or synthetic polymers.

A "multi-layer polymer film" is composed of two or more different compositions and may be up to about ^7.5-8 x ^10-4 m thick. The layers are at least partially adjacent and preferably but optionally coextensive.

As used herein, the terms "plastic", "plastic mass", and "mass of plastic material" (all of which may be used interchangeably) refer to any object comprising or consisting of a polymer composition of one or more polymers, and/or a multi-layer polymer film.

As used herein, the term "chemical signature" refers to a unique pattern (e.g., a fingerprint) generated by one or more analytical instruments that indicates the presence of one or more specific elements or molecules (including polymers) in a sample. The elements or molecules can be organic and/or inorganic. Such analytical instruments include any sensor system disclosed herein. According to embodiments of the present invention, one or more sensor systems disclosed herein can be configured to generate a chemical signature of a mass of material (e.g., a mass of plastic).

As used herein, "fraction" refers to any specific combination of organic and/or inorganic elements or molecules, polymer types, plastic types, polymer compositions, chemical characteristics of plastics, physical properties of plastic pieces (e.g., color, transparency, strength, melting point, density, shape, size, type of manufacturing, homogeneity, response to stimuli, etc.), etc., including any and all of the various classifications and types of plastics disclosed herein. A small number of non-limiting examples are one or more different types of plastic chunks, including: LDPE plus a relatively high percentage of aluminum; LDPE and PP plus a relatively low percentage of iron; PP plus zinc; a combination of PE, PET, and HDPE; any type of red LDPE plastic chunks; any combination of plastic chunks except PVC; black plastic chunks; a combination of #3-#7 types of plastics containing specific organic/inorganic molecules; a combination of one or more different types of multi-layer polymer films; a specific combination of plastics that do not contain specific contaminants or additives; any type of plastic with a melting point greater than a specified threshold; any thermosetting plastic of a plurality of specified types; a specified plastic that does not contain chlorine; a combination of plastics of specific similar density; a combination of plastics with similar polarity; a plastic bottle without a cap, or vice versa.

"Catalytic pyrolysis" involves the degradation of polymeric materials by heating them in the absence of oxygen and in the presence of a catalyst.

The term "predestination" refers to something that is established or determined in advance.

"Spectroscopic imaging" is imaging that uses multiple bands across the electromagnetic spectrum. While common cameras can capture light in three bands of the visible spectrum, namely red, green, and blue (RGB), spectral imaging encompasses a variety of techniques, including but not limited to RGB. Spectroscopic imaging can use infrared, visible, ultraviolet, and/or X-ray spectra, or some combination of these. Spectral data, or spectral image data, is a digital representation of a spectral image. Spectroscopic imaging may include the acquisition of spectral data in both visible and non-visible bands, illumination from outside the visible range, or the use of filters to capture specific spectral ranges. It is also possible to capture hundreds of bands for each pixel in a spectral image.

As used herein, an "image data packet" refers to a digital data packet associated with a captured spectral image of a single block of material.

As used herein, the terms "identify" and "classify", the terms "identification" and "classification" and any derivatives thereof may be used interchangeably. As used herein, to "classify" a block of material is to determine (i.e., identify) the type or category of material to which the block of material belongs. For example, according to certain embodiments of the present invention, a sensor system (as further described herein) may be configured to collect and analyze any type of information for classifying materials, which classifications may be used within a sorting system to selectively classify blocks of material based on variations in a set of one or more physical and/or chemical characteristics (e.g., which may be user-defined), including but not limited to color, texture, hue, shape, brightness, weight, density, composition, size, uniformity, type of manufacture, chemical characteristics, predetermined fractions, radioactive characteristics, transmittance to light, sound or other signals, and response to stimuli such as various fields, including emission and/or reflection of electromagnetic radiation ("EM") by the blocks of material. As used herein, "manufacturing type" refers to the type of manufacturing process used to make a block of material, such as a metal part formed by a forging process, has been cast (including but not limited to one-shot die casting, permanent die casting, and powder metallurgy), has been forged, a material removal process, etc.

The types or categories (i.e., classes) of materials may be user-definable and are not limited to any known material classifications. The particle size of a type or class may range from very coarse to very fine. For example, a type or class may include plastics, ceramics, glass, metals, and other materials, where the particle size of these types or classes is relatively coarse; different metals and metal alloys, such as zinc, copper, brass, chrome, and aluminum, where the particle size of these types or classes is finer; or between specific types of plastics, where the particle size of these types or classes is relatively fine. Thus, types or classes may be configured to distinguish between materials with significantly different compositions, such as different types of plastics (e.g., between any of types #1 through #7 of plastic), or to distinguish between materials of nearly identical composition, such as different subclasses of plastics that may belong to a specific plastic type. It should be understood that the methods and systems discussed herein can be used to accurately identify/classify blocks of material whose composition is completely unknown prior to classification.

Embodiments of the present invention improve plastic classification capabilities by fusing multiple sensor technologies and machine learning systems. Sensor-based classifier technology is limited by the use of a single sensor, as each sensor can only detect a narrow range of signals. The most common classifier sensor types are eddy current, visible camera, X-ray transmission, near infrared, and X-ray fluorescence ("XRF"), which are summarized in the table below.

However, plastic pieces in MSW can be composed of one or more organic polymers, one or more inorganic elements, and come in a variety of different colors, shapes, and sizes. Examples of these plastics include potato chip bags, squeezable juice boxes, select beverage containers, and electronic electromagnetic sensitive packaging. Embodiments of the present invention utilize sensor-based technology to generate new fractions from the waste stream that are able to sort these different types of plastics into unique classifications that can explain their organic polymer composition and/or inorganic element composition. For example, a transformation chemist who pays close attention to the relative composition of polymers and inorganic elements will be able to select one or more new fractions that can produce specific products from recycled plastics sorted into these fractions. As a result, a classification system configured according to an embodiment of the present invention can produce fractions beyond what is possible with existing state-of-the-art classification techniques.

For example, certain embodiments of the present invention may be configured to sort and/or classify a predetermined fraction from a #3-#7 type plastic bale to produce new products (e.g., via a recycling process) and/or fuel. Exemplary end uses of such fractions may include, but are not limited to, gases (e.g., C1-C4), fuels (e.g., gasoline, diesel), and vacuum gas oil. However, the classification of #3-#7 type plastics based on their organic and inorganic elemental composition has never been successfully accomplished.

Embodiments of the present invention may be configured to sort pieces of plastic material according to different predetermined combinations of sub-portions or features or types, which are disclosed below and elsewhere in the present invention.

Based on their properties, plastics can be divided into three categories based on their chemical structure, polarity and application.

Based on their chemical structure and temperature behavior, plastics can be divided into thermoplastics, thermosets and elastomers.

Regarding polarity, the presence of atoms of different natures causes electrons to move to the most electronegative atom in the covalent bond, thus generating a dipole. Polymers containing these extremely negative atoms (such as CI, O, N, F, etc.) will be polar compounds, which will have an impact on the properties of the material. If the polarity is increased, the mechanical resistance, hardness, rigidity, heat resistance, water and moisture absorption, chemical resistance, permeability and adhesion to polar compounds such as water vapor, and adhesion to metals will also increase. At the same time, the increase in polarity reduces thermal expansion, electrical insulation ability, the tendency of electrostatic charge accumulation, and the permeability of polar molecules (O ₂ , N ₂ ). In this way, different families can be distinguished, such as polyolefins, polyesters, acetals, halogenated polymers, etc.

The third classification (according to their application) applies to thermoplastic materials. There are four types of plastics in the third category: Standard plastics or commodities: plastics that are manufactured and used in large quantities due to their price and many favorable properties. Some examples are polyethylene ("PE"), polypropylene ("PP"), polystyrene ("PS"), polyvinyl chloride ("PVC"), or the copolymer acrylonitrile butadiene styrene ("ABS").

Engineering plastics: used when good structure, transparency, self-lubrication and thermal properties are required. Some examples are polyamide ("PA"), polyacetal ("POM"), polycarbonate ("PC"), polyethylene terephthalate ("PET"), polyphenylene ether ("PPE") and polybutylene terephthalate ("PBT").

Specialty plastics: those with special properties to an extraordinary degree, such as polymethyl methacrylate ("PMMA") with high transparency and light stability, or polytetrafluoroethylene (Teflon) with good temperature and chemical resistance.

High-performance plastics: mainly thermoplastics with high heat resistance. In other words, they have good mechanical properties at high temperatures, especially up to 150°C. Polyimide ("PI"), polysulfone ("PSU"), polyether sulfone ("PES"), polyarylene sulfone ("PAS"), polyphenylene sulfide ("PPS") and liquid crystal polymer ("LCP") are high-performance plastics.

Many plastic products carry symbols that identify the type of polymer from which they are made. These resin identification codes, often abbreviated as RIC, are used internationally. There are seven codes in total, six for the most common commercial plastic types and one for all others. These types are also referred to herein as polymer types #1-#7. Polymer type #1 refers to polyethylene terephthalate ("PET"), #2 refers to high-density polyethylene ("HDPE"), #3 refers to polyvinyl chloride ("PVC"), #4 refers to low-density polyethylene ("LDPE"), #5 refers to polypropylene ("PP"), #6 refers to polystyrene ("PS"), and #7 refers to other polymers not in polymer types #1-#6 (e.g., acrylic, polycarbonate ("PC"), polylactic acid, polylactide, nylon, fiberglass, ABS). The EU maintains a similar nine-digit list that also includes ABS and polyamide.

PET plastic is used to make many common household items, such as drink bottles, medicine jars, ropes, clothing, and carpet fibers. HDPE plastic is commonly used to make containers for milk, motor oil, shampoo and conditioner, soap bottles, detergents, and bleach. PVC is used in a variety of pipes and tiles, most commonly in plumbing. LDPE products include plastic wrap, sandwich bags, squeezable bottles, and plastic grocery bags. PP is used to make lunch boxes, margarine containers, yogurt bottles, syrup bottles, prescription bottles, and plastic bottle caps. Polystyrene items include disposable coffee cups, plastic food boxes, plastic cutlery, and packaging foam. Polycarbonate is used in baby bottles, compact discs, and medical storage containers. Thus, according to embodiments of the present invention, a vision system implemented as a machine learning system can be trained to discern and classify between these different types of plastics based on the types of products they have been made into.

Pieces of plastic can be sorted based on the types of additives they may contain. Additives are chemical compounds mixed into plastics to improve performance and include stabilizers, fillers, and dyes. Clear plastics have the highest value because they may not have been dyed, while black or dark plastics have a much lower value because they can cause discoloration in the product. Therefore, plastics may need to be sorted by polymer type and color to provide material suitable for recycling.

Plastics can also be classified according to density. Certain polymers have a similar density range (for example, PP and PE, or PET, PS and PVC). If a piece of plastic contains a high percentage of fillers, this may affect its density.

Plastic waste can also be roughly divided into two categories: industrial waste (sometimes called post-industrial resins) and post-consumer waste.

Plastic pieces can also be sorted according to how they are recycled. During mechanical recycling, plastics may be reprocessed at temperatures anywhere between 150-320°C, depending on the polymer type, which can lead to unwanted chemical reactions that cause polymer degradation. This can reduce the physical effectiveness and overall quality of the plastic and produce volatile low molecular weight compounds that can produce unpleasant tastes or odors and cause thermal discoloration. Therefore, embodiments of the present invention can be configured to sort plastic pieces to avoid such undesirable chemical reactions. Additives present in the plastic can accelerate this degradation. For example, oxygen-containing biodegradable additives intended to increase the biodegradability of plastics can increase the degree of thermal degradation. Similarly, flame retardants can also have an adverse effect. Thus, embodiments of the present invention may be configured to sort plastic pieces and thereby discard plastic pieces that have certain such additives.

The quality of the product may also depend greatly on how well the plastics are sorted. Many polymers are immiscible in each other when melted and phase separate during reprocessing (e.g., oil and water). Products made from such blends contain many boundaries between the different polymer types, and the cohesion across these boundaries is weak, resulting in poor mechanical properties. Therefore, embodiments of the present invention can be configured to sort the plastic pieces so that certain immiscible plastic pieces are not sorted together into the same group.

Systems and methods described according to certain embodiments of the present invention receive a heterogeneous mixture of a plurality of blocks of material (e.g., any combination of various plastics disclosed herein), wherein at least one block of material in the heterogeneous mixture includes a different elemental composition (e.g., chemical signature) than one or more other blocks of material and/or at least one block of material in the heterogeneous mixture is distinguishable (e.g., visually discernible properties or characteristics, different chemical signatures, etc.), and the systems and methods are configured to identify/sort the block of material into a group separate from other blocks of material of the same type. Embodiments of the present invention may be used to sort any type or class of materials or sub-portions as defined herein.

Embodiments of the present invention will be described herein as sorting blocks of material into separate groups by physically depositing (e.g., transferring or ejecting) the blocks of material into separate containers or bins according to user-defined groupings (e.g., material type classifications or sub-portions). As an example, in certain embodiments of the present invention, blocks of material may be sorted into separate bins to separate blocks of material having physical properties that are distinguishable from the physical properties of other blocks of material (e.g., visually discernible properties or characteristics, different chemical characteristics, etc.).

FIG. 1 illustrates an example of a sorting system 100 configured according to various embodiments of the present invention. A conveying system 103 may be implemented to convey one or more flows of individual material blocks 101 through the sorting system 100 so that each of the individual material blocks 101 can be tracked, sorted, and organized into predetermined desired groups. Such a conveying system 103 may be implemented with one or more conveyor belts on which the material blocks 101 travel, typically at a predetermined constant speed. However, certain embodiments of the present invention may be implemented with other types of conveying systems, including systems in which material blocks freely fall through various components of the sorting system 100 (or any other type of vertical sorter), or vibrating conveying systems. Hereinafter, the conveying system 103 may also be referred to as a conveyor belt 103, where applicable. In one or more embodiments, some or all of the actions of transmitting, stimulating, detecting, and classifying can be performed automatically, i.e., without human intervention. For example, in the classification system 100, one or more stimulus sources, one or more emission detectors, classification modules, classification devices, and/or other system components can be configured to automatically perform these and other operations.

Furthermore, although FIG. 1 depicts a single stream of chunks 101 of material on a conveyor system 103, embodiments of the present invention may be implemented in which a plurality of such streams of chunks of material pass through the various components of the sorting system 100 in parallel with one another. For example, as further described in U.S. Patent No. 10,207,296, the chunks of material may be distributed in two or more parallel single streams traveling on a single conveyor belt or a set of parallel conveyor belts. Thus, certain embodiments of the present invention are capable of simultaneously tracking and sorting multiple such parallel-traveling streams of chunks of material. According to certain embodiments of the present invention, a sorting machine need not be incorporated or used. Instead, a conveyor system (e.g., conveyor system 103) may simply convey a large number of material chunks that have been deposited on conveyor system 103 in a random manner.

According to certain embodiments of the present invention, some suitable feeder mechanism (e.g., another conveyor system or hopper 102) can be used to feed the material block 101 onto the conveyor system 103, whereby the conveyor system 103 conveys the material block 101 through various components within the sorting system 100. After the material block 101 is received by the conveyor system 103, an optional drum/vibrator/sorter 106 can be used to separate single material blocks from collections of material blocks. In certain embodiments of the present invention, the conveyor system 103 is operated by a conveyor system motor 104 to travel at a predetermined speed. The predetermined speed can be programmed and/or adjusted by an operator in any known manner. Monitoring of the predetermined speed of the conveyor system 103 can alternatively be performed using a position detector 105. In certain embodiments of the present invention, control of the transmission system motor 104 and/or the position detector 105 may be performed by an automated control system 108. Such an automated control system 108 may operate under the control of a computer system 107 and/or the functions for performing automated control may be implemented in software within the computer system 107.

The conveyor system 103 may be a conventional endless belt conveyor that employs a learned drive motor 104 adapted to move the belt conveyor at a predetermined speed. A position detector 105, which may be a learned encoder, is operably coupled to the conveyor system 103 and the automated control system 108 to provide information corresponding to the movement (e.g., speed) of the conveyor belt. Thus, as will be further described herein, by utilizing control of the conveyor system drive motor 104 and/or the automated control system 108 (and optionally including the position detector 105), since each block of material 101 traveling on the conveyor system 103 is identified, they can be tracked by position and time (relative to the various components of the sorting system 100), allowing the various components of the sorting system 100 to be activated/deactivated as each block of material 101 passes near them. As a result, the automated control system 108 is able to track the position of each block of material 101 as they travel along the conveyor system 103.

Referring again to FIG. 1 , certain embodiments of the present invention may utilize a vision or optical identification system 110 and/or a block tracking device 111 as a means of tracking each block 101 as it travels on the transport system 103. The vision system 110 may utilize one or more still or live cameras 109 to record the location (i.e., position and time) of each block 101 on the moving transport system 103. The vision system 110 may further or alternatively be configured to perform some type of identification (e.g., classification) of all or a portion of the blocks 101, as will be further described herein. For example, such a vision system 110 may be used to capture or obtain information about each block 101. For example, as described herein, the vision system 110 may be configured to capture or collect any type of information from the blocks (e.g., with a machine learning system) that may be used within the classification system 100 to classify and/or selectively classify the blocks 101 according to a set of one or more characteristics (e.g., physical and/or chemical and/or radiological, etc.). According to certain embodiments of the present invention, the vision system 110 may be configured to capture a visual image (including one-dimensional, two-dimensional, three-dimensional, or holographic imaging) of each block 101, such as by using an optical sensor used in conventional digital cameras and photographic equipment. The visual image captured by the optical sensor is then stored as image data (e.g., formatted as an image data packet) in a memory device. According to certain embodiments of the invention, such image data may represent images captured within optical wavelengths of light (i.e., wavelengths of light generally observable by the human eye). However, alternative embodiments of the invention may utilize a sensor system configured to capture images of materials composed of wavelengths of light outside of the wavelengths visible to the human eye.

According to certain embodiments of the present invention, the classification system 100 can be implemented with one or more sensor systems 120, which can be used alone or in combination with the vision system 110 to classify/identify the material block 101. The sensor system 120 may be configured to use any type of sensor technology useful for determining the chemical characteristics of the plastic pieces and/or classifying the plastic pieces for classification, including sensor systems that utilize radiated or reflected electromagnetic wave radiation (e.g., utilizing infrared (“IR”), Fourier transform IR (“FTIR”), forward looking infrared (“FLIR”), very near infrared (“VNIR”), near infrared (“NIR”), short wave infrared (“SWIR”), long wave infrared (“LWIR”), mid wave infrared (“MWIR” or “MIR”), X-ray transmission (“XRT”), gamma ray, ultraviolet (“UV”), X-ray fluorescence (“XRF”), radar, or other similar sensors). Induced breakdown spectroscopy ("LIBS"), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, and including one-, two-, or three-dimensional imaging with any of the foregoing), or any other type of sensor technology, including but not limited to chemical or radioactive. Implementations of exemplary XRF systems (e.g., used as sensor system 120 herein) are further described in U.S. Patent No. 10,207,296. XRF can be used in embodiments of the present invention to identify inorganic materials within a plastic mass (e.g., for inclusion in a chemical signature).

The following sensor system may also be used in certain embodiments of the present invention to determine the chemical characteristics of plastic pieces and/or to classify plastic pieces for classification.

Various forms of infrared spectroscopy previously disclosed can be used to obtain a specific chemical signature of each piece of plastic, which provides information about the base polymer of any plastic material as well as other components present in the material (mineral fillers, copolymers, polymer blends, etc.).

Differential Scanning Calorimetry ("DSC") is a thermal analysis technique that obtains the heat changes produced during heating of an analysis material specific to each material.

Thermogravimetric analysis ("TGA") is another thermal analysis technique that can provide quantitative information about the composition of plastic materials, including polymer percentage, other organic components, mineral fillers, carbon black, etc.

Capillary and rotational rheometers can determine the rheological properties of polymer materials by measuring their resistance to creep and deformation.

Optical and scanning electron microscopy ("SEM") can provide information about the structure of the material being analyzed, including the number and thickness of layers in multilayer materials (e.g. multilayer polymer films), the dispersed size of pigment or filler particles in the polymer matrix, coating defects, interface morphology between components, etc.

Chromatographic methods (e.g., LC-PDA, LC-MS, LC-LS, GC-MS, GC-FID, HS-GC) can quantify the trace components of plastic materials, such as UV stabilizers, antioxidants, plasticizers, anti-slip agents, etc., as well as residual monomers, residual solvents in inks or adhesives, and degradation substances.

It should be noted that while FIG. 1 depicts a combination of a vision system 110 and one or more sensor systems 120, embodiments of the present invention may be implemented using any combination of sensor systems of any sensor technology disclosed herein, or any other sensor technology currently available or developed in the future. Although FIG. 1 is depicted as including one or more sensor systems 120, implementation of such sensor systems is optional in certain embodiments of the present invention. In certain embodiments of the present invention, a combination of a vision system 110 and one or more sensor systems 120 may be used to classify a block of material 101. In certain embodiments of the present invention, any combination of one or more of the different sensor technologies disclosed herein may be used to classify a block of material 101 without using a vision system 110. Furthermore, embodiments of the present invention may include any combination of one or more sensor systems and/or vision systems, wherein the output of such sensor/vision systems is processed within a machine learning system (as further disclosed herein) to classify/identify materials from a heterogeneous mixture of materials, and then from each other.

According to an alternative embodiment of the present invention, the vision system 110 and/or sensor system may be configured to identify which blocks of material 101 are not of the type classified by the classification system 100 (e.g., blocks of plastic containing specific contaminants, additives, or undesirable physical characteristics (e.g., an attached container lid is made of a different type of plastic than the container)) and send a signal to reject such blocks of material. In such a configuration, the identified blocks of material 101 may be transferred/ejected using one of the mechanisms described below for physically transferring classified blocks of material into separate bins.

In certain embodiments of the present invention, a block tracking device 111 and accompanying control system 112 may be utilized and configured to measure the size and/or shape of each of the blocks 101 as they pass near the block tracking device 111 and the location (i.e., position and time) of each block 101 on the mobile conveyor system 103. U.S. Patent No. 10,207,296 further describes exemplary operations of such block tracking device 111 and control system 112. Alternatively, as previously disclosed, a vision system 110 may be used to track the location (i.e., position and time) of each block 101 as it is transported by the conveyor system 103. Thus, some embodiments of the present invention can be implemented without a material block tracking device (e.g., material block tracking device 111) to track the material block.

In certain embodiments of the present invention implementing one or more sensor systems 120, the sensor system 120 may be configured to assist the vision system 110 in identifying the chemical composition, relative chemical composition, and/or manufacturing type of each block of material 101 as the block of material 101 passes near the sensor system 120. The sensor system 120 may include an energy emitting source 121, which may be powered by a power source 122, for example, to stimulate a reaction from each block of material 101.

According to certain embodiments of the present invention in which an XRF system is implemented as sensor system 120, source 121 may include an inline X-ray fluorescent ("IL-XRF") tube, such as further described in U.S. Patent No. 10,207,296. Such an IL-XRF tube may include separate X-ray sources, each dedicated to one or more (e.g., unitized) streams of transported material blocks. In this case, one or more detectors 124 may be implemented as XRF detectors to detect fluorescent X-rays from the material blocks 101 within each unitary stream. Examples of such XRF detectors are further described in U.S. Patent No. 10,207,296.

1)個分類裝置126...129中的一個，用於根據判定的分類將材料塊101分類(例如，轉移/彈出)到一或多個N(N

1)個分類箱136...139中。四個分類裝置126...129及與分類裝置相關聯的四個分類箱136...139在圖1中僅作為非限制性實例繪示。 In certain embodiments of the present invention, as each block of material 101 passes near the emission source 121, the sensor system 120 may emit appropriate sensing signals to the block of material 101. One or more detectors 124 may be positioned and configured to sense/detect one or more characteristics from the block of material 101 in a manner appropriate to the type of sensor technology used. The one or more detectors 124 and associated detector electronics 125 capture these received sensed characteristics to perform signal processing thereon and generate digitized information (e.g., spectral data) representing the sensed characteristics, which is then analyzed according to certain embodiments of the present invention, which may be used to assist the vision system 110 in classifying each of the blocks of material 101. This classification, which may be performed within the computer system 107, may then be utilized by the automation control system 108 to enable the N (N

1) One of the sorting devices 126 ... 129 for sorting (e.g., transferring/ejecting) the material block 101 to one or more N (N

1) in four sorting boxes 136 ... 139. The four sorting devices 126 ... 129 and the four sorting boxes 136 ... 139 associated with the sorting devices are shown in FIG. 1 only as a non-limiting example.

Existing plastic sorters sort materials in a binary manner, where an air nozzle at the end of a conveyor ejects the identified plastic into one of two bins. For example, if four types of plastic need to be separated, the entire stream will need to be passed through such a binary sorter four different times, which takes four times as long as trying to remove a single object from the stream. According to an embodiment of the present invention, the sorting system 100 allows for sorting of multiple types of plastic at once.

The sorting device may include any known mechanism for redirecting the selected material block 101 to a desired location, including but not limited to transferring the material block 101 from a conveyor system to a plurality of sorting boxes. For example, the sorting device may utilize air ejectors, each of which is assigned to one or more sorts. When one of the air ejectors (e.g., 127) receives a signal from the automated control system 108, the air ejector emits a stream of air, causing the material block 101 to be transferred/ejected from the conveyor system 103 to the sorting box (e.g., 137) corresponding to the air ejector.

Other mechanisms may be used to transfer/eject the pieces of material, such as mechanically removing the pieces of material from the conveyor, pushing the pieces of material from the conveyor (e.g., using a paint brush type plunger), forming an opening (e.g., a gate) in the conveyor system 103 from which the pieces of material can fall, or using an air ejector to transfer the pieces of material into a separate bin as they fall from the edge of the conveyor. A pusher device (as used herein) may refer to any form of device that can be activated to dynamically move objects from a conveyor system/device, using pneumatic, mechanical or other means to do so, such as any suitable type of mechanical push mechanism (e.g., ACME screw drive), pneumatic push mechanism, or air ejector push mechanism. Some embodiments may include multiple pusher devices located at different locations and/or with different turn path orientations along the path of the conveyor system. In various different embodiments, the classification systems described herein can determine which pusher device (if any) to activate based on the classification of the material block performed by the machine learning system. In addition, the determination of which pusher device to activate can be based on the detected presence and/or characteristics of other objects that may also be in the transfer path of the pusher device at the same time as the target item. Furthermore, even for facilities with conveyor systems that do not separate perfectly along the way, the disclosed classification system can identify when multiple objects are not separated well and dynamically select which pusher should be activated from a plurality of pushers based on which pusher is likely to best fit the separation path for separating nearby objects. In some embodiments, objects identified as target objects may represent material that should be diverted out of the conveyor system. In other embodiments, objects identified as target objects represent material that should be allowed to remain on the conveyor system, allowing non-target material to be diverted.

In addition to the N classification boxes 136...139 into which the material blocks 101 are transferred/ejected, the classification system 100 may also include a receiver or box 140 that receives material blocks 101 that have not been transferred/ejected from the conveying system 103 into any of the aforementioned classification boxes 136...139. For example, when the classification of the material block 101 has not been determined (or simply because the classification device failed to adequately transfer/eject the block), the material block 101 may not be transferred/ejected from the conveying system 103 into one of the N classification boxes 136...139. Therefore, the box 140 can be used as a default container into which the unclassified material blocks are dumped. Alternatively, the box 140 can be used to receive material blocks of one or more classifications that are intentionally not assigned to any of the N classification boxes 136...139. These blocks of material can then be further classified according to other characteristics and/or by another classification system.

Depending on the diversity of the classifications of the desired material blocks, multiple classifications can be mapped to a single classification device and associated classification boxes. In other words, there does not need to be a one-to-one correlation between the classified boxes. For example, a user may expect certain categories of materials to be classified into the same classification box (e.g., different plastic types belonging to a small portion). To accomplish this classification, when material blocks 101 are classified as falling into a predetermined classification group (e.g., a small portion), the same classification device can be enabled to classify them into the same classification box. This combination classification can be used to generate any desired combination of classified material blocks. The mapping of the classifications can be programmed by the user (e.g., using a classification algorithm operated by the computer system 107 (e.g., see Figure 7)) to generate such desired combinations. Furthermore, the classification of blocks of material is user-definable and is not limited to any particular known block of material classification (e.g., the small subset disclosed herein).

The conveyor system 103 may include a circular conveyor (not shown) so that unsorted material blocks return to the starting point of the sorting system 100 and pass through the sorting system 100 again. In addition, because the sorting system 100 is able to specifically track each material block 101 as it travels on the conveyor system 103, some sorting device (e.g., sorting device 129) can be implemented to guide/eject material blocks 101 that the sorting system 100 fails to sort after a predetermined number of cycles through the sorting system 100 (or material blocks 101 are collected in the bin 140).

In some embodiments of the present invention, the conveyor system 103 may be divided into multiple belts (e.g., two belts) in a serial configuration, wherein the first belt conveys the material blocks through the vision system 110, and the second belt conveys certain classified material blocks through the implemented sensor system 120 for a second classification. In addition, such a second conveyor belt may be lower than the height of the first conveyor belt, so that the material blocks fall from the first conveyor belt to the second conveyor belt.

In some embodiments of the invention implementing sensor system 120, emission source 121 may be located above the detection region (i.e., above delivery system 103); however, some embodiments of the invention may position emission source 121 and/or detector 124 at other locations that still produce acceptable sensing/detection physical characteristics.

The systems and methods described herein may be used to classify and/or sort single blocks of material having any of a variety of sizes and shapes. Although the systems and methods described herein are primarily described with respect to classifying single blocks of material, the systems and methods described herein are not limited thereto. Such systems and methods may be used to simultaneously stimulate and/or detect emissions from a plurality of materials. For example, as opposed to a single stream of material being conveyed in series along one or more conveyor belts, multiple single streams may be conveyed in parallel. Each stream may be on the same belt or arranged in parallel on different belts. In addition, the blocks may be randomly distributed (e.g., across and along) on one or more conveyor belts. Thus, the systems and methods described herein may be used to simultaneously stimulate and/or detect emissions from a plurality of blocks of material. In other words, multiple blocks of material can be treated as a single block, rather than considering each block of material individually. Thus, multiple blocks of material can be sorted together (e.g., transferred/discharged from a conveyor system).

Although the systems and methods described herein are primarily described with respect to classifying blocks of material, such systems and methods are not limited to that use. They may be used in other applications, such as identifying elements (e.g., contaminants) within a block of material or determining the composition of a block of material.

As previously mentioned, certain embodiments of the present invention may implement one or more vision systems (e.g., vision system 110) to identify, track, and/or classify blocks of material. According to embodiments of the present invention, such vision systems may operate alone to identify and/or classify blocks of material, or may be combined with one or more sensor systems (e.g., sensor system 120) to identify and/or classify blocks of material. If a classification system (e.g., classification system 100) is configured to operate solely with such a vision system 110, sensor system 120 may be omitted from classification system 100 (or simply disabled).

Regardless of the type of sensory features/information captured from the blocks of material, the information (e.g., image data packets) can then be sent to a computer system (e.g., computer system 107) to be processed by a machine learning system to identify and/or classify each block of material. Such a machine learning system can implement any known machine learning system, including implementing neural networks (e.g., artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, automatic encoders, reinforcement learning, etc.), fuzzy logic, artificial intelligence ("AI"), deep learning algorithms, deep structured learning hierarchical learning algorithms methods, support vector machines (“SVM”) (e.g., linear SVM, nonlinear SVM, SVM regression, etc.), decision tree learning (e.g., classification and regression trees (“CART”), ensemble methods (e.g., ensemble learning, random forest, bagging and pasting, patching and subspace, boosting, stacking, etc.), dimensionality reduction (e.g., projection, manifold learning, principal component partitioning, etc.), Analysis, etc.) and/or deep machine learning algorithms, such as those described and publicly available on the deeplearning.net website (including all software, publications, and hyperlinks to available software referenced on this website), are hereby incorporated by reference into this document. Non-limiting examples of publicly available machine learning software and libraries that can be utilized in embodiments of the present invention include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet (MATLAB toolbox for implementing convolutional neural networks for computer vision applications), DeepLearnToolbox (Matlab toolbox for deep learning (from Rasmus Berg Palm)), BigDL, Cuda-Convnet (fast C++/CUDA implementation of convolutional (or more generally, feedforward) neural networks) Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-way factored RBM and mcRBM, mPoT (Python code to train natural image models using CUDAMat and Gnumpy), ConvNet, Elektronn, OpenNN, NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa, Lightnet and SimpleDNN.

According to certain embodiments of the invention, machine learning can be performed in two phases. For example, training occurs first, which can be performed offline because the classification system 100 is not used to perform actual classification of the blocks of material (e.g., see Figures 3-4). The classification system 100 can be used to train the machine learning system so that a homogeneous group of blocks of material (i.e., materials of the same type or category, or falling into the same predetermined sub-section) (also referred to herein as control samples) is passed through the classification system 100 (e.g., by the conveyor system 103); and all of these materials may not be classified, but may be collected in a common bin (e.g., bin 140). Alternatively, training may be performed at another location remote from the classification system 100, including using some other mechanism to collect sensory information (features) of a control set of material blocks. During this training phase, an algorithm in the machine learning system extracts features from the captured information (e.g., using image processing techniques known in the art). Non-limiting examples of training algorithms include, but are not limited to, linear regression, gradient descent, feedforward, polynomial regression, learning curves, regularized learning models, and logistic regression. It is during this training phase that the algorithms in the machine learning system learn the relationships between materials and their features/characteristics (e.g., captured by the vision system and/or sensor system) creating a knowledge base for later classification of heterogeneous mixtures of blocks of material received by the classification system 100, which can then be classified by a desired classification. Such a knowledge base may include one or more libraries, each of which includes parameters (e.g., neural network parameters) for use by the machine learning system in classifying blocks of material. For example, a particular library may include parameters configured by the training phase to identify and classify a particular type or class of materials, or one or more materials belonging to a predetermined subset. According to certain embodiments of the present invention, such a library may be input into a machine learning system, and a user of the classification system 100 may then be able to adjust certain parameters in order to adjust the operation of the classification system 100 (e.g., adjust the critical effectiveness of the machine learning system in identifying a particular material from a heterogeneous mixture of materials).

As shown in FIG2 , during a training phase, a plurality of material blocks 201 of one or more specific types, classifications, or fractions of materials of a reference sample may be passed through a vision system and/or one or more sensing systems (e.g., via a transmission system 203) so that an algorithm in a machine learning system can detect, extract, and learn features representative of such materials. For example, each material block 201 may be a separate plastic block of a specific type, category, or predetermined fraction, which, through such a training phase, enables an algorithm in a machine learning system to “learn” (train) how to detect, identify, and classify such plastic blocks accordingly. In the case of training a vision system (e.g., vision system 110), it is trained to visually discriminate blocks of material. This will build a parameter library specific to one or more specific types, classes, or fractions of plastic material. The same process can then be performed for blocks of plastic of different types, classes, or fractions, building parameter libraries specific to that type, class, or fraction, and so on. For each type, class, or fraction of plastic to be classified by the machine learning system, any number of exemplary blocks of plastic of that type, class, or fraction of plastic can be passed through the system. Given the captured sensory information as input data, the algorithm in the machine learning system can use N classifiers, each classifier testing one of the N different material types, classes, or fractions. Note that machine learning systems can be "taught" (trained) to detect any type, category, or subset of materials, including any type, category, or subset of materials found in MSW, or any other material disclosed herein.

After the algorithm is built and the machine learning system has sufficiently learned (trained) the differences (e.g., visually discernible differences) in material classifications (e.g., within a user-defined statistical confidence level), the library for the different material classifications is then implemented into a material classification system (e.g., classification system 100) for use in identifying and/or classifying material blocks from a heterogeneous mixture of material blocks (e.g., contained in MSW), and potentially classifying such classified material blocks if classification is to be performed.

Techniques for building, optimizing, and utilizing machine learning systems are known to those of ordinary skill in the art, as seen in the relevant literature. Examples of such literature include the following publications: Krizhevsky et al., “ ImageNet Classification with Deep Convolutional Networks ”, Proceedings of the 25th International Conference on Neural Information Processing Systems, December 3-6, 2012, Lake Tahoe, Nevada, and LeCun et al., “ Gradient-Based Learning Applied to Document Recognition ”, Proceedings of the IEEE Conference, Institute of Electrical and Electronics Engineers (IEEE), November 1998, both of which are incorporated herein by reference.

In one example technique, data captured by a vision or sensor system about a particular block of material may be processed as an array of data values (within a data processing system (e.g., data processing system 3400 of FIG. 9 ) executing (configured with) a machine learning system). For example, the data may be spectral data about a particular block of material captured by a digital camera or other type of sensor system and processed as an array of data values (e.g., image data packets). Each data value may be represented by a single number, or as a series of numbers representing values. These values may be multiplied by neuron-like weight parameters (e.g., using a neural network), and a bias may be added. This may be fed into the neuron-like nonlinearity. The resulting number output by the neuron can be processed like a value, multiplied by the subsequent neuron weight value, a bias is optionally added, and fed to the neuron nonlinearity again. Each iteration of this procedure is called a "layer" of the neural network. The final output of the last layer can be interpreted as the probability of the material being present or absent in the captured data associated with the block of material. Examples of this procedure are described in detail in the aforementioned "ImageNet Classification with Deep Convolutional Networks" and "Gradient-Based Learning Applied to Document Recognition" references.

According to certain embodiments of the present invention in which a neural network is implemented, as a final layer ("classification layer"), a final set of neuron outputs is trained to represent the likelihood that a block of material is associated with the captured data. During operation, if the likelihood that the block of material is associated with the acquired data exceeds a user-specified threshold, then the block of material is determined to be indeed associated with the captured data. These techniques can be extended to determine not only the presence of a material type associated with a particular captured data, but also whether a sub-region of a particular captured data belongs to one type of material or another type of material. This process is called segmentation, and techniques exist in the literature that use neural networks, such as those called "fully convolutional" neural networks, or networks that contain parts of convolutions (i.e., partial convolutions) if they are not fully convolutional. This allows the location and size of materials to be determined.

It should be understood that the present invention is not exclusively limited to machine learning techniques. Other commonly used techniques for material classification/identification may also be used. For example, the sensor system may utilize optical spectroscopy techniques using a multispectral or hyperspectral camera to provide a signal that indicates the presence or absence of a type, class, or fraction of a material by examining the spectral emissions of the material (i.e., spectral imaging). Spectral images of blocks of material may also be used in template matching algorithms, where a database of spectral images is compared to a captured spectral image to find the presence or absence of certain types of materials from the database. Histograms of captured spectral images may also be compared to a database of histograms. Similarly, bag-of-words models can be used with feature extraction techniques such as Scale-Invariant Feature Transform ("SIFT") to compare the extracted features between the captured spectral image and those in the database.

Thus, as disclosed herein, certain embodiments of the present invention provide for identification/classification of one or more different types, categories, or sub-portions of materials in order to determine which pieces of material should be diverted from a conveyor system in a defined group. According to certain embodiments, machine learning techniques are utilized to train (i.e., configure) a neural network to identify a plurality of one or more different types, categories, or sub-portions of materials. Spectral images or other types of sensory information of the materials are captured (e.g., traveling on a conveyor system), and based on such identification/classification of such materials, the systems described herein can determine which pieces of material should be allowed to remain on the conveyor system and which should be diverted/removed from the conveyor system (e.g., into a collection bin, or transferred to another conveyor system).

According to certain embodiments of the present invention, a machine learning system (e.g., classification system 100) used in an existing device can be dynamically reconfigured to recognize/classify features of new types, categories, or subsets of materials by replacing a current set of neural network parameters with a new set of neural network parameters.

It is important to mention here that, according to certain embodiments of the present invention, the detected/captured features/characteristics of a piece of material (e.g., a spectral image) are not necessarily simply specifically recognizable or identifiable physical properties; they may be abstract formulas that can only be expressed mathematically, or not at all; however, a machine learning system may be configured to parse the spectral data to find patterns that allow for classification of control samples during the training phase. Furthermore, the machine learning system may obtain sub-portions of the captured information (e.g., a spectral image) of a piece of material and attempt to find correlations between predefined classifications.

According to certain embodiments of the present invention, instead of using a control sample of a material block through a training phase passed by a vision system and/or a sensor system, the training of the machine learning system can utilize labeling/annotation technology, when the vision/sensor system captures data/information of the material block, the user inputs a label/annotation identifying each material block, which is then used to build a library for the machine learning system to use when classifying material blocks in a heterogeneous mixture of material blocks.

3-6, embodiments of the present invention combine or fuse multiple sensor technologies (e.g., any combination of visual ("VIS"), XRF, NIR, and MWIR) in a manner that uniquely identifies various types, categories, or small portions of plastics so that they can be classified by their organic and inorganic chemical components. However, because these pieces of plastic in MSW have many different sizes and shapes, the signals generated by these different sensors may have a large degree of variation between them. Therefore, machine learning combined with the fusion of various sensor technologies can improve the classification accuracy of these signals even in the presence of such large variations. Because implementing multiple different sensors in a system may increase the cost of the system and also reduce sorting speed, certain embodiments of the present invention may implement a system (e.g., sorting system 100) with a reduced number of sensor systems (and thus lower capital and operating costs) to improve economic feasibility while still being able to adequately sort material.

FIG. 4 shows a simplified schematic diagram of a system (e.g., classification system 100) in which blocks of material (e.g., plastic blocks) 401 are transported by a transport system 403 through a sensor system that captures spectral data from each block of material 401. In this non-limiting example, the sensor system is a camera 410 (e.g., vision system 110) that captures visible image data of each block of material 401, an XRF system 411, a NIR system 412, and a MWIR system 413. However, it should be noted that any other sensor system disclosed herein may be used in any combination.

3 and 4, the chemical characteristics of the material are determined using one or more sensor systems in program block 301. The sensed/detected/captured signals from the sensor systems are combined (e.g., in a multidimensional data array) for each piece of material to establish a chemical characteristic. Recall that an XRF sensor system is capable of determining the presence of inorganic elements or molecules in a piece of plastic, while a combination of one or more other sensor systems (e.g., NIR and MWIR) is capable of determining the presence of organic elements or molecules in a piece of plastic. In program block 302, a visible image of each piece of material is captured. In program block 303, the captured visible image of each piece of material (i.e., its associated image data) is correlated with its determined chemical characteristics (i.e., spectral image data). Figures 5 and 6 show non-limiting exemplary representations of chemical signatures and associated image data for two different types of plastic materials - potato chip bags and electronic packaging. It can be easily seen that different types or categories of plastic pieces will have different (unique) chemical signatures, which are used in embodiments of the present invention to generate fractions and/or classifications (which may be user-defined) of plastic waste. According to embodiments of the present invention, a control set of plastic pieces of a particular type or category can be run through the system shown in Figure 4 to train the machine learning system to associate a particular chemical signature with a particular type or category of plastic pieces.

For example, with respect to the example shown in FIG. 5 , images captured from multiple potato chip bags (which may include bags of different physical conditions or orientations, or even bags associated with different brands of chips and/or manufacturers) may be processed to train a machine learning system.

Block 304 may involve separating the plastic mass into one or more sub-fractions. There are many ways to create these sub-fractions. One approach is to create a first layer based on major elements and then create a second and even third layer based on minor elements. For example, sub-fractions may be first determined by polymer type and then branched into inorganic elements such as aluminum and zinc. Other exemplary sub-fractions may then be created for blends of polymers and then branched into their inorganic elemental compositions. There are also computational methods that can perform this type of clustering to determine sub-fractions, such as principal component analysis, K-means clustering, and unsupervised and semi-supervised learning. Sub-fractions are further defined herein.

In process block 305, after the fractions have been determined, the plastic pieces associated with the fractions can be sorted (e.g., manually) to create a control set for each fraction. Because each fraction is measured with a sensor system, each control set contains chemical information about the piece. A vision system (e.g., vision system 110) can be used to train a machine learning system to recognize those fractions. Using this method, the chemical information in the plastic is converted into visual features, and the machine learning system can learn to classify it. And when the classification system 100 is used to classify based on visual images, it also separates plastics by chemical composition. This method is effective when two objects look different and have different chemical compositions. When two objects appear identical or very similar and have different chemical compositions, classification can be performed using two or more sensor systems (e.g., VIS plus XRF, etc.).

Since the determined fractions may constitute any desired types of specified organic and/or inorganic elements or molecules, process 300 may be used to train a machine learning system implemented within the classification system so that it is configured to classify a heterogeneous mixture of different plastic chunks to produce at least a fraction of the plastic chunks containing one or more different types or categories. For example, if the machine learning system has been trained to recognize any plastic chunks containing a specific combination of organic and/or inorganic elements or molecules, then when the classification is completed, the classified fractions may not contain exactly the same plastic chunks (i.e., multiple plastic potato chip bags belong to different brands of potato chips because each plastic potato chip bag is composed of organic and/or inorganic elements or molecules defined by a predetermined fraction).

FIG. 7 depicts a flow chart depicting an exemplary embodiment of a process 3500 for classifying a mass of material using a vision system and/or one or more sensor systems according to certain embodiments of the present invention. The process 3500 may be performed to classify a heterogeneous mixture of plastic masses into any combination of predetermined types, categories, and/or fractions. The process 3500 may be configured to operate in any embodiment of the present invention described herein, including the classification system 100 of FIG. 1 . The operations of the process 3500 may be performed by hardware and/or software, including within a computer system (e.g., the data processing system 3400 of FIG. 9 ) of a control system (e.g., the computer system 107 of FIG. 1 , the vision system 110, and/or the sensor system 120). In process block 3501, blocks of material may be placed on a conveyor system. In process block 3502, the position of each block of material on the conveyor system is detected so that each block of material can be tracked as it travels through the sorting system 100. This may be performed by the vision system 110 (e.g., by distinguishing the blocks of material from the conveyor system material below when communicating with a conveyor system position detector (e.g., position detector 105)). Alternatively, a material tracking device 111 may be used to track the blocks. Alternatively, any system that can generate a light source (including but not limited to visible light, UV, and IR) and has a detector that can be used to locate the blocks. In program block 3503, sensory information/characteristics of the block of material are captured/acquired when the block of material has traveled close to one or more of the vision system and/or sensor system. In program block 3504, the vision system (e.g., implemented within computer system 107) (as previously disclosed) can perform pre-processing of the captured information, which can be used to detect (extract) information of each block of material (e.g., from the background (e.g., conveyor belt)): in other words, pre-processing can be used to identify the difference between the block of material and the background. Known image processing techniques (e.g., dilation, thresholding, and contouring) can be used to identify the block of material as different from the background. In program block 3505, segmentation can be performed. For example, the captured information may include information relating to one or more blocks of material. In addition, a particular block of material may be located on a seam of a conveyor belt when its image is captured. Therefore, in this case, it may be necessary to isolate the image of the single block of material from the background of the image. In the exemplary technique of program block 3505, the first step is to apply a high contrast image; in this way, the background pixels are reduced to essentially all black pixels, and at least some of the pixels associated with the block of material are illuminated to essentially all white pixels. The image pixels of the white block of material are then expanded to cover the entire size of the block of material. After this step, the location of the block of material is a high contrast image of all white pixels on a black background. A contour algorithm can then be used to detect the boundaries of the block of material. The boundary information is saved and the boundary location is then transferred to the original image. Segmentation is then performed on the original image over regions larger than the previously defined boundaries. In this way, blocks of material are identified and separated from the background.

In optional program block 3506, the material blocks can be conveyed along the conveyor system in the vicinity of a material block tracking device and/or sensor system to track each material block and/or determine the size and/or shape of the material block, which may be useful if an XRF system or some other spectroscopic sensor is also implemented in the classification system. In program block 3507, post-processing can be performed. Post-processing may involve adjusting the size of the captured information/data in preparation for use in the machine learning system. This may also include modifying certain attributes (e.g., enhancing image contrast, changing the image background, or applying filters) to enhance the ability of the machine learning system to classify the material blocks. In program block 3509, the size of the data can be adjusted. In some cases, it may be necessary to adjust the data size to match the data input requirements of certain machine learning systems (e.g., neural networks). For example, a neural network may require image sizes that are much smaller than the size of images captured by a typical digital camera (e.g., 225x255 pixels or 299x299 pixels). In addition, the smaller the input data size, the less processing time is required to perform the classification. Therefore, smaller data sizes can ultimately increase the throughput of the classification system 100 and increase its value.

In program blocks 3510 and 3511, each block of material is identified/classified based on the sensed/detected features. For example, program block 3510 may be configured with a neural network that employs one or more machine learning algorithms that compares the extracted features with features stored in a previously generated knowledge base (e.g., generated during a training phase) and assigns the classification with the highest degree of match to each block of material based on this comparison. The algorithm of the machine learning system may process the captured information/data in a hierarchical manner by using automatically trained filters. The filter responses are then successfully combined in the next level of the algorithm until a probability is obtained in the final step. In program block 3511, these probabilities may be applied to each of the N classifications to determine into which of the N classification containers the respective block of material should be classified. For example, each of the N classifications may be assigned to a classification container, and the block of material under consideration is classified into the container corresponding to the classification that returns the highest probability greater than a predetermined threshold. In an embodiment of the present invention, such a predetermined threshold may be pre-set by a user. If none of the probabilities is greater than the predetermined threshold, the particular block of material may be classified into an abnormal container (e.g., classification container 140).

Next, in block 3512, one or more sorting devices corresponding to the classification of the block of material are activated. Between the time the image of the block of material is captured and the time the sorting device is activated, the block of material has moved from the vicinity of the vision system and/or sensor system to a location downstream of the conveyor system (e.g., at the conveyor system's conveying speed). In an embodiment of the present invention, the activation of the sorting device is timed so that when the block of material passes through the sorting device mapped to the classification of the block of material, the sorting device is activated and the block of material is transferred/ejected from the conveyor system to its associated classification container. In an embodiment of the present invention, activation of the sorting device can be timed by individual position detectors, which detect when a material block passes in front of the sorting device and send a signal to activate the sorting device. In program block 3513, the sorting container corresponding to the activated sorting device receives the transferred/ejected material block.

FIG8 depicts a flow chart depicting an exemplary embodiment of a process 800 for classifying blocks of material according to certain embodiments of the present invention. Process 800 may be configured to operate in any embodiment of the present invention described herein, including classification system 100 of FIG1. Process 800 may be configured to operate in conjunction with process 3500. For example, according to certain embodiments of the present invention, process blocks 803 and 804 may be incorporated into process 3500 (e.g., operated in series or in parallel with process blocks 3503-3510) in order to combine the efforts of vision system 110 implemented in conjunction with a machine learning system with a sensor system not implemented in conjunction with a machine learning system (e.g., sensor system 120) to classify and/or organize materials.

The operations of process 800 may be performed by hardware and/or software, including within a computer system (e.g., data processing system 3400 of FIG. 9 ) of a control system (e.g., computer system 107 of FIG. 1 ). In process block 801 , a block of material may be deposited on a conveyor system. Next, in optional process block 802 , the block of material may be conveyed along the conveyor system in the vicinity of a block tracking device and/or an optical imaging system in order to track each block of material and/or determine the size and/or shape of the block of material. In process block 803 , once the block of material has traveled in the vicinity of a sensor system, the block of material may be interrogated or stimulated with EM energy (waves) or some other type of stimulation appropriate to the particular type of sensor technology used by the sensor system. In process block 804, physical properties of the blocks of material are sensed/detected and captured by the sensor system. In process block 805, for at least some of the blocks of material, the type of material is identified/classified based (at least in part) on the captured features, which may be combined with the classification performed by the machine learning system in conjunction with the vision system 110.

Next, if the material block is to be classified, then in program block 806, one or more classification devices corresponding to the material block are activated. Between the time when the material block is sensed and the time when the classification device is activated, the material block has moved from the vicinity of the sensor system to a position downstream of the conveyor system at the conveying speed of the conveyor system. In some embodiments of the present invention, the activation of the classification device is timed so that when the material block passes through the classification device mapped to the classification of the material block, the classification device is activated and the material block is transferred/ejected from the conveyor system to its associated classification container. In some embodiments of the present invention, activation of the sorting device can be timed by a respective position detector, which detects when a block of material passes in front of the sorting device and sends a signal to activate the sorting device. In program block 807, the sorting container corresponding to the activated sorting device receives the transferred/ejected block of material.

According to certain embodiments of the present invention, multiple at least portions of the classification system 100 may be serially linked together to perform multiple generations or classification layers. For example, when two or more sorting systems 100 are linked in this manner, the conveying system can be implemented with a single conveyor belt or multiple conveyor belts, conveying material blocks through a first vision system (and, in some embodiments, a sensor system) configured to sort material blocks of a first group of heterogeneous material mixtures into a first set of one or more containers (e.g., sorting bins 136...139) by a sorter (e.g., a first automated control system 108 and associated one or more sorting devices 126...129), and then conveying the material blocks through a second vision system (and, in some embodiments, another sensor system), configured to sort material blocks of a second group of heterogeneous material mixtures into a second set of one or more sorting bins by a second sorter. Further discussion of this multi-stage classification is found in U.S. published patent application No. 2022/0016675, which is hereby incorporated by reference herein.

Such a continuous classification system 100 may include any number of such systems connected together in this manner. According to certain embodiments of the present invention, each continuous system may be configured to classify materials of a different classification or type than the previous system.

According to various embodiments of the present invention, different types, categories, or sub-portions of materials can be classified by different types of sensors, each used in a machine learning system, and combined to classify waste materials or blocks of material in a waste stream.

According to various embodiments of the present invention, data (e.g., spectral data) from two or more sensors can be combined using a single or multiple machine learning systems to classify a block of material.

According to various embodiments of the present invention, multiple sensor systems can be installed on a single transport system, each sensor system utilizing a different machine learning system. According to various embodiments of the present invention, multiple sensor systems can be installed on different transport systems, each sensor system utilizing a different machine learning system.

Certain embodiments of the present invention may be configured to produce, after classification, a bulk material having a content of a certain element or material that is less than a predetermined weight percentage or volume percentage.

According to various embodiments of the present invention, any combination of different types of sensor systems may be used to identify/classify and potentially categorize materials as disclosed herein. For example, each imaging or spectroscopic sensor as disclosed herein may be used to generate data from sensed information/properties of a block of material for processing by a machine learning system specific to that sensor system. Alternatively, any sensor system may be used without processing by a machine learning system, or with processing by a machine learning system, or a combination of both.

According to various embodiments of the present invention, different types, categories and/or fractions of materials can be classified by different types of sensor systems, each of which is used in a machine learning system and combined to classify blocks of material in a waste stream.

Referring now to FIG. 9 , a block diagram is depicted illustrating a data processing (“computer”) system 3400 in which aspects of embodiments of the present invention may be implemented. (The terms “computer,” “system,” “computer system,” and “data processing system” are used interchangeably herein.) Aspects of computer system 107, automation control system 108, sensor system 120, and/or vision system 110 may be configured similarly to data processing system 3400. Data processing system 3400 may employ a local bus 3405 (e.g., a Peripheral Component Interconnect (“PCI”) local bus architecture). Any suitable bus architecture may be used, such as Accelerated Graphics Port (“AGP”) and Industry Standard Architecture (“ISA”), among others. One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., through a PCI bridge (not shown)). An integrated memory controller and cache memory may be coupled to the one or more processors 3415. The one or more processors 3415 may include one or more central processing units and/or one or more graphics processing units 3401 and/or one or more tensor processing units. Additional connections to the local bus 3405 may be made through direct component interconnection or through add-in boards. In the depicted example, communication (e.g., network (LAN)) adapter 3425, I/O (e.g., small computer system interface ("SCSI") host bus) adapter 3430, and expansion bus interface (not shown) can be connected to local bus 3405 by direct assembly. Audio adapter (not shown), graphics adapter (not shown), and display adapter 3416 (coupled to display 3440) can be connected to local bus 3405 (e.g., via an add-in board inserted into an expansion slot).

User interface adapter 3412 may provide connections for keyboard 3413 and mouse 3414, a modem (not shown), and additional memory (not shown). I/O adapter 3430 may provide connections for hard disk drive 3431, tape drive 3432, and CD-ROM drive (not shown).

The operating system may run on one or more processors 3415 and is used to coordinate and provide control of various components within the data processing system 3400. In FIG. 9 , the operating system may be a commercially available operating system. An object-oriented programming system (e.g., Java, Python, etc.) may run with the operating system and provide calls to the operating system from one or more programs (e.g., Java, Python, etc.) executing on the data processing system 3400. Instructions for the operating system, the object-oriented operating system, and the programs may be located on a non-volatile memory 3435 storage device (e.g., hard drive 3431) and may be loaded into the volatile memory 3420 for execution by the processor 3415.

A person of ordinary skill in the art will appreciate that the hardware in FIG. 9 may vary depending on the implementation. Other internal hardware or volatile devices (e.g., flash ROM (or equivalent non-volatile memory) or optical disk drives, etc.) may be used to supplement or replace the hardware shown in FIG. 9. In addition, any processing of the present invention may be applied to a multi-processor computer system, or performed by a plurality of such data processing systems 3400. For example, training of a machine learning system may be performed by a first data processing system 3400, while operation of a classification system 100 for classification may be performed by a second computer data processing system 3400.

As another example, data processing system 3400 may be a stand-alone system configured to be bootable without reliance on a certain type of network communication interface, regardless of whether data processing system 3400 includes a certain type of network communication interface. As a further example, data processing system 3400 may be an embedded controller configured with ROM and/or flash ROM to provide non-volatile memory for storing operating system files or user-generated data.

The example depicted in FIG. 9 and the examples described above are not meant to imply architectural limitations. In addition, the computer program form of aspects of the present invention may reside on any computer-readable storage medium (e.g., floppy disk, optical disk, hard disk, tape, ROM, RAM, etc.) used by a computer system.

As described herein, embodiments of the present invention may be implemented to perform the various functions described for identifying, tracking, classifying and/or categorizing blocks of material. Such functions may be implemented in hardware and/or software, such as in one or more data processing systems (e.g., data processing system 3400 of FIG. 9 ), such as the aforementioned computer system 107, vision system 110, sensor system 120, and/or automation control system 108. However, the functions described herein are not limited to implementation in any particular hardware/software platform.

As will be understood by those of ordinary skill in the art, aspects of the present invention may be implemented as systems, procedures, methods, and/or computer program products. Thus, various aspects of the present invention may take the form of an all-hardware embodiment, an all-software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which are generally referred to herein as "circuits," "circuitry," "modules," or "systems." Additionally, aspects of the present invention may take the form of a computer program product in one or more computer-readable storage media having computer-readable program code implemented thereon. (However, any combination of one or more computer-readable media may be used. The computer-readable media may be computer-readable signal media or computer-readable storage media.)

The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, biological, atomic, or semiconductor system, apparatus, controller, or device, or any suitable combination of the foregoing, where the computer-readable storage medium itself is not a transient signal. More specific examples of computer-readable storage media (a non-exhaustive list) may include the following: an electrical connection having one or more lines, a portable computer disk, a hard drive, a random access memory ("RAM") (e.g., RAM 3420 of FIG. 9 ), a read-only memory ("ROM") (e.g., ROM 3435 of FIG. 9 ), an erasable programmable read-only memory ("EPROM" or flash memory), an optical fiber, a portable compact disc read-only memory ("CD-ROM"), an optical storage device, a magnetic storage device (e.g., hard drive 3431 of FIG. 9 ), or any suitable combination of the foregoing. In the context of this document, computer-readable storage media can be any tangible media that can contain or store programs for use by or in conjunction with an instruction execution system, apparatus, controller, or device. Program codes implemented on computer-readable signal media can be transmitted using any appropriate media, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer-readable signal media may include a transmitted data signal having computer-readable program code implemented thereon, such as in a baseband or carrier wave portion. Such transmitted signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination. Computer-readable signal media may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, device, controller, or device.

The flowcharts and block diagrams in the figures illustrate the possible implementation architecture, functions, and operations of the systems, methods, programs, and program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or code portion, which includes one or more executable program instructions for implementing the specified logical functions. It should also be noted that in some embodiments, the functions labeled in the blocks may not appear in the order labeled in the figure. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functions involved.

In the description herein, the flowchart technique may be described as a series of sequential actions. The order of the actions and the party performing the actions may be freely changed without departing from the scope of the teachings herein. Actions may be added, deleted, or altered in several ways. Similarly, actions may be reordered or looped. Furthermore, although a procedure, method, algorithm, etc. may be described in sequence, such procedure, method, algorithm, or any combination thereof may be operable to be performed in an alternate sequence. Furthermore, some actions within a procedure, method, algorithm may be performed simultaneously (e.g., actions performed in parallel) during at least one point in time, or may be performed as a whole, in whole, in part, or in any combination thereof.

A module implemented in software for execution by various types of processors (e.g., GPU 3401, CPU 3415) may, for example, include one or more physical or logical blocks of computer instructions, which may, for example, be organized into objects, procedures, or functions. However, an identified module's executable need not be physically located together, but may include different instructions stored in different locations that, when logically joined together, comprise the module and achieve the module's stated purpose. Indeed, a module of executable code may be a single instruction or many instructions and may even be distributed over many different code segments, over different programs, and across many memory devices. Similarly, operational data (e.g., the material classification library described herein) may be identified and instantiated herein in a module and may be implemented in any suitable form and organized in any suitable type of data structure. The operational data may be collected as a single data set or may be distributed among different locations, including in different storage devices. The data may be provided as an electronic signal on a system or network.

These program instructions may be provided to one or more processors and/or controllers of a general purpose computer, a special purpose computer, or other programmable data processing device (e.g., a controller) to produce a machine, so that the instructions executed by the processor (e.g., GPU 3401, CPU 3415) of the computer or other programmable data processing device create blocks or multiple blocks for implementing the functions/actions and/or block diagram blocks specified in the flowchart.

It should be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by a dedicated hardware-based system (e.g., which may include one or more graphics processing units (e.g., GPU 3401)) or a combination of dedicated hardware and computer instructions. For example, a module can be implemented as a hardware circuit, including a custom VLSI circuit or gate array, an existing semiconductor (e.g., a logic chip, a transistor, a controller, or other discrete components). A module can also be implemented in a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

Computer program code (i.e., instructions) for performing operations of aspects of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, Python, C++, etc., known procedural programming languages, such as the "C" programming language or similar programming languages, or any machine learning software disclosed herein. The program code may be executed entirely on the user's computer system, partially on the user's computer system as a stand-alone software package, partially on the user's computer system (e.g., a computer system used for classification) and partially on a remote computer system (e.g., a computer system used for training a sensor system), or entirely on a remote computer system or server. In the latter case, the remote computer may be connected to the user's computer system through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer system (for example, through the Internet using an Internet service provider).

These program instructions may also be stored in a computer-readable storage medium that can instruct a computer system, other programmable data processing device, controller, or other device to act in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture that includes instructions for implementing the functions/actions specified in the flowchart and/or block diagram block or blocks.

Program instructions may also be loaded onto a computer, other programmable data processing device, controller, or other device to cause a series of operating steps to be performed on the computer, other programmable device, or other device to produce a computer-implemented program, so that the instructions executed on the computer or other programmable device provide a program for implementing the functions/actions specified in or in the blocks of the flowchart and/or block diagram.

One or more databases may be included in the host computer for storing and providing access to data of various implementations. It will also be understood by those skilled in the art that for security reasons, any database, system, or component of the present invention may include any combination of databases or components located in a single location or multiple locations, wherein each database or system may include any variety of suitable security features, such as firewalls, access codes, encryption, decryption, etc. The database may be any type of database, such as relational, hierarchical, object-oriented, etc. Common database products that may be used to implement the database include IBM's DB2, any database product available from Oracle Corporation, Microsoft Access from Microsoft Corporation, or any other database product. The database may be organized in any suitable manner, including as a table or a lookup table.

The association of certain data (e.g., for each material block of the classification system described herein) can be accomplished by any data association technique known and practiced in the art. For example, the association can be accomplished manually or automatically. Automatic association techniques can include, for example, database searches, database merges, GREP, AGREP, SQL, etc. The association step can be accomplished by a database merge function, for example, using key fields in each manufacturer and retailer data table. Key fields partition the database according to high-level object classes defined by the key fields. For example, a class can be designated as a key field in a first table and a second table, and then the two tables can be merged based on the class data in the key field. In these embodiments, the data corresponding to the key fields in each merged data table is preferably the same. However, data tables with similar but different data in the key fields can be merged by using, for example, AGREP.

Aspects of the invention provide a method that includes capturing a first visual image of a first block of material to generate a first image data packet associated with the first block of material; capturing a second visual image of a second block of material to generate a second image data packet associated with the second block of material, wherein the first block of material has a first chemical characteristic, and wherein the second block of material has a second chemical characteristic different from the first chemical characteristic; processing the first image data packet and the second image data packet with a machine learning system that has learned to visually distinguish between blocks of material having different chemical characteristics; and classifying the first block of material and the second block of material into two different categories based on the learned visual distinction between blocks of material having the different chemical characteristics with the machine learning system. The method further includes classifying the first material block and the second material block according to the classification. The material block may be a plastic block. The first chemical characteristic may be spectral data measured by a plurality of different sensor systems from at least one sample of the same type of plastic block as the first plastic block, and wherein the second chemical characteristic may be spectral data measured by the plurality of different sensor systems from at least one sample of the same type of plastic block as the second plastic block. The spectral data may be related to the invisible spectrum. The plurality of different sensor systems may be selected from a group consisting of near infrared ("NIR"), mid-wave infrared ("MWIR"), and X-ray fluorescence ("XRF") systems. The plurality of different sensor systems can be selected from infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray transmission ("XRT"), gamma ray, ultraviolet ("UV"), X-ray The group consisting of X-ray diffraction (XRF), laser induced breakdown spectroscopy (LIBS), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, differential scanning calorimetry (DSC), thermogravimetric analysis (TGA), capillary and rotational rheology, optical and scanning electron microscopy (SEM), and chromatography. The first chemical signature may include measurements of organic and inorganic elements or molecules from at least one sample of a plastic block of the same type as the first plastic block, and wherein the second chemical signature may include measurements of organic and inorganic elements or molecules from at least one sample of a plastic block of the same type as the second plastic block. The plastic block may be selected from the group consisting of Type #1 Polyethylene Terephthalate ("PET"), Type #2 High Density Polyethylene ("HDPE"), Type #3 Polyvinyl Chloride ("PVC"), Type #4 Low Density Polyethylene ("LDPE"), Type #5 Polypropylene ("PP"), Type #6 Polystyrene ("PS"), and Type #7 Other Polymers. The first material block may be polyvinyl chloride. The two different classifications may be different fractions.

Aspects of the present invention provide a system including a camera configured to capture a first visual image of a first block of material to generate a first image data packet associated with the first block of material, and to capture a second visual image of a second block of material to generate a second image data packet associated with the second block of material, wherein the first block of material has a first chemical characteristic, and wherein the second block of material has a second chemical characteristic different from the first chemical characteristic; and a data processing system, wherein the camera is configured to capture a first visual image of a first block of material to generate a first image data packet associated with the first block of material. The invention relates to a method for processing the first image data packet and the second image data packet using a machine learning system, wherein the machine learning system has learned to visually distinguish between blocks of material with different chemical characteristics, wherein the machine learning system classifies the first block of material and the second block of material into two different small parts based on the learned visual distinction between blocks of material with different chemical characteristics; and a classification device configured to classify the first block of material from the second block of material based on the small parts. The block of material may be a block of plastic. The first chemical characteristic may be spectral data related to the non-visible spectrum measured by a plurality of different sensor systems from at least one sample of the same type of plastic block as the first plastic block, and wherein the second chemical characteristic may be spectral data related to the non-visible spectrum measured by the plurality of different sensor systems from at least one sample of the same type of plastic block as the second plastic block. The plurality of different sensor systems may be a group of near infrared ("NIR"), mid-wave infrared ("MWIR"), and X-ray fluorescence ("XRF") systems. The plurality of different sensor systems may be infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray transmission ("XRT"), gamma ray, ultraviolet ("UV"), X-ray The group of methods include X-ray diffraction (XRF), laser induced breakdown spectroscopy (LIBS), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, differential scanning calorimetry (DSC), thermogravimetric analysis (TGA), capillary and rotational rheology, optical and scanning electron microscopy (SEM), and chromatography. The first chemical signature may include measurements of organic and inorganic elements or molecules from at least one sample of the same type of plastic block as the first plastic block, and wherein the second chemical signature may include measurements of organic and inorganic elements or molecules from at least one sample of the same type of plastic block as the second plastic block, wherein the plastic block is selected from the group consisting of Type #1 Polyethylene Terephthalate ("PET"), Type #2 High Density Polyethylene ("HDPE"), Type #3 Polyvinyl Chloride ("PVC"), Type #4 Low Density Polyethylene ("LDPE"), Type #5 Polypropylene ("PP"), Type #6 Polystyrene ("PS"), and Type #7 Other Polymers.

Aspects of the invention provide a method comprising determining a chemical signature of each of a mixture of different plastic chunks using a plurality of different sensor systems; capturing visual images of each of the plastic chunks; digitally associating the visual images with the chemical signature of each plastic chunk; determining a specific sub-portion for classifying the plastic chunks; using the visual images to identify which of the plastic chunks in the mixture have chemical signatures that fall into the specific sub-portion; and training a machine learning system to visually identify the plastic chunks that fall into the specific sub-portion, wherein the training is performed using a control set generated from the identified plastic chunks. The control set may be composed of captured visual image data of each of the identified plastic chunks. wherein the small portion may be composed of a specific combination of organic and inorganic elements or molecules. The plurality of different sensor systems may be selected from the group consisting of near infrared ("NIR"), mid-wave infrared ("MWIR"), and X-ray fluorescence ("XRF") systems. The mixture of different plastic pieces is selected from the group consisting of Type #1 polyethylene terephthalate ("PET"), Type #2 high density polyethylene ("HDPE"), Type #3 polyvinyl chloride ("PVC"), Type #4 low density polyethylene ("LDPE"), Type #5 polypropylene ("PP"), Type #6 polystyrene ("PS"), and Type #7 other polymers. The plurality of different sensor systems can be selected from infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray transmission ("XRT"), gamma ray, ultraviolet ("UV"), X-ray The group of methods includes X-ray fluorescence ("XRF"), laser induced breakdown spectroscopy ("LIBS"), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, differential scanning calorimetry ("DSC"), thermogravimetric analysis ("TGA"), capillary and rotational rheology, optical and scanning electron microscopy ("SEM"), and chromatography.

Reference is made herein to a "configured" device or a device "configured to" perform certain functions. It should be understood that this may include selecting predefined logic blocks and logically associating them so that they provide specific logic functions, including monitoring or control functions. It may also include programming computer software-based logic to modify control devices, wiring discrete hardware components, or any or all of the above.

In the description of this article, many specific details are provided, such as programming entities, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, etc., to provide a thorough understanding of the embodiments of the present invention. However, a person with ordinary knowledge in the relevant technical field will understand that the present invention can be practiced without one or more specific details or with other methods, components, materials, etc. In other cases, known structures, materials, or operations may not be shown or described in detail to avoid obscuring the state of the present invention.

It will be understood by those skilled in the art that the various settings and parameters of the components of the classification system 100 (including the neural network parameters) can be customized, optimized, and reconfigured over time based on the type of material being classified, the desired classification results, the type of equipment used, the empirical results of previous classifications, available data, and other factors.

References to "one embodiment", "embodiment" or similar terms throughout this specification mean that the particular features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the terms "in one embodiment", "in an embodiment", "embodiment", "certain embodiments", "various embodiments" and similar language appearing throughout the specification may, but do not necessarily, refer to the same embodiment. In addition, the features, structures, aspects and/or characteristics described in the present invention may be combined in one or more embodiments in any suitable manner. Accordingly, even if the features originally claimed work in certain combinations, in some cases, one or more features in the claimed combination may be deleted from the combination, and the claimed combination may be directed to subcombinations or variations of subcombinations.

Benefits, advantages, and solutions to problems have been described herein with respect to specific embodiments. However, benefits, advantages, solutions to problems, and any elements that may cause any benefit, advantage, or solution to occur or become more apparent may not be construed as key, necessary, or essential features and elements of any or all of the claims. Furthermore, unless expressly described as necessary or critical, no component described herein is necessary to practice the invention.

Although this specification contains many details, these should not be construed as limitations on the scope of the invention or what may be protected, but rather as descriptions of features specific to particular embodiments of the invention. The headings herein may not be intended to limit the invention, embodiments of the invention, or other matters disclosed under the headings.

As used herein, the term "or" may be intended to be inclusive, where "A or B" includes A or B and also includes both A and B. As used herein, the term "and/or" when used in the context of a list of entities refers to the entities present individually or in combination. Thus, for example, the term "A, B, C, and/or D" includes A, B, C, and D, respectively, but also includes any and all combinations and subcombinations of A, B, C, and D.

The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

All components or steps in the scope of the patent application below plus the corresponding structures, materials, actions and equivalents of the functional elements may be intended to include any structure, material or action used to perform a function in combination with other claimed elements as specifically claimed.

As used herein, terms such as "controller", "processor", "memory", "neural network", "interface", "classifier", "device", "push mechanism", "push device", "imaging sensor", "box", "container", "system", and "circuit system" refer to non-common device components that can be recognized and understood by a person of ordinary skill in the art, and are not used herein to refer to 35 U.S.C. 112(f).

As used herein with respect to an identified characteristic or condition, "substantially" means a degree of deviation that is sufficiently small that the identified characteristic or condition is not significantly impaired. In some cases, the exact degree of deviation permitted may depend on the specific circumstances.

As used herein, for convenience, multiple objects, structural elements, compositional elements, exemplary moieties, and/or materials may be presented in a common list. However, such lists should be interpreted as though each member of such list is independently identified as a separate and unique member. Therefore, no single member of such list should be interpreted as a de facto equivalent of any other member of the same list solely based on its presentation in a common group, without indication to the contrary.

Unless otherwise defined, all technical and scientific terms used herein (such as acronyms of polymers or chemical elements in the periodic table) have the same meaning as commonly understood by those of ordinary skill in the art to which the subject matter of the invention belongs. All publications, patent applications, patents, and other references mentioned herein are incorporated herein by reference unless a specific passage is cited. In the event of a conflict, the present specification (including definitions) shall prevail. In addition, materials, methods, and examples (e.g., listed small parts, plastics) are illustrative only and not limiting.

To the extent not described herein, many details regarding specific materials, processing actions, and circuits are known and can be found in textbooks and other sources in the fields of computing, electronics, and software technology.

Unless otherwise indicated, all numbers used in the specification and claims indicating the amounts of ingredients, reaction conditions, etc. should be understood to be modified in all cases by the term "about". Therefore, unless otherwise indicated, the numerical parameters set forth in this specification and the attached claims are approximate values, which may vary depending on the desired properties sought to be obtained by the subject matter of the invention.

401: Material block

403:Transmission system

410: Camera

411:XRF system

412:NIR system

413:MWIR system

Claims (17)

A method for plastic classification, comprising: capturing a first visual image of a first material block, thereby generating a first image data packet related to the first material block, wherein the first material block is a first plastic block; capturing a second visual image of a second material block, thereby generating a second image data packet related to the second material block, wherein the second material block is a second plastic block, wherein the first material block has a first chemical characteristic, and wherein the second material block has a second chemical characteristic different from the first chemical characteristic, wherein the first chemical characteristic comprises a plurality of different sensor systems from at least one sample of plastic blocks of the same type as the first plastic block. The method comprises: processing the first image data packet and the second image data packet using a machine learning system, the machine learning system having learned to visually distinguish between blocks of material having different chemical characteristics; and classifying the first block of material and the second block of material into two different categories based on the learned visual distinction between blocks of material having the different chemical characteristics. The method of claim 1 further includes classifying the first material block and the second material block according to the classification. The method of claim 1, wherein the spectral data belongs to the invisible spectrum. The method of claim 1, wherein the plurality of different sensor systems are selected from infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray transmission ("XRT"), gamma ray, ultraviolet ("UV") ), X-ray fluorescence ("XRF"), laser induced breakdown spectroscopy ("LIBS"), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, differential scanning calorimetry ("DSC"), thermogravimetric analysis ("TGA"), capillary and rotational rheology, optical and scanning electron microscopy ("SEM"), and chromatography. The method of claim 1, wherein the first chemical characteristic comprises measurements of organic and inorganic elements or molecules from at least one sample of a plastic block of the same type as the first plastic block, and wherein the second chemical characteristic comprises measurements of organic and inorganic elements or molecules from at least one sample of a plastic block of the same type as the second plastic block. The method of claim 1, wherein the plastic block is selected from the group consisting of type #1 polyethylene terephthalate ("PET"), type #2 high-density polyethylene ("HDPE"), type #3 polyvinyl chloride ("PVC"), type #4 low-density polyethylene ("LDPE"), type #5 polypropylene ("PP"), type #6 polystyrene ("PS"), and type #7 other polymers. A method as claimed in claim 1, wherein the first material block comprises polyvinyl chloride. The method of claim 1, wherein the two different categories are different sub-sections. A system for plastic sorting, comprising: a camera configured to capture a first visual image of a first block of material, thereby generating a first image data packet associated with the first block of material, and to capture a second visual image of a second block of material, thereby generating a second image data packet associated with the second block of material, wherein the first block of material has a first chemical characteristic, and wherein the second block of material has a second chemical characteristic different from the first chemical characteristic, wherein the first block of material is a first block of plastic, wherein the second block of material is a second block of plastic, wherein the first chemical characteristic comprises spectral data associated with an invisible spectrum measured by a plurality of different sensor systems from at least one sample of plastic blocks of the same type as the first block of plastic, and wherein the second chemical characteristic comprises spectral data associated with an invisible spectrum measured by a plurality of different sensor systems from at least one sample of plastic blocks of the same type as the first block of plastic, and wherein the second chemical characteristic comprises spectral data associated with an invisible spectrum measured by a plurality of different sensor systems from at least one sample of plastic blocks of the same type as the first block of plastic. The image data packet includes spectral data related to the invisible spectrum measured by the plurality of different sensor systems from at least one sample of the same type of plastic block as the second plastic block, wherein the plurality of different sensors include infrared and X-ray fluorescence systems; a data processing system configured to process the first image data packet and the second image data packet with a machine learning system, the machine learning system having learned to visually distinguish between blocks of material with different chemical characteristics, wherein the machine learning system classifies the first block of material and the second block of material into two different sub-portions based on the learned visual distinction between blocks of material with the different chemical characteristics; and a classification device configured to classify the first block of material and the second block of material based on the sub-portions. The system of claim 9, wherein the plurality of different sensor systems are selected from infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray transmission ("XRT"), gamma ray, ultraviolet ("UV") ), X-ray fluorescence ("XRF"), laser induced breakdown spectroscopy ("LIBS"), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, differential scanning calorimetry ("DSC"), thermogravimetric analysis ("TGA"), capillary and rotational rheology, optical and scanning electron microscopy ("SEM"), and chromatography. The system of claim 9, wherein the first chemical characteristic comprises measurements of organic and inorganic elements or molecules from at least one sample of a plastic block of the same type as the first plastic block, and wherein the second chemical characteristic comprises measurements of organic and inorganic elements or molecules from at least one sample of a plastic block of the same type as the second plastic block, wherein the plastic block is selected from the group consisting of Type #1 polyethylene terephthalate ("PET"), Type #2 high density polyethylene ("HDPE"), Type #3 polyvinyl chloride ("PVC"), Type #4 low density polyethylene ("LDPE"), Type #5 polypropylene ("PP"), Type #6 polystyrene ("PS"), and Type #7 other polymers. A method for plastic classification comprises: determining the chemical signature of each of a mixture of different plastic chunks using a plurality of different sensor systems; capturing visual images of each of the plastic chunks; digitally associating the visual images with the chemical signature of each plastic chunk; determining a specific fraction for classification of the plastic chunks, wherein the fraction is composed of a specific combination of organic and inorganic elements or molecules; using the visual images to identify which of the plastic chunks in the mixture have chemical signatures that fall into the specific fraction; and training a machine learning system to visually identify the plastic chunks that fall into the specific fraction, wherein the training is performed using a control set generated from the identified plastic chunks. The method of claim 12, wherein the control set is composed of captured visual image data of each of the identified plastic blocks. The method of claim 12, wherein the plurality of different sensor systems are selected from the group consisting of near infrared ("NIR"), mid-wave infrared ("MWIR"), and X-ray fluorescence ("XRF") systems. The method of claim 14, wherein the mixture of different plastic pieces is selected from the group consisting of type #1 polyethylene terephthalate ("PET"), type #2 high density polyethylene ("HDPE"), type #3 polyvinyl chloride ("PVC"), type #4 low density polyethylene ("LDPE"), type #5 polypropylene ("PP"), type #6 polystyrene ("PS"), and type #7 other polymers. The method of claim 12, wherein the plurality of different sensor systems are selected from infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray penetrating ("XRT"), gamma ray, ultraviolet ("UV ”), X-ray fluorescence (“XRF”), laser induced breakdown spectroscopy (“LIBS”), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, megahertz spectroscopy, differential scanning calorimetry (“DSC”), thermogravimetric analysis (“TGA”), capillary and rotational rheology, optical and scanning electron microscopy (“SEM”), and chromatography. The method of claim 12, wherein the plurality of different sensors include infrared and X-ray fluorescence systems.

Images