FR3148538A1 - Tire chaining by robotic means using three-dimensional vision - Google Patents

Abstract

The invention relates to a method for automatically chaining tires from a group of tires in an unknown arrangement and for which a predetermined target location is to be achieved, the chaining method implemented by at least one processor comprising an image processing module that uses a positioning algorithm to select the optimal location of a target tire to be chained. The invention also relates to an automatic tire chaining system comprising at least one gripping device (100), a detection system for collecting information on the physical environment around each gripping device (100), and a communication network comprising at least one communication server for executing programmed instructions stored in a memory of one or more processors of the chaining system that uses a positioning algorithm to implement the disclosed chaining method. Figure for abstract: Fig. 6

Description

Tire chaining by robotic means using three-dimensional vision

The invention relates to a system which performs a method of chaining tires from a group of tires in an unknown arrangement. More particularly, the invention relates to an automatic system capable of chaining tires in any target location, regardless of the parameters of the target location and the varying parameters of different tires being arranged.

Context

In the field of tire chaining, there are tire arrangements that facilitate their handling and ensure their optimal storage in the available storage space. Referring to the , an embodiment of tire storage is shown in which several layers of tires 10 partially overlap each other. In this type of tire storage (known in the art by the designation "rick-rack"), the tires are stacked (or "chained") in a container 12, the overlapping direction being reversed from one layer to the next according to a chaining process. In this configuration, the space between side portions 12a of the container 12 is optimally utilized. The container 12 can be selected from known containers for carrying tires, including, without limitation, pallets, truck beds, chained trucks, box bodies and the like. The structure of this stacking pattern is described in detail in patent DE2426471A1.

Other types of tire storage are also known to effect the transportation of such tires in containers. In one embodiment of tire storage called “roll storage,” the tires are stored side by side on their tread along a common horizontal axis. In one embodiment of tire storage called “stack storage,” the tires are stacked side by side on their sidewalls along a common vertical axis.

There are automated solutions for chaining tires in containers depending on the type of storage chosen. These solutions incorporate the piloting of a robot by vision in the context of gripping tires. Examples are provided by patent US8,244,400 (which discloses an automated tire chaining device on a support which comprises a handling device with one or more gripping tools coupled in order to receive and deposit the tires), patent US8,538,579 (which discloses a depalletizing system for implementing a method of depalletizing tires deposited on a support, the system being guided by a robot with a gripping tool), and patent US9,440,349 (which discloses an automatic tire loader/unloader for stacking/unstacking them in a trailer, comprising an industrial robot capable of selective articulated movement).

Chaining technologies often use a combination of laser scanning of the surface supposed to contain the objects to be picked up and knowledge of the object being sought (CAD). The system seeks to superimpose the elements measured in real space with the known elements of the CAD to precisely find the object and its spatial configuration and then be able to grasp it in a way that matches the design of the gripper. Thus, the majority of methods widespread in the industry work by trying to control the environment. This can be done from a hardware point of view by requiring very specific installations for the task, or by learning references in the fixed work environment, or even by trying to recalibrate a CAD model in a point cloud type scene to detect an object.

A method requiring a specific hardware installation, however sophisticated, will no longer work in the event of significant variations in the installation. A model recalibration requires that all objects in the container are identical (within a scale factor) and largely visible in order to have a suitable match. For example, US patent 8,538,579 proposes using CAD data of tires to carry out the work of “storage densification”. This requires either a completely homogeneous pallet of identical tires whose dimensions are entered once and then the system processes them automatically, or a case-by-case reading of the tire reference, the call for its dimensions in a CAD database, the calculation of the optimal storage position and then its manipulation.

In addition, there are also tire-specific storages outside of product recognition. For example, patent EP2062824B1 discloses unit boxes that facilitate the transportation of cylindrical items such as tires. Particularly, a folded box is disclosed that allows tires to be handled and stored stably. As a result, the storage of the boxes themselves introduces a significant loss of volume and a high cost.

The filling and emptying of containers known to carry out the transport of tires (and particularly, trucks) holding are tasks that are by definition random: the order of the chained tires is not known in advance, nor their size; accessibility is reduced, both for gripping; and the tires can only be seen partially. Controlling the environment is therefore not a viable situation.

Thus, the disclosed invention makes it possible to establish a link between chaining processes and the management of the chaining of tires arranged in a group of arranged tires without prior knowledge of their precise configuration.

Summary of the invention
The invention relates to a method for automatically chaining tires from a group of tires in an unknown arrangement and intended for chaining in a predetermined target location, the chaining method implemented by at least one processor comprising an image processing module which employs a positioning algorithm to choose the optimal location of a target tire to be chained, characterized in that the chaining method comprises the following steps:
- a step of providing a chaining system of which the processor is a part, in which the chaining system comprises at least one gripping device which performs the gripping of a target tire of the tire group;
- a step of estimating the dimensions of a target location for which the tires of the tire group are intended for chaining, this step comprising a step of scanning the target location to determine its parameters;
- a step of creating the origin of a location-device reference mark during which the chaining system creates a reference mark to know the positioning coordinates of a first tire in the target location;
- a step of estimating the dimensions of a target tire to be chained in the target location, during which the device identifies the first tire to be taken from the group of tires;
- a positioning step for each tire to be chained, this step including the following steps:
- a step of carrying out a tire positioning process which is carried out for each tire to be chained;
- a step of approaching the device towards the identified target tire; and
- a step of taking the identified target tire;
- a tire scanning step after its positioning carried out during the positioning step; and
- a final tire position update step, during which the actual position of the last positioned tire, estimated during the positioned tire scan step, includes the 2D center of the positioned tire and its orientation, which allows the positioning algorithm to update the actual position of the last positioned tire,
so that the update allows to check that the estimated actual position is consistent with the physical limits of the target location; and
so that the tire chaining is achieved when the gripping device releases the tire.

In certain embodiments of the chaining method of the invention, during the step of carrying out a positioning process for each tire to be chained:
- the tire group is modeled in two-dimensional (2D) space as a rectangle, where the width and height of the rectangle are equal to the width and height of the interior of the target location; and
- each tire in the tire group is modeled using a rectangle whose width and height are equal to the width and diameter of the tire represented.

In some embodiments of the chaining method of the invention, the step of estimating the dimensions of a target tire to be chained comprises a few-shot learning process comprising the following steps:
- a step of acquiring data corresponding to the arranged tires, during which a detection system of the gripping device captures an initial image of the tires arranged in random manners;
- a step of feeding an extraction neural network and an attention neural network, during which the two neural networks are trained by taking a plurality of sample images obtained during the data acquisition step as training data and a plurality of image object classifications as data labels;
- a step of carrying out a three-dimensional (3D) reconstruction process carried out entirely on the basis of the data from the extraction neural network and the attention neural network, during which coordinates corresponding to the location of an identified target tire and its orientation are reconstructed from this data, such that this reconstruction serves to provide the geometric information necessary to generate an ideal grip point for the identified target tire;
- a step of approaching the gripping device towards the identified target tire, during which the attention neural network sends to the gripping device the coordinates corresponding to the location of the identified target tire and its orientation; and
- a step of removing the identified target tire from the tire arrangement to place it in the target location.

In some embodiments of the chaining method of the invention, the step of feeding the extraction neural network and the attention neural network comprises the following steps:
- a step of training the extraction neural network to segment a scene seen by the detection system of the management system; and
- an attention mechanism construction step, during which the attention neural network extracts differentiated features among different categories in a target tire detection model, so that the model is guided to locate key areas in a segmented image;
wherein the step of training the extraction neural network comprises a step of segmenting the data based on a plurality of cycles of a repetitive movement of the gripping device during one or more chaining cycles.

In some embodiments of the chaining method of the invention, during the step of performing the 3D reconstruction process, the orientation, dimensions and location of the identified target tire are reconstructed from the data of the extraction neural network and the attention neural network.

In certain embodiments of the chaining method of the invention:
- the data acquisition step of the few-shot learning process includes a step of constructing a point cloud from the RGB-D images; and
- the step of approaching the gripping device includes a step of gripping the target tire identified at the ideal gripping point calculated during the step of carrying out the 3D reconstruction process.

In some embodiments of the chaining method of the invention, the tire positioning process comprises the following steps:
- a step of positioning the first tire which can be taken from the group of tires of which this first tire is positioned flat at a place corresponding to the choice of the location-device reference mark created during the step of creating the origin of the reference mark; and
- a step of arranging the tires starting from the second tire, during which the positioning algorithm uses a notion of row which is a sequence of tires positioned in the same direction, and in which each type of row is associated with a set of orientations to be tested for the tire to be positioned.
In some embodiments of the chaining method of the invention, during the tire storage step, the positioning algorithm applies the following steps to select the candidate tire positions:
- a step of first calculating all the positions which bring the tire concerned into contact with the limits of the target location;
- a step of then calculating all the positions which put the tire in contact with the other tires already chained;
- a step of conservation, among all the positions calculated during the previous step, of the positions allowing the tire to have at least two points of contact with the limits of the tire group or with the other tires; and
- a final step of preserving the positions allowing the current row to be completed which represents the only position allowing the current row to be completed.
In certain embodiments of the chaining method of the invention, the step of storing the tires further comprises:
- a step of calculating a stability and chaining criterion making it possible to know whether the position of the tire is stable in each position retained at the end of the selection of the candidate positions of the tire; and
- a conversion step during which the 2D center of the tire from the positioning algorithm is converted to the 3D center of the tire in the target location;
so that the positioning algorithm indicates the center of the tire in the rear plane of the tire group as well as its orientation to position the tire in the target location so as to obtain the 3D center of the tire and couple it with the orientation.

In certain embodiments of the chaining method of the invention, during the step of calculating a stability and chaining criterion:
- the stability of the position of the new tire depends on the vertical starting from its center of gravity and the support zone of the new tire; and
- the chaining criterion is based on a signed horizontal distance between the center of the last tire installed and the end of the new tire.

In certain embodiments of the chaining method of the invention, during the tire scanning step carried out during the positioning step:
- the chaining system scans an area in which the last tire was positioned; and
- an attention mechanism is then used to detect the last positioned tire.

The invention also relates to a system for automatic chaining of tires from a group of tires in an unknown arrangement and for which a predetermined target location is to be achieved, characterized in that the chaining system comprises:
- at least one gripping device comprising a robot having a gripping device supported by a pivotable elongated arm and extending from the elongated arm to a free end where a gripper is arranged along a common longitudinal axis;
a detection system for collecting information about the physical environment around each gripping device, the detection system comprising one or more sensors configured to perform two-dimensional (2D) and/or three-dimensional (3D) image detection; and
- a communication network that manages incoming data to the chaining system from at least one socket device, the communication network comprising at least one communication server for executing programmed instructions stored in a memory of one or more processors of the chaining system that employs a positioning algorithm to implement the disclosed chaining method;
such that each cleaning device is configured on one or more parameters of at least one identified tire calculated by an image processing module incorporated in the memory of the processor to set the cleaning device in motion so that the gripping device can carry out the gripping of a target tire during the chaining process carried out by the chaining system.

In some embodiments of the chaining system of the invention, the detection system comprises at least one RGB-D type camera attached to at least one of the gripping device, the elongated arm and the gripper of the cleaning device.

In some embodiments of the chaining method of the invention, the predetermined target location comprises at least one location selected from one or more containers, one or more trucks, one or more trailers, one or more warehouses, one or more distribution centers, one or more conveyors, one or more pallets and one or more storage means.

Other aspects of the invention will become apparent from the following detailed description.

The nature and various advantages of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals designate like parts throughout, and in which:
There represents a perspective view of an embodiment of tire storage.
Figures 2 and 3 represent components of a known tire in a meridian plane.
There represents a partial perspective view of an embodiment of a gripping device employed in a chaining system of the invention.
There represents an example of a group of arranged tires being processed by a gripping device of the type shown in the .
There represents a flow diagram of an embodiment of a chaining method of the invention performed by the chaining system.
There depicts an example of a trailer being used at a target location having tire chaining performed during the chaining method of the invention.
There depicts an example of a first pickable tire that is identified for positioning in a target location during the chaining method of the invention.
There represents a tire positioning process carried out during a positioning step of the chaining method of the invention.
There represents a schematic view of the types of tire positioning directions achieved during the positioning process.
There represents a schematic view of an orientation assembly to be tested for the tire to be positioned during a tire storage step of the positioning process.
There represents an example of an implementation of a tire storage in a target location.
Figures 13 and 14 show, respectively, a stable position and an unstable position representing the stability of the position of a positioned tire during the positioning process.
[ ] Figures 15 and 16 show an illustration of a chaining criterion used during the positioning process.
There represents a schematic view of a conversion step performed during the positioning process.
Figures 18 and 19 show schematic views, respectively, of an actual position of the last positioned tire and the updating of this actual position during the positioning process.

Detailed description

In considering the type of tire storage that best utilizes the available storage space, the geometry of the tires being transported must be considered. Figures 2 and 3 show schematics of a tire P that typically includes two circumferential beads intended to allow the tire to be attached to a rim. Each bead includes an annular reinforcing bead. The constitution of a tire is typically described by a representation of its constituents in a meridian plane, i.e. a plane containing the axis of rotation of the tire. The radial, axial and circumferential directions respectively designate the directions perpendicular to the axis of rotation of the tire, parallel to the axis of rotation of the tire, and perpendicular to any meridian plane. The expressions "radially", "axially" and "circumferentially" respectively mean "in a radial direction", "in the axial direction" and "in a circumferential direction" of the tire. The expressions "radially inner" and "radially outer" respectively mean "closer, respectively further, from the axis of rotation of the tire, in a radial direction.

Referring to the , the tire P includes an inner boundary F _I and an outer boundary F _E which together define the boundaries of a sidewall F of the tire P. The inner boundary F _I separates the sidewall F of the tire and a rim (not shown) to which the tire is intended for mounting. The tire P also includes a rim radius R _J defined as the distance between a center point C of the tire and the inner boundary F _I which separates the rim and the sidewall F of the tire. The tire P also includes an inner sidewall diameter defined as twice the rim radius R _J . The tire P also includes a tire radius R _P defined as the distance between the center point C and an outer boundary F _E of the sidewall F which represents the tread surface of the tire. The tire P also includes a tire diameter defined as twice the tire radius R _P .

Referring to the , the inflated and unloaded tire P comprises several parameters of its geometry, including a nominal section width L _P and a height H _P (the height H _P often being expressed as a percentage of the width L _P ). The tire P also comprises a measurement D _J which represents the diameter of a rim to which the tire is intended for mounting (this measurement being substantially equal to the internal sidewall diameter F _I ). It is understood that each of these parameters can be expressed in equivalent known length measurements (for example, in millimeters (mm) or in inches (in)).

Referring now to Figures 4-5, in which like numerals identify like elements, Figures 4 and 5 show one embodiment of a gripping device (or “device”) 100 that performs gripping of a target tire of an unknown arrangement of tires and for which a target location is to be achieved during a chaining cycle of the invention. The device 100 is part of a tire chaining system (or “chaining system” or “system”) of the invention.

It is understood that the term “chaining” includes the functions of storing and retrieving arranged (or “chained”) tires as well as the target arrangement (including loading and unloading) of the tires. It is understood that the term “target tire” (in the singular or plural) is used herein to refer to a tire that is present in the physical environment of the device 100 and that is identified for positioning during a chaining cycle. It is understood that the term “target location” (in the singular or plural) includes a dedicated space where the tires will be arranged (e.g., a trailer, a belt, a conveyor, a crate, a rack, etc.). The term “target arrangement” (in the singular or plural) includes a desired arrangement for the tires arranged in a target location (e.g., in a “rick-rack” or “chained” manner, “roll storage,” or “stack storage”).

The system of the invention performs a motion chaining method of the device 100 (or “chaining method”) which incorporates a combination of vision techniques to correctly and quickly reconstruct the observed scene from scattered three-dimensional (or “3D”) point clouds, resulting from a fragmented view of the target tires.

The device 100 is usable in spaces where tires are arranged in an unknown manner and in spaces where their target arrangement is to be achieved. As an example, the device 100 can be used with respect to a box 200 having tires P ₂₀₀ arranged therein. The device 100 can take the tires P ₂₀₀ arranged in the box 200 to chain them in one or more target locations (e.g., in a trailer 300 as shown in FIG. ). It is understood that any suitable type of container could be used instead of the 200 box.

The device 100 thus performs a target arrangement of tires in a predetermined target location. It is understood that the device 100 can operate in several physical environments without knowledge of their parameters in advance (for example, an initial or targeted arrangement of tires in a truck, in a warehouse, in a distribution center, on a conveyor, on a pallet or in relation to other known storage and/or transport means).

Referring further to Figures 4 and 5, in one embodiment of the device 100, the gripping device comprises a robot of the type disclosed by the Applicant's publication WO2022/135968. The robot has a gripping device 104 supported by a pivotable elongated arm 106. The gripping device 104 extends from the elongated arm 106 to a free end 104a where a gripper 108 is disposed along a common longitudinal axis. The robot is set in motion so that the gripper 108 can perform the gripping of a target tire by the device 100 during a chaining process performed by the system of the invention (as described below).

It is understood that the precise configuration of the device 100 shown in Figures 4 and 5 is given by way of example. For example, the device 100 could be provided with several types of grippers depending on the characteristics of a group of tires intended to be chained. In another example, the device 100 may comprise a fixed robot installed at a chaining installation, fixed, for example, to a support relative to which the robot extends. In this case, it is understood that the robot can be fixed to a ceiling, a wall, a floor or any support which allows the chaining method of the invention to be carried out. It is understood that the device 100 may comprise at least one roving robot. By “roving”, it is understood that the gripping device can be set in motion either by integrated movement means (for example, one or more integrated motors) or by non-integrated movement means (for example, one or more mobile means including autonomous mobile means). It is understood that the gripping device can be a classic industrial robot or a collaborative robot or even a delta or cable robot.

The device 100 also includes a sensing system (not shown) for collecting information about the physical environment around the device. The sensing system includes one or more sensors (including one or more cameras) configured to perform two-dimensional (2D) and/or three-dimensional (3D) image sensing, 3D depth sensing, and/or other types of sensing of the physical environment around the gripping device (it is understood that the terms "sensor" and "camera" are used interchangeably). In embodiments of the device 100 of the type shown in Figures 4 and 5, the one or more sensors of the sensing system are attached to at least one of the gripping device 104, the elongated arm 106, and the gripper 108 of the robot. In embodiments of the system of the invention, these sensors could be part of an overall detection system that employs these sensors together with one or more sensors positioned in the physical environment in which the device 100 operates (e.g., one or more cameras 400) (see ).

In one embodiment of the device 100, the sensing system includes at least one camera that provides 3D images represented as a set of 3D points with X, Y, Z coordinates, and red, green, blue color values (the "RGB" or "RGB-D" format) (referred to as an "RGB-D camera"). In this embodiment, an RGB-D camera is attached to at least one of the gripping device 104, the elongated arm 106, and the gripper 108 of the device 100. Two or more RGB-D cameras may be oriented to provide a predetermined overlap between the cameras' fields of view. As used herein, the term "camera" includes one or more cameras.

RGB-D cameras typically provide depth information using depth maps, being images where each pixel contains the distance between the camera and the corresponding point in space. Compared with traditional measurement methods such as manual measurement and other measurements based on electronic devices, 3D point cloud data from RGB-D type cameras have a much higher measurement rate. By using a sparser structure, a point cloud can be constructed from the RGB-D images by calculating the real world (e.g., X, Y, Z coordinates) with the intrinsic data of a scanning camera. Thus, information about the physical environment around the device 100 is obtained from the 3D point cloud data obtained from sensing technologies that are capable of capturing the 3D surface geometries of the target tires accurately and efficiently.

The term "point cloud" (singular or plural) is used herein to refer to a collection or collections of data points in space. A camera or cameras (or equivalent device or devices) collect three-dimensional (3D) data and detect the surfaces of objects (e.g., arranged tires) using a series of coordinates. Storing the information as a collection of spatial coordinates can save space, since many objects do not fill a large portion of the environment. Even if the information is not visual, interpreting the data as a point cloud helps to understand the relationship between multiple variables through classification and segmentation.

It is understood that one or more cameras may include one or more programming modes, including by learning, to feed, modify, and train at least one neural network. Although the embodiments are described herein with respect to the use of one or more neural networks (e.g., convolutional neural networks or “CNNs”) as a machine learning model, other types of machine learning models may be used. These include, but are not limited to, models using linear regression, logistic regression, decision trees, support vector machines, naive Bayes, nearest neighbor (knn), K is clustering, random forest, dimensionality reduction algorithms, gradient-based algorithms, neural networks (e.g., autoencoders, CNNs, RNNs, perceptrons, logarithmic short-term memory (LSTM), Hopfield, Boltzmann, deep belief, deconvolution, generative adversarial (GAN), etc.) and their complements and equivalents. The CNN(s) may be trained with ground truth data that is generated using sensor data representative of the motion of the device 100, including the positioning of the gripper 108.

The detection system of the device 100 detects the presence of a group of tires in the field of view of the detection system (e.g., the field of view of a camera of the device 100), which triggers it to capture the image of a target tire (see, for example, the target tire P _200* shown in FIG. ). In all embodiments of the device 100, the system “searches”, in the image obtained by the detection system, for the presence of a tire in the environment around the device 100. If no tire is detected, the detection system continues to obtain the images until the search of the environment around the device 100 is exhausted.

The sensing system may determine information about the physical environment that can be used by a control system (which includes, for example, software for planning movements of the device 100). The control system could be located on the device 100 or it could be in remote communication with the device. In embodiments of the device, one or more 2D or 3D sensors mounted on the device 100 (including, without limitation, navigation sensors) may be integrated to provide a digital model of the physical environment (including, where applicable, the side(s), floor, and ceiling). Using the obtained data, the control system may cause the device 100 to move to navigate between the target tire engagement positions.

To properly manage the manipulation of the device 100 that provides for the secure gripping of the target tire (e.g., the manipulation of the robot and the positioning of the gripper 108 as shown in Figures 4 and 5), it is necessary to detect the arrangement of the tires arranged in the tire group and identify the ideal target tire for gripping. Thus, the detection data refers to a plurality of records representative of the locations of at least one tire or a portion of a tire tracked over time. For example, the detection data may include one or more of records of the positions of a reference point on a portion of the tire (e.g., the sidewall) over time or at defined time intervals; sensor data taken over time; a video stream that has been processed using a computer vision technique; and/or data indicative of the operating status of the device 100 over time. In some cases, the sensing data may include data representative of one or more continuous movements of the gripping device before it stops to take one or more images of the arranged tires. The sensing system of the device 100 is thus configured to generate the device motion data.

In embodiments of the invention, the detection system of the device 100 may also include a motion capture device selected from infrared sensors, ultrasonic sensors, accelerometers, gyroscopes, pressure sensors, and/or other equivalent devices. For example, a motion capture device of the system of the invention may include one or a pair of digital gloves for performing management movements of the device 100 remotely. In these embodiments, the system (and particularly the device 100) learns the movements that achieve the target arrangement of the tires without operator intervention during one or more subsequent positioning processes forming part of the chaining method.

To implement the method of the invention by computer means, the chaining system of the invention comprises a communication network (or "network") which manages the data incoming to the system from various sources (for example, from at least one device 100 and its associated detection system). The communication network incorporates one or more communication servers (or "servers") each comprising one or more processors operatively connected to a memory. The memory is configured to store an application for analyzing data representative of the imaged tires. The one or more processors comprise an analysis application execution module which performs the processing of the images (or "image processing module"), the one or more processors of which are capable of executing programmed instructions stored in the memory to perform the steps of the chaining method (as described below).

Considering the device 100, its initial positioning (and, in applicable cases, the initial orientation of the gripper 108) is determined from data obtained via the acquisition of images of the chaining system and the physical environment in which the chaining system operates. An analysis application execution module of the processor employs an automatic and adaptive repositioning algorithm to find an ideal starting position of the device 100, thus making it possible to execute programmed instructions stored in the memory to perform the gripping of a first identified target tire to be positioned in a predetermined target location (for example, a trailer 300). The repositioning algorithm allows for continuous improvement on all tire grips, ensuring that the chaining system (and particularly the device 100) improves from the experience it acquires, in particular on the choice of tires to be released from the first tire identified for chaining.

The term "processor" (or, alternatively, the term "programmable logic circuit") refers to one or more devices capable of processing and analyzing data and comprising one or more software programs for their processing (e.g., one or more integrated circuits known to those skilled in the art as being included in a computer, one or more controllers, one or more microcontrollers, one or more microcomputers, one or more programmable logic controllers (or "PLCs"), one or more application-specific integrated circuits, one or more neural networks, and/or one or more other known equivalent programmable circuits). The processor comprises one or more software programs for processing the data captured by the detection system of the device 100 (and the corresponding data obtained) as well as one or more software programs for identifying and locating variances and identifying their sources in order to correct them.

In the system of the invention, the memory may include both volatile and non-volatile memory devices. The non-volatile memory may include solid state memories, such as NAND flash memory, "keep-alive memory" (or "KAM") for saving various operating variables while the processor is powered off, magnetic and optical storage media, or any other suitable data storage device that retains data when the device 100 is disabled or loses power. The volatile memory may include static and dynamic RAM that stores program instructions and data, including a learning application.

In embodiments of the system of the invention, the processor can configure the device (and in particular the gripper 108) on one or more parameters of the identified tire calculated by an image processing module incorporated in the memory of the processor.

In embodiments of the chaining method of the invention, the processor can configure the chaining system (and in particular the device 100) on one or more parameters of a target tire calculated by an image processing module. In these embodiments, it is understood that one or more reinforcement learning means could be used. Those skilled in the art in this field will recognize that many image processing techniques can be used to choose and determine the parameters of the target tires. Several commercially available image processing systems can be used.

To implement the method of the invention by computer means, the system of the invention comprises a communication network (or "network") which manages the data incoming to the system from various sources (for example, from at least one device 100 and its associated detection system). The communication network incorporates one or more communication servers (or "servers") each comprising one or more processors operatively connected to a memory. The memory is configured to store an application for analyzing data representative of the positions of the imaged tires. The one or more processors comprise an analysis application execution module which performs the processing of the images, the one or more processors of which are capable of executing programmed instructions stored in the memory to perform the steps of the method (as described below).

The input data to the chaining system of the invention may include general information regarding the identified tire. The general information includes stored data regarding the identification of the identified tire (including, without limitation, its production origin, distribution and/or storage, production date, retreading history if applicable, and mounting position and history). The corresponding data could be collected by one or more known devices (e.g., an RFID-type device disposed in or on the identified tire).

The processor may also refer to a reference (e.g., a table of various tire sizes) to make a final determination of a parameter or parameters of the identified tire. The reference may include known tire parameters corresponding to a plurality of known commercially available tires. For example, after the image processing module calculates one or more identified tire parameters, the processor may compare the calculated parameters with the known parameters stored in the reference. The processor may retrieve the known tire parameters corresponding to the commercially available tires that most closely match the calculated parameters to configure the gripper 108 (and thus place it in a precise position to pick up and position the identified tire). The tire reference may include measurements corresponding to a plurality of commercially available tires. For example, for a tire size 225/50R17, the number "225" identifies the tire's cross-section in millimeters, the number "50" indicates the sidewall aspect ratio, and the measurement "R17" represents the rim diameter in inches (being approximately 43.18 centimeters).

Referring further to Figures 1-5, and further to Figures 6-19, a detailed description is provided by way of example of embodiments of a chaining method (or "method") of the invention for chaining tires in a target location (e.g., the interior of a trailer 300 of a truck) (see ). The chaining method is implemented by the system of the invention in any physical environment without prior knowledge of the dimensions of the tires in the group of tires intended for chaining and without prior knowledge of their arrangement. It is therefore understood that the location positions of the tires are generated on the fly to take into account the different dimensions of the successive tires according to the parameters of the target location.

As used herein, the term “method” or “process” may include one or more steps performed by at least one computer system having one or more processors to execute instructions that perform the steps (e.g., the steps of a positioning algorithm implemented by the system of the invention). Unless otherwise indicated, any sequence of steps is exemplary and does not limit the methods described to any particular sequence.

In performing the chaining method of the invention, the system of the invention incorporates a combination of vision and machine learning techniques to correctly and quickly reconstruct the observed scene from scattered three-dimensional (or "3D") point clouds, derived from a view of the tire identified for chaining. The system of the invention therefore achieves a continuous improvement in the recognition of different tires and their positioning relative to the target location.

There represents a flow diagram of an embodiment of operation of the system of the invention during a chaining process performed by the system. As shown in the , the “simulator” represents a positioning algorithm implemented by the system of the invention to choose the optimal location of a tire to be chained.

The positioning algorithm (or "simulator") implemented by the system of the invention can be seen as a physical model seeking to represent the physics associated with the chaining of tires in a predetermined target location. Like other models, it presents assumptions:
- The workspace and tires are represented in a two-dimensional ("2D") space. Each tire is represented by a rectangle (where the width of the rectangle represents the width of the tire envelope and the height of the rectangle represents its diameter) which will be positioned in a larger rectangle representing the group of tires being chained (see the ).
- The positioning algorithm does not chain the tires, but it looks for positions that favor their chaining. Interaction with a chaining system detection system allows the chaining of tires to be taken into account in the positioning algorithm.
- It is assumed that there is no slippage between different tires when constructing a stacked tire group. Thus, the phenomenon of tire settling is not taken into account.
- The positioning algorithm is based on rules put in place to position each tire (for example, the orientation of the tires, the notion of “row”, etc.).

It is obvious that this physical model could be refined to improve performance. For example:
- The positioning algorithm could be improved by working in a three-dimensional space, representing each tire by a three-dimensional (3D) shape (e.g., a shape from a CAD model).
- Tire chaining could be taken into account by the positioning algorithm by allowing the nesting of tires represented in 3D.
- Tire slippage and compaction could be integrated.
- The rules used to position each tire could be replaced by a reinforcement learning algorithm that would seek to learn the most relevant rules by itself. This would increase the chaining density obtained.

Referring again to Figures 6 and 7, in initiating the chaining method of the invention, the method comprises a step of estimating the dimensions of a target location whose tires are intended for chaining.

As an example, in the following description, the 300 trailer of the is referenced as the target location that is the subject of this estimate. It is understood that the target location could be chosen from one or more known target locations for carrying out the chaining and transport of tires (including, without limitation, containers, pallets, truck bins, chained trucks, box vans and their equivalents).

The estimation step of the chaining method includes a step of scanning the interior of the target location whose tires are intended for chaining. During this step, the chaining system uses its detection system to scan the interior of the target location and to estimate its parameters (i.e., its dimensions). Taking the example of the trailer 300 of the , during this step, the detection system scans the interior of the trailer to determine its length, width and depth.

The chaining method further comprises a step of creating the origin of a location-device reference mark. During this step, in order to start chaining tires, it is necessary to know where to start placing the first tire in the target location from the parameters obtained during the step of scanning the interior of the target location (for example, it is necessary to know where to start placing in the trailer 300 the tire P _200* of a group of tires P ₂₀₀ stored in a box 200) (see ). As an example, for the remainder of the description, a marker is positioned at the coordinates that correspond to the lower left corner of the trailer 300 (located at the bottom of the trailer 300). This marker will allow the device 100 to be controlled so that it positions the tires in the trailer 300. If the marker is placed in another corner, the complete system would operate with some adaptations.

The chaining method further comprises a step of estimating the dimensions (the diameter and the width) of a target tire to be chained in the predetermined target location (see tire P _200* of the ). During the estimation step, the detection system is used to identify the first tire to be taken from among several tires arranged in an unknown arrangement (for example, the group of tires P ₂₀₀ ). It is understood that a known system for supplying tires to be loaded could be used to bring the tires to the device 100 (for example, a system incorporating one or more belts for transporting the tires to the device 100 and its detection system). The diameter and width of the tire P _200* are represented in an initial position in the (the initial positioning being defined based on the location-device reference frame created during the reference origin creation step).

In one embodiment of a chaining method performed by the chaining system incorporating the device 100 (and/or an equivalent device), the system of the invention performs a learning process from a single or a small number of examples (called "few-shot learning" or "FSL"). Few-shot learning can reduce the data collection effort for data-intensive applications (including image classification and video event detection), helping to alleviate the burden of large-scale supervised data collection (see "One-Shot Learning of Object Categories", Fei-Fei, Li, Fergus, Rob and Perona Pietro, IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 28, Issue 4, pp. 594-611 (April 2006) (https://doi.org/10.1109/TPAMI.2006.79). This process is performed as disclosed by the Applicant's application FR2113035. In one embodiment of the chaining method of the device 100, the system employs few-shot learning to facilitate a storage optimization function for enhancing tire gripping. The system of the invention thus achieves continuous improvement in the choice of tires to take.

In this embodiment of the chaining method performed by the chaining system incorporating the device 100, the few-shot learning process uses an attention mechanism to perform the capture of target tires. The few-shot learning process benefits from a neural network structure allowing it to focus only on the tires likely to be captured by recognizing them immediately. This makes the device 100 (and therefore the system of the invention) faster and more robust because it can adapt to all the orientations of the tires that it sees. In addition, the adaptation to the tire arrangements is performed regardless of the configuration of the capture device of the management system.

In this embodiment of the chaining method, the few-shot learning process comprises a step of acquiring data corresponding to the arranged tires. In the group of tires intended for chaining in the target location, during this step, the detection system (for example, an RGB-D 400 camera of the chaining system of the invention and/or the device 100) (see ) captures an initial image of a group of tires P ₂₀₀ arranged randomly in an unknown location (e.g., a crate 200 as shown in Figures 5 and 8). In this example, several overlapping tires appear in the field of view of the detection system. During this step, a point cloud is constructed from the RGB-D images as described above.

The few-shot learning process also includes a step of feeding an extraction neural network (or “extraction network”) and an attention neural network (or “attention network”). This step includes a step of training the extraction network to segment the scene seen by the detection system. In one embodiment of the method, this step may include a step of segmenting the data based on a plurality of cycles of a repetitive movement of the gripping device during one or more chaining cycles. The segmentation performed during this step differentiates between an object in the image that includes a tire (either a whole tire or a partial tire) and an object in the image that does not include any tire.

During the feeding step, the extraction network and the attention network are trained by taking a plurality of sample images (obtained during the acquisition step) as training data and a plurality of image object rankings (or “heat maps”) as data labels. For example, based on the image object ranking, an image may be evaluated to determine whether the image is capable of attracting the attention of the detection system after the image including the tire targeted for capture is returned to the detection system. During this step, the RGB-D camera provides depth information of the arranged tires.

The feeding step includes a step of building an attention mechanism. During this step, the attention network extracts differentiated features among different categories in a target tire detection model (or “template”), so that the model is guided to locate key areas with important features in a segmented image (i.e., an image incorporating the most likely tire for gripping). The model gives better supervision to the key areas in order to learn differences among easy-to-confuse categories (e.g., the most likely tire for gripping among tires arranged from a labeled set of images). The accuracy of detecting the target tire in the image is therefore improved to arrive at the choice of the identified target tire for gripping.

In this embodiment of the chaining method, the process further comprises a step of performing a three-dimensional (3D) reconstruction process that serves to provide the geometric information necessary to generate the ideal gripping point by the gripping device (for example, the gripping by a gripper 108 of a device 100). The 3D reconstruction process comprises a reconstruction process entirely performed on the basis of the data of the extraction and attention networks. During this step, the orientation, dimensions and location of the identified target tire are reconstructed from this info data. In doing so, the management system (including the device 100) has obtained recognition of the arranged tires (including their orientations and positions) from examples of tires during the few-shot learning process.

During this step, the management system constructs a virtual tire in the shape of a cylinder on the visible surface of the cluster representing a target tire. During this step, the identification of the center of the identified target tire is done to estimate its inner and outer diameter (the inner diameter being represented by twice the rim radius R _J as discussed above with respect to the ). Once the attention network learns the location of the target tire and its orientation, it sends the corresponding coordinates (e.g., the X,Y,Z coordinates and the orientation of the tire axis) to the device 100. At this stage of the process, path plans and distance transformations are already made to extract the identified target tire. Thus, it is the management system that has dictated the information expected in return from the detection system (being the RGB-D camera). Consequently, the device 100 applies the movements necessary to perform the gripping and disengagement of the target tire at the ideal gripping point.

The chaining method of the invention further comprises a step of carrying out a positioning process for each tire to be chained. This step comprises a step of approaching the device 100 towards the target tire identified during the data acquisition step (for example, the target tire P _200* shown in the ). This step further comprises a step of gripping the target tire identified at the ideal gripping point during the data acquisition step (e.g., as calculated during the step of performing the 3D reconstruction process). For the configuration of a device 100 as shown in FIGS. 4 and 5, during this step, the gripper 108 is managed to engage a sidewall of the target tire (e.g., by extending one or more fingers of the gripper toward a gripping point of the inner boundary F _I of the sidewall F) (see FIG. ). It is understood that the implementation of the method of the invention is not limited by the configuration of the gripper device 108 of the device 100.

During the step of positioning the target tire identified for pickup, the positioning algorithm implemented by the processor of the chaining system considers only the group of tires being chained. This group is modeled in a two-dimensional (2D) space in the form of a rectangle. The width and height of the rectangle are equal to the width and height of the interior of the target location (for example, the width and height of the trailer 300). The goal of the positioning algorithm is then to position the successive tires in this rectangle. Each tire is also modeled using a rectangle whose width and height are equal to the width and diameter of the tire represented.

Referring now to the , there represents a tire positioning process performed during the positioning step. The tire positioning process includes a step of positioning the first tire that is pickable, this being the “target tire”. This first tire is positioned flat at a location corresponding to the choice of the location-device marker (created during the step of creating the origin of the marker). As an example, this first tire is positioned at the lower left corner of the trailer 300 (this behavior being linked to the choice of the location-device marker made previously) (see ).

The tire positioning process further includes a step of arranging the tires starting from the second tire. During this step, the positioning algorithm uses a notion of row which is a sequence of tires positioned in the same direction. Referring to the , we distinguish four (4) types of directions including:
- To the right (represented by the arrows A of the ) ;
- To the left (represented by the arrows B of the ) ;
- Upwards following a row to the right (represented by the arrows C of the ) ; And
- Upwards following a row to the left (represented by arrow D of the ).

During the tire stacking stage, the chaining system begins by filling a row based on the starting point (which could be “to the right” as represented by arrows A or “to the left” as represented by arrows B) (see Figure 1). ). Each type of row is associated with a set of orientations to be tested for the tire to be positioned (see the ). The chaining system therefore begins by orienting the tire according to the first orientation value to be tested (for example, the orientation represented by the angle Θ or the orientation represented by the angle - Θ in the ).

Referring to the , an example is provided of an embodiment of a storage of tires in a target location (here, the trailer 300 which is also shown in the ). During the tire storage step, the positioning algorithm will then apply the following steps to select candidate tire positions:
- A step of first calculating all the positions which bring the tire concerned into contact with the limits of the target location;
- A calculation step then of all the positions which put the tire in contact with the other tires already chained (represented in the by the P _C tires);
- A step of conservation, among all the positions calculated during the previous step, of the positions allowing the tire to have at least two points of contact with the limits of the target location or with the other tires (represented in the by the P _D tires); and
- A final step of preserving positions to complete the current row (shown as an example in the by the surrounded P _BON tire which represents the only position allowing the current row to be completed) (here, “to the right”).

It is understood that the steps of positioning the surrounded P _BON tire could be different depending on the parameters of the target location (e.g., the height, width and depth of the trailer 300). Furthermore, the relative dimensions of the tires in the tire group must be taken into account to select the candidate tire positions (these dimensions are not assumed to be equal for all tires being chained).

The tire storage step further includes a step of calculating a stability and chaining criterion. The stability criterion makes it possible to know whether the position of the tire is stable in each position retained at the end of the selection of the candidate positions of the tire. Referring to Figures 13 and 14, the tire P _200* represents a tire already positioned, and the tire P _BON represents a tire to be positioned (it is understood that the precise positions of the tires P _200* and P _BON are given as an example and that other positions could be taken). The stability of the position of the new tire depends on the vertical starting from its center of gravity (see arrow E in Figures 13 and 14) and on the support zone of the new tire (see zone F shown in Figures 13 and 14). If the vertical does not cross the support zone, the position is unstable (see arrow E in Figures 13 and 14). ). Otherwise, the position is stable and the stability of the position can be estimated by dividing the length of the support zone F by the diameter of the tire (this estimate being "the stability criterion") (see the ). In this case, the position is preserved if its stability is strictly greater than 0.5.

It is understood that one or more other embodiments could incorporate slip between tires into the stability criterion to improve the physics of the model.

The chaining criterion is based on the horizontal signed distance between the center of the last tire installed and the end of the new tire. Only positions that are sufficiently far from the last tire are retained. If several positions are retained, the final position of the new tire will be the one that is closest to the last tire positioned (to avoid forming gaps, or "holes", that are too large between two successive tires). Conversely, if no position is retained, this means that we have reached the end of the row and that we must move on to the next row. The system therefore moves on to the next row, and it applies the previous steps again to find the best position for the new tire.

Referring to Figures 15 and 16, an illustration of the chaining criterion is provided which is based on the horizontal signed distance between the center of the last installed tire (see tire 1) and the end of the new tire 2. In the , the new tire 2 covers the center of the last tire 1, which results in a negative distance and an unfavorable chaining situation (the hollow area of the last tire that could accommodate part of a future tire installed is covered). In the , the new tire 2 is sufficiently far from the last tire 1 to bring about a favorable chaining situation (which results in a positive distance). For example, another tire 3 installed later can fit into the hollow part of the tire 1, and a chaining of the tires is also obtained.

It is understood that one or more other embodiments could integrate the penetration between tires into the chaining criterion to improve the physics of the model.

The tire storage step further comprises a conversion step. During this step, the 2D center of the tire from the positioning algorithm is converted to the 3D center of the tire in the predetermined target location. Referring to the , the positioning algorithm indicates the center of the tire in the rear plane of the tire group (shown in the by the dotted segments) as well as its orientation (represented in the by the angle α). To position the tire in the target location (e.g., the trailer 300), it is necessary to know its position in the 3D frame of the target location (defined according to the location-device frame created during the step of creating the origin of the frame). For this, the 2D center is shifted forward by a value equal to the radius of the tire (e.g., the radius R _P ) (see ). The 3D center of the tire is therefore obtained and coupled with the orientation (angle α) preserved to position the tire in the target location.

The chaining method of the invention further comprises a step of scanning the positioned tire after its positioning (performed during the previous step). During this step, the positioning algorithm uses a chaining criterion to choose the position of the tire but it does not chain the tires (it does not manage the penetration of the tires into each other). The chaining of the tires is carried out naturally when the gripper of the device 100 releases the tire. It should therefore be understood that the positioning algorithm calculates a “position before chaining”. During this step, it is necessary to update this “position before chaining” so that it becomes a “position after chaining” which is the actual position of the tire in the predetermined target location (for example, the trailer 300). This is necessary to correctly position the next tires in the target location. Without this update, there would be a drift or a gap between the “positions before chaining” of the positioning algorithm and the actual positions of the tires in the target location.

To know this real position, the chaining system incorporates a combination of vision and machine learning techniques to correctly and quickly reconstruct the observed scene from 3D scattered point clouds, resulting from a view of a group of stacked tires. The system therefore achieves continuous improvement in the recognition of tires and their relative positioning along the internal surface of the target location (e.g., trailer 300).

During the tire scanning step, the chaining system uses its detection system to scan an area where the last tire was positioned. During this step, the attention mechanism (described above) is then used to detect the last tire positioned in the color image.

In one embodiment, a reference may be created that incorporates coordinates of the centers of the tires sought in images captured by the detection system of the device 100 (for example, an RGB-type camera). The coordinate reference that is created during this step includes expected images corresponding to the centers distributed in the target location. The images obtained during the scanning step revealing one or more positions of the centers of the tires intended for chaining, train at least one neural network to identify all the expected positions of the centers in the imaged target location. Thus, these image variations serve as input to the neural network whose outputs are the classification of the coordinates of the centers of the tires. In this embodiment of the method, at least a portion of the center reference may be created by one or more skilled persons.

The 3D points of this tire being recovered, a cylinder at these 3D points is then adjusted. The 3D points of the cylinder are projected into the rear plane of the tire group to obtain a set of 2D points. In embodiments of the method, during this step, a neural network can be trained to recognize the true coordinates of the centers and to create bounding boxes (or “boxed regions”) around the recognized centers. During this training, the coordinates of the bounding box of the recognized center are correlated with the coordinates of the sought centers to calculate displacements between them. The boxed regions and the displacement calculations are transmitted to a neural network (for example, one or more CNNs) to jointly learn the representation of a tire in different perspectives of the images taken by the detection system of the device 100. Thus, the bounding box of these 2D points provides the real center and the real orientation of the tire to be transmitted to the positioning algorithm to update it.

Referring now to Figures 18 and 19, the chaining method of the invention further comprises a final step of updating the position of the tire. Thanks to the scan carried out during the step of scanning the positioned tire, the actual position of the last positioned tire has been estimated (see whose tire 7 is not chained relative to tires 1 to 6). This position includes the 2D center of the tire as well as its orientation. By modifying the values of the 2D center and the orientation with those estimated during the scan, the positioning algorithm is updated with the actual position of the last tire positioned. By displaying the actual positions after this update, it is noted that the tires are indeed chained (see the whose tire 7 is properly chained after updating the positions of tires 1 to 6).

This update step also allows to check that the actual position estimated in the previous step is consistent with the positions of the other tires (ensuring that the overlap between the last tire (e.g., tire 7 in Figures 18 and 19) and the other tires (e.g., tires 1 to 6 in Figures 18 and 19) is not too high). In addition, this update step allows to check that the actual position estimated in the previous step is consistent with the physical limits of the target location (ensuring that the actual position of the last tire (e.g., tire 7 in Figures 18 and 19) is inside the target location (e.g., the inside of the trailer 300). This step therefore improves the robustness of the complete system by allowing it to detect certain inconsistencies.

The chaining system of the invention can easily repeat the steps of the chaining method in an order to properly chain the tires of a group in an arrangement that suits the parameters of the target location.

In all embodiments of a chaining method carried out by the chaining system incorporating at least one device 100, the steps can be carried out simultaneously on all of the tires arranged via parallelization carried out by the processor used.

In a management installation incorporating the chaining system, the data collected by the sensors can be used in the management of an apparatus which performs the taking of the tires arranged in the target location. In one embodiment of the management installation, this apparatus comprises at least one device 100 of the type described above and shown in Figures 4 and 5.

In one embodiment, the sensors may include one or more detection sensors (not shown) that detect the presence of one or more tires based on properties of the target location (e.g., the height and/or width and/or depth of the trailer 300). The detection sensors may be selected from commercially available sensors (e.g., reflector type sensors). Detection of a first tire at the defined marker in the cargo space may trigger the chaining process performed by the chaining system.

In one embodiment of the system, the sensors may include one or more detection sensors (not shown) that detect the presence of one or more tires based on properties of the target location (e.g., the height and/or width and/or depth of the trailer 300). The detection sensors may be selected from commercially available sensors (e.g., reflector type sensors). Detection of a first tire at the defined marker in the cargo space may trigger the chaining process performed by the chaining system.

In a tire management system incorporating the chaining system of the invention, a detection system may be used to detect the presence of an arrangement of tires in the field of view of a camera of the detection system (e.g., camera 400), thereby triggering the camera to capture an image of one or more tires. In cases where a portion of the tire is not visible in the image obtained by the camera, an arbitrary point may be placed at a known position relative to the sensor of the detection system (e.g., at a known horizontal distance and a known vertical distance from the sensor position).

The sensors of the target location, the sensors of the device 100 and the vision system of the management facility can thus provide information on the physical environment that can be used by a control system (which includes, for example, tire chaining planning software, and/or software for controlling the corresponding movements of the device 100). The control system could be in remote communication. In embodiments, one or more sensors mounted on the device 100 (including, without limitation, navigation sensors) can be integrated to form a digital model of the physical environment (including, where applicable, the side(s), floor and ceiling). Using the obtained data, the control system can cause the movement of the device 100 to navigate between the target tire chaining positions according to the parameters of the target location.

To avoid a complex and slow approach that is difficult to maintain and evolve, the disclosed invention combines a positioning algorithm with a 3D vision system of the detection system to take into account the complexities of interactions between different tires (including slippage and chaining). The great interest of this approach lies in the association of the positioning algorithm with steps of scanning the tire after positioning to take into account the complexities of interactions between different tires (including slippage and chaining). It is expected that the positioning algorithm used could operate without additional work on any type of tire of any size. This avoids having to use a positioning algorithm based on a very advanced physical model that would be more complex to use, maintain, evolve and that would have longer calculation times and therefore potentially incompatible with the requested cycle times. Consequently, the invention allows the chaining of tires within a constrained space, which presents a complex situation because it is necessary to manage collisions with the internal limits of the space provided for the chaining.

In advance of initiating the method of the invention, orders for particular tires could be received by known means (for example, by one or more communication networks integrated in a facility incorporating the chaining system of the invention). The facility allows receiving the orders up to a specific time to ensure the chaining of ordered tires in a “just in time” manner (where the chaining process starts).

A method of the invention may be performed by PLC control and may include preprogramming of management information. For example, a method setting may be associated with the parameters of the target location being anticipated and/or the properties of the tires intended for chaining. The system of the invention (and/or a facility incorporating this system) may easily repeat one or more steps of the method in a determined order to properly provide ordered tires to achieve a desired tire chaining.

The system of the invention (and/or an installation incorporating this system) may include preprogramming of management information. For example, a process setting may be associated with the parameters of the typical physical environments in which the system operates. In embodiments of the invention, the system (and/or an installation incorporating this system) may receive voice commands or other audio data representing, for example, a step or stop of the device 100 and/or a chaining of a tire or tires in an identified tire group. The request may include a request for the current state of a chaining cycle. A generated response may be represented audibly, visually, tactilely (e.g., using a haptic interface), and/or virtually and/or augmented. This response, associated with the corresponding data, may be recorded in a neural network.

For all embodiments of the system, a monitoring system could be implemented. At least part of the monitoring system may be provided in a portable device such as a mobile network device (e.g., a mobile phone, a laptop, one or more network-connected wearable devices (including "augmented reality" and/or "virtual reality" devices), network-connected wearables, and/or any combinations and/or equivalents). It is envisaged that detection and comparison steps may be performed iteratively.

The terms "at least one" and "one or more" are used interchangeably. Ranges that are presented as being "between a and b" encompass the values "a" and "b".

Although particular embodiments of the disclosed apparatus have been illustrated and described, it will be understood that various changes, additions and modifications may be practiced without departing from the spirit and scope of the present disclosure. Accordingly, no limitations should be imposed on the scope of the disclosed invention except as set forth in the appended claims.

Claims (14)

A method of automatically chaining tires from a group of tires in an unknown arrangement and intended for chaining in a predetermined target location, the chaining method implemented by at least one processor comprising an image processing module which employs a positioning algorithm to select the optimal location of a target tire to be chained, characterized in that the chaining method comprises the following steps:
- a step of providing a chaining system of which the processor is a part, in which the chaining system comprises at least one gripping device (100) which performs the gripping of a target tire of the tire group;
- a step of estimating the dimensions of a target location for which the tires of the tire group are intended for chaining, this step comprising a step of scanning the target location to determine its parameters;
- a step of creating the origin of a location-device reference mark during which the chaining system creates a reference mark to know the positioning coordinates of a first tire in the target location;
- a step of estimating the dimensions of a target tire to be chained in the target location, during which the device (100) identifies the first tire to be taken from the group of tires;
- a positioning step for each tire to be chained, this step including the following steps:
- a step of carrying out a tire positioning process which is carried out for each tire to be chained;
- a step of approaching the device (100) towards the identified target tire; and
- a step of taking the identified target tire;
- a tire scanning step after its positioning carried out during the positioning step; and
- a final tire position update step, during which the actual position of the last positioned tire, estimated during the positioned tire scan step, includes the 2D center of the positioned tire and its orientation, which allows the positioning algorithm to update the actual position of the last positioned tire,
so that the update allows to check that the estimated actual position is consistent with the physical limits of the target location; and
so that the chaining of the tires is carried out when the gripping device (100) releases the tire. The chaining method of claim 1, wherein, during the step of performing a positioning process for each tire to be chained:
- the tire group is modeled in two-dimensional (2D) space as a rectangle, where the width and height of the rectangle are equal to the width and height of the interior of the target location; and
- each tire in the tire group is modeled using a rectangle whose width and height are equal to the width and diameter of the tire represented. The chaining method of claim 1 or claim 2, wherein the step of estimating the dimensions of a target tire to be chained comprises a few-shot learning process comprising the following steps:
- a step of acquiring data corresponding to the arranged tires, during which a detection system of the gripping device (100) captures an initial image of the tires (P ₂₀₀ ) arranged randomly;
- a step of feeding an extraction neural network and an attention neural network, during which the two neural networks are trained by taking a plurality of sample images obtained during the data acquisition step as training data and a plurality of image object classifications as data labels;
- a step of carrying out a three-dimensional (3D) reconstruction process carried out entirely on the basis of the data from the extraction neural network and the attention neural network, during which coordinates corresponding to the location of an identified target tire and its orientation are reconstructed from this data, such that this reconstruction serves to provide the geometric information necessary to generate an ideal grip point for the identified target tire;
- a step of approaching the gripping device (100) towards the identified target tire, during which the attention neural network sends to the gripping device the coordinates corresponding to the location of the identified target tire and its orientation; and
- a step of removing the identified target tire from the tire arrangement to place it in the target location. The chaining method of claim 3, wherein the step of feeding the extraction neural network and the attention neural network comprises the following steps:
- a step of training the extraction neural network to segment a scene seen by the detection system of the management system (100); and
- an attention mechanism construction step, during which the attention neural network extracts differentiated features among different categories in a target tire detection model, so that the model is guided to locate key areas in a segmented image;
wherein the step of training the extraction neural network comprises a step of segmenting the data based on a plurality of cycles of a repetitive movement of the gripping device during one or more chaining cycles. The chaining method of claim 3 or claim 4, wherein, during the step of performing the 3D reconstruction process, the orientation, dimensions and location of the identified target tire are reconstructed from the data of the extraction neural network and the attention neural network. The chaining method of claim 4 or claim 5, wherein:
- the data acquisition step of the few-shot learning process includes a step of constructing a point cloud from the RGB-D images; and
- the step of approaching the gripping device (100) comprises a step of gripping the identified target tire at the ideal gripping point calculated during the step of carrying out the 3D reconstruction process. The chaining method of any preceding claim, wherein the tire positioning process comprises the following steps:
- a step of positioning the first tire which can be taken from the group of tires of which this first tire is positioned flat at a place corresponding to the choice of the location-device reference mark created during the step of creating the origin of the reference mark; and
- a step of arranging the tires starting from the second tire, during which the positioning algorithm uses a notion of row which is a sequence of tires positioned in the same direction, and in which each type of row is associated with a set of orientations to be tested for the tire to be positioned.
The chaining method of claim 7, wherein, during the tire arranging step, the positioning algorithm applies the following steps to select candidate tire positions:
- a step of first calculating all the positions which bring the tire concerned into contact with the limits of the target location;
- a step of then calculating all the positions which put the tire in contact with the other tires already chained (P _C );
- a step of conservation, among all the positions calculated during the previous step, of the positions allowing the tire to have at least two points of contact with the limits of the tire group or with the other tires (P _D ); and
- a final step of preserving the positions allowing the current row to be completed (P _BON ) which represents the only position allowing the current row to be completed.
The chaining method of claim 7 or claim 8, wherein the step of storing the tires further comprises:
- a step of calculating a stability and chaining criterion making it possible to know whether the position of the tire is stable in each position retained at the end of the selection of the candidate positions of the tire; and
- a conversion step during which the 2D center of the tire from the positioning algorithm is converted to the 3D center of the tire in the target location;
such that the positioning algorithm indicates the center of the tire in the rear plane of the tire group as well as its orientation (α) to position the tire in the target location so as to obtain the 3D center of the tire and couple it with the orientation (α). The chaining method of claim 9, wherein, during the step of calculating a stability and chaining criterion:
- the stability of the position of the new tire depends on the vertical (E) starting from its center of gravity and the support zone (F) of the new tire; and
- the chaining criterion is based on a signed horizontal distance between the center of the last tire installed and the end of the new tire. The chaining method of any one of claims 4 to 10, wherein, during the tire scanning step performed during the positioning step:
- the chaining system scans an area in which the last tire was positioned; and
- an attention mechanism is then used to detect the last positioned tire. A system for automatically chaining tires from a group of tires in an unknown arrangement and for which a predetermined target location is to be achieved, characterized in that the chaining system comprises:
- at least one gripping device (100) comprising a robot having a gripping device (104) supported by an elongated arm (106) pivotable and extending from the elongated arm (106) to a free end (104a) where a gripper (108) is arranged along a common longitudinal axis;
a detection system for collecting information on the physical environment around each gripping device (100), the detection system comprising one or more sensors configured to perform two-dimensional (2D) and/or three-dimensional (3D) image detection; and
- a communication network which manages incoming data to the chaining system from at least one socket device (100), the communication network comprising at least one communication server for executing programmed instructions stored in a memory of one or more processors of the chaining system which employs a positioning algorithm to implement the chaining method of any one of claims 1 to 11;
such that each cleaning device (100) is configured on one or more parameters of at least one identified tire calculated by an image processing module incorporated in the memory of the processor to set the cleaning device (100) in motion so that the gripping device (104) can carry out the gripping of a target tire during the chaining process carried out by the chaining system. The chaining system of claim 12, wherein the detection system comprises at least one RGB-D type camera (200) attached to at least one of the gripping device (104), the elongated arm (106) and the gripper (108) of the cleaning device (100). The chaining system of claim 12 or claim 13, wherein the predetermined target location comprises at least one location selected from one or more containers, one or more trucks, one or more trailers, one or more warehouses, one or more distribution centers, one or more conveyors, one or more pallets, and one or more storage means.