CN118354874A

CN118354874A - Front loading algorithm of large-object robot

Info

Publication number: CN118354874A
Application number: CN202280080332.8A
Authority: CN
Inventors: J·D·哈米尔顿; J·A·班尼斯特-苏顿; B·J·弗里泽尔
Original assignee: Kratebot Co ltd
Current assignee: Kratebot Co ltd
Priority date: 2021-10-08
Filing date: 2022-10-11
Publication date: 2024-07-16
Also published as: CN116709962A

Abstract

Disclosed herein is a method and system in which a robot handles objects that are large, awkward, highly deformable, or otherwise difficult to accommodate and carry. The robot is operated to navigate through the environment and detect and classify objects using the sensing system. The robot determines the type, size, and location of the object and classifies the object based on the detected attributes. The gripper pad arm and gripper pad move the other object away and the target object to be carried onto the shovel. The robot follows a procedure optimized for the type of object to be transported, maneuvering the object into and out of a receiving area comprising a shovel and a gripper pad arm. Large, clumsy, highly deformable, or otherwise difficult to manipulate objects may be managed by the methods disclosed herein.

Description

Front loading algorithm of large-object robot

The present application claims the benefit of U.S. provisional patent application No. 63/253,812, filed on 8 th 10 of 2021, and U.S. provisional patent application No. 63/253,867, filed on 8 th 10 of 2021, the contents of each of which are incorporated herein by reference in their entirety.

Background

The object under the foot not only means trouble but also is a safety hazard. Thousands of people fall in the home each year to be injured. The floor filled with scattered objects may represent a hazard, but many people have limited time to handle their home debris. Automated cleaning or robotics may represent an effective solution.

However, some objects present various challenges in how they are effectively captured and contained for transport to the appropriate repository or storage location. Objects that are proportionally larger than the containment area may not be simply swept and moved. A group of small, lightweight objects may scatter or scroll with initial contact and capturing one of them at a time may present a drain on both time and robot energy. A highly deformable object may simply slide out of the rigid capture mechanism or come together away from the rigid capture mechanism. And some objects may be carefully stacked in order, but if simply thrown away and left when they land, they may exhibit an unpleasant dispersion and messy stacking.

Thus, there is a need for a capture, containment, transport, and storage algorithm that takes into account the geometry and functionality of robotic components, as well as the potential difficulties associated with certain object types.

In one aspect, a method includes receiving a starting position and an attribute of a target object to be lifted by a robot. The robot comprises a robot control system, a shovel, a gripper pad arm with a gripper pad, and at least one wheel or one track for mobility of the robot. The method also includes determining an object isolation policy, including at least one of: reinforcement learning based policies, rules based policies including rewards and penalties are used, depending on observations, current object states and sensor data. The method also includes executing an object isolation policy to isolate the target object from other objects. The method further comprises the steps of: determining a pick-up strategy, comprising: a path of approach for the robot to go to the target object; a grip height for initial contact with the target object; a capture mode for movement of the gripper pad when capturing the target object; and a load bearing position of the gripper pad and the shovel, the load bearing position securing the target object in a containment region on the robot for transport, the containment region including at least two of the gripper pad arm, the gripper pad, and the shovel. The method further comprises the steps of: executing a pick-up strategy comprising: extending the gripper pad outwardly and forwardly relative to the gripper pad arm and raising the gripper pad to a gripping height; approaching the target object via the approach path, stopping when the target object is positioned between the gripper pads; performing a capture mode to allow capture of a target object within the containment area; and confirming that the target object is within the containment area. The method includes, under conditions where the target object is within the containment area: applying pressure to the target object with the gripper pad to hold the target object stationary in the containment region; and raising the shovel and gripper pad holding the target object to the load-bearing position. The method further includes, under a condition that the target object is not within the accommodation region: the pick-up strategy is changed using at least one of: different reinforcement learning based policies, different rule based policies, and depending on different observations, current object states, and sensor data; and executing the changed pick-up policy.

In one aspect, a robotic system includes: a robot comprising a shovel, a gripper pad arm with a gripper pad, at least one wheel or one track for mobility of the robot; a processor; and a memory storing instructions that, when executed by the processor, allow operation and control of the robot. The robotic system also includes a base station. The robotic system also includes a plurality of bins storing objects. The robotic system also includes a robotic control system. The robotic system also includes logic that allows the robot and robotic system to perform the disclosed actions.

Drawings

To readily identify the discussion of any particular element or act, the most significant bit or bits in the reference numeral refer to the figure number that first introduced the element.

Fig. 1A-1D illustrate aspects of a robot 100 according to one embodiment.

Fig. 2A illustrates a lowered shovel position and a lowered gripper position 200a of the robot 100 according to one embodiment.

Fig. 2B illustrates a lowered shovel position and an elevated gripper position 200B of the robot 100 according to one embodiment.

Fig. 2C illustrates an elevated shovel position and elevated gripper position 200C of robot 100 according to one embodiment.

Fig. 2D illustrates a robot 100 with an extended gripper pad 200D according to one embodiment.

Fig. 2E illustrates the robot 100 with the gripper pad 200E retracted according to one embodiment.

Fig. 3A illustrates a lowered shovel position and a lowered gripper position 300a of the robot 100 according to one embodiment.

Fig. 3B illustrates a lowered shovel position and an elevated gripper position 300B of the robot 100 according to one embodiment.

Fig. 3C illustrates an elevated shovel position and elevated gripper position 300C of robot 100 according to one embodiment.

Fig. 4A illustrates a lowered shovel position and a lowered gripper position 400a of the robot 100 according to one embodiment.

Fig. 4B illustrates a lowered shovel position and an elevated gripper position 400B of the robot 100 according to one embodiment.

Fig. 4C illustrates an elevated shovel position and elevated gripper position 400C of robot 100 according to one embodiment.

Fig. 5 illustrates a forward drop position 500 of the robot 100 according to one embodiment.

Fig. 6 illustrates a robot 600 according to one embodiment.

Fig. 7 illustrates a robot 700 according to one embodiment.

Fig. 8 illustrates a robot 800 according to one embodiment.

Fig. 9 illustrates a robot 900 according to one embodiment.

Fig. 10 illustrates a robot 1000 according to one embodiment.

Fig. 11 illustrates a robot 1100 according to one embodiment.

Fig. 12 illustrates a robot 1200 according to one embodiment.

Fig. 13 illustrates a robot 1200 according to one embodiment.

Fig. 14 illustrates a robot according to one embodiment.

Fig. 15 illustrates a robot 1500 according to one embodiment.

Fig. 16 illustrates a robot 1600 according to one embodiment.

Fig. 17 illustrates a robot 1700 according to one embodiment.

Fig. 18A and 18B illustrate a robot 1800 according to one embodiment.

Fig. 19 illustrates an embodiment of a robotic control system 1900 for implementing the components and processing steps of the systems described herein.

FIG. 20 illustrates a sensor input analysis 2000, according to one embodiment.

Fig. 21 illustrates a main navigation, collection, and deposit process 2100, according to one embodiment.

Fig. 22 illustrates policy steps for the quarantine policy, pick policy, and drop policy 2200 in accordance with one embodiment.

FIG. 23 illustrates a process 2300 for determining an action according to a policy, according to one embodiment.

Fig. 24 illustrates a deposit process 2400 portion of the disclosed algorithm, according to an embodiment.

FIG. 25 illustrates a capture process 2500 portion of the disclosed algorithm, according to one embodiment.

Fig. 26 illustrates a first portion of a process diagram 2600, according to one embodiment.

Fig. 27A-27D illustrate a process for stackable object 2700, according to one embodiment.

Fig. 28A-28D illustrate a process 2900 for a large, highly deformable object, according to one embodiment.

Fig. 29A and 29C illustrate a process 2900 for a large, highly deformable object, according to one embodiment.

Fig. 29B illustrates an aspect of the subject matter in accordance with one embodiment.

Fig. 29D illustrates an aspect of the subject matter in accordance with one embodiment.

Fig. 30A and 30C illustrate a process 3000 for a small, easily scattered object, according to one embodiment.

FIG. 30B illustrates an aspect of the subject matter in accordance with one embodiment.

FIG. 30D illustrates an aspect of the subject matter in accordance with one embodiment.

Fig. 31 depicts a robotic system 3100 according to one embodiment.

Fig. 32 depicts a robotic process 3200 according to one embodiment.

Fig. 33 depicts another robotic process 3300 according to one embodiment.

Fig. 34 depicts a state space diagram 3400 for a robotic system according to one embodiment.

Fig. 35 depicts a robot control algorithm 3500 for a robotic system, according to one embodiment.

Fig. 36 depicts a robot control algorithm 3600 for a robotic system, according to one embodiment.

Fig. 37 depicts a robot control algorithm 3700 according to one embodiment.

Fig. 38 illustrates a robotic system 3800 according to one embodiment.

Detailed Description

Embodiments of a robotic system are disclosed that operate a robot to navigate an environment using cameras to map types, sizes, and positions of toys, clothing, obstacles, and other objects. The robot includes a neural network for determining the type, size, and position of an object based on inputs from a sensing system, such as images from forward cameras, rear cameras, forward and rear left/right stereo cameras, or other camera configurations, and data from Inertial Measurement Units (IMUs), lidar, odometers, and actuator force feedback sensors. The robot selects a particular object to pick up, performs path planning, and navigates to a point adjacent to and facing the target object. The actuated gripper pad arm moves the other object away and manipulates the gripper pad to move the target object to be carried onto the blade. The shovel is tilted slightly upward and, if desired, the gripper pads can be closed in front to hold the object in place while the robot navigates to the next location in the planned path (such as the storage destination).

In some embodiments, the system may include a robotic arm for reaching and gripping the raised objects and moving them down onto the shovel. In some embodiments, a companion "portable elevator" robot may also be used to raise the main robot onto a countertop, table, or other raised surface, and then lower it back onto the floor. Some embodiments may utilize an up/down vertical lift (e.g., a scissor lift) to change the height of the shovel as the item is lowered into a container, rack, or other high or raised position.

Some embodiments may also utilize one or more of the following components:

a left/right rotating brush on the actuator arm that pushes the object onto the shovel

A driving gripper that grabs objects and moves them onto the shovel

A rotating wheel with a flap which pushes the object from above onto the shovel

A servo or other actuator for lifting the front shovel into the air; and a separate actuator for tilting the shovel forward and downward to drop the object into the container

A variant of scissor lift, which lifts the shovel upwards and gradually tilts backwards as it gets height

A ramp on the container, with the front shovel on the hinge, so that the robot pushes the item up the ramp, so that the object falls by gravity into the container at the top of the ramp

The bin on the robot is used for additional load carrying capacity so that the target object is pushed up a ramp into the bin instead of using a front shovel, and the bin is tilted up and back like a dump truck to drop items into the container.

Robotic systems may be used for automated organization of surfaces, wherein items left on the surface are automatically binned into containers in a regular schedule. In one particular embodiment, the system may be used to automatically groom a child's play area (e.g., in a home, school, or business) where toys and/or other items are automatically returned to a container specific to a different object type after the child completes the game. In other embodiments, the system may be used to automatically pick up laundry from the floor and organize the laundry into wash basket(s) for washing, or to automatically pick up waste from the floor and place it (e.g., by type (plastic, cardboard, glass)) into a waste bin or recovery bin(s). In general, the system may be deployed for efficiently picking up various different objects from a surface, and may learn to pick up new types of objects.

Some objects have properties that make them difficult to maneuver and carry using the gripper pad, gripper pad arm, and spade. These difficulties can be overcome by following algorithms that specifically consider the attributes that create the difficulties. For example, an object that is too large to fit completely within the shovel may be partially secured within the shovel by: the gripper pad is positioned above the center of gravity of the subject and the gripper pad arm is lowered slightly so that the gripper pad applies a slight downward pressure and holds the subject firmly within the shovel or even against the bottom or edge of the shovel. Small, light and easily scattered objects such as plastic building blocks or marble may be scattered if swept too fast with the gripper pads. Alternatively, the pad may contact such objects at a point of height, wherein the direct and constant pressure of the pad may be used to firmly press the objects to the floor rather than sweeping them along the floor. In such cases, an initially decreasing force may be applied and then increased as the object begins to move, or a series of gentle slapping motions may be employed by the gripper pads to apply a horizontal force to move the object while avoiding downward forces that may increase friction of the object with the floor and prevent movement of the object. While many objects may simply fall from the scoop (such as into a designated bin) at their destination, a flat, stackable object of the type such as a book, CD, DVD, narrow box, etc. may be more easily and neatly placed by being lifted over a previously stacked object and manipulated out of the scoop by a gripper pad. An algorithm for handling objects such as these is disclosed herein.

Fig. 1A to 1D illustrate a robot 100 according to one embodiment. Fig. 1A illustrates a side view of the robot 100, and fig. 1B illustrates a top view. The robot 100 may include a chassis 102, a mobility system 104, a sensing system 106, a capture and containment system 108, and a robot control system 1900. The capture and containment system 108 may further include a blade 110, a blade arm 112, a blade arm pivot point 114, two gripper pads 116, two gripper pad arms 118, and two pad arm pivot points 122.

Chassis 102 may support and house other components of robot 100. The mobility system 104 may include wheels as indicated, as well as cattler tracks, conveyor belts, and the like, as is well known in the art. The mobility system 104 may further include motors, servers, or other rotational or kinetic energy sources to propel the robot 100 along a desired path of the robot 100. The components of mobility system 104 may be mounted on chassis 102 for the purpose of moving the entire robot without impeding or inhibiting the range of motion required by capture and containment system 108. In at least some configurations of the chassis 102, the blade 110, the gripper pad 116, and the gripper pad arm 118 with respect to each other, elements of the sensing system 106 (such as a camera, lidar sensor, or other components) may be mounted on the chassis 102 in a position that imparts a clear line of sight to the robot 100 about the environment of the robot 100.

The chassis 102 may house and protect all or a portion of the robotic control system 1900 (portions of the robotic control system 1900 may also be accessed via a connection to a cloud server), in some embodiments, the robotic control system 1900 includes a processor, memory, and connections to the mobility system 104, the sensing system 106, and the capture and containment system 108. The chassis 102 may contain other electronic components (such as batteries, wireless communication devices, etc.), as is well known in the robotic arts. The robotic control system 1900 may function as described in more detail with respect to fig. 19. The mobility system 104 and/or the robotic control system 1900 may include motor controllers for controlling the speed, direction, position, and smooth movement of the motors. Such controllers may also be used to detect force feedback and limit maximum current (providing over-current protection) to ensure safety and prevent damage.

The capture and containment system 108 may include a blade 110, a blade arm 112, a blade arm pivot point 114, a gripper pad 116, a gripper pad arm 118, a pad arm pivot point 120, and a pad arm pivot point 122. In some embodiments, the capture and containment system 108 may include two gripper pad arms 118, a gripper pad 116, and their pivot points. In other embodiments, the gripper pad 116 may be directly attached to the blade 110 without the gripper pad arm 118. Such embodiments are illustrated later in this disclosure.

As more clearly illustrated in fig. 2A-2E, the geometry of the shovel 110, as well as the placement of the gripper pads 116 and gripper pad arms 118 relative to the shovel 110, may describe a containment region in which objects may be safely carried. A servo, direct Current (DC) motor or other actuator at the shovel arm pivot point 114, the pad pivot point 120, and the pad arm pivot point 122 may be used to adjust the settings of the shovel 110, the gripper pad 116, and the gripper pad arm 118 between the fully lowered shovel and gripper position and the raised shovel and gripper position, as illustrated with respect to fig. 2A-2C.

The connection points shown between the shovel arm and the gripper pad arm are exemplary locations and are not intended to limit the physical location of such connection points. Such connection may be made in a variety of locations suitable for the construction of the chassis and arms, as well as the application of the intended use.

In some embodiments, the gripping surface may be configured on a side of the gripper pad 116 facing the object to be lifted. These gripping surfaces may provide cushioning, gravel, elasticity, or some other feature that increases friction between the gripper pad 116 and the object to be captured and contained. In some embodiments, gripper pad 116 may include suction cups to better grip objects with smooth, flat surfaces. In some embodiments, the gripper pad 116 may be configured with sweeping bristles. These sweeping bristles may help move the small object up from the floor onto the shovel 110. In some embodiments, the sweeping bristles may angle downwardly and inwardly from the grabber pad 116 such that when the grabber pad 116 sweeps the object towards the shovel 110, the sweeping bristles form a ramp, allowing the foremost bristles to slide under the object and guide the object upwardly towards the grabber pad 116, which helps to capture the object within the shovel and reduces the tendency of the object to be pressed against the floor, increasing its friction and making it more difficult to move.

Fig. 1C and 1D illustrate side and top views, respectively, of chassis 102, and general connectivity of components of mobility system 104, sensing system 106, and communication 134 associated with robotic control system 1900. In some embodiments, the communication 134 may include a network interface 1912 described in more detail with respect to the robotic control system 1900.

In one embodiment, mobility system 104 may include a right front wheel 136, a left front wheel 138, a right rear wheel 140, and a left rear wheel 142. The robot 100 may have a front wheel drive in which the right front wheel 136 and the left front wheel 138 are actively driven by one or more actuators or motors, while the right rear wheel 140 and the left rear wheel 142 support the rear portion of the chassis 102 while passively rotating on an axis. In another embodiment, the robot 100 may have a rear wheel drive in which the right and left rear wheels 140, 142 are actuated and the front wheels are passively rotated. In another embodiment, each wheel may be actively actuated by a separate motor or actuator.

The sensing system 106 may further include cameras 124, such as front camera 126 and rear camera 128, light detection and ranging (LIDAR) sensors, such as LIDAR sensor 130, and Inertial Measurement Unit (IMU) sensors, such as IMU sensor 132. In some embodiments, the front cameras 126 may include a right front camera 144 and a left front camera 146. In some embodiments, the rear camera 128 may include a rear left camera 148 and a rear right camera 150.

Additional embodiments of robots that can be used to perform the disclosed algorithms are illustrated in fig. 2A-2E, 3A-3C, 4A-4C, 5, 6,7, 8, 9, 10, 11, 12, and 13. Fig. 15, 16 and 17, and 18A and 18B illustrate exemplary mechanical embodiments of robots that may be used to execute the disclosed algorithms.

Fig. 2A illustrates a robot 100 such as introduced with reference to fig. 1A disposed in a lowered shovel position and a lowered gripper position 200 a. In this configuration, gripper pad 116 and gripper pad arm 118 rest in lowered gripper position 204, and shovel 110 and shovel arm 112 rest in lowered shovel position 206 at front 202 of robot 100. In this position, the blade 110 and gripper pad 116 may roughly describe the receiving area 210 as shown.

Fig. 2B illustrates robot 100 with lowered shovel position and raised gripper position 200B. By actuation of a servo or other actuator at pad pivot point 120 and pad arm pivot point 122, gripper pad 116 and gripper pad arm 118 may be raised to raised gripper position 208 while blade 110 and blade arm 112 remain in lowered blade position 206. In this configuration, the gripper pad 116 and the shovel 110 may be roughly described as a receiving region 210 as shown, wherein an object that is higher than the shovel 110 may rest within the shovel 110 and be held in place by the pressure exerted by the gripper pad 116.

Pad arm pivot point 122, pad pivot point 120, shovel arm pivot point 114, and shovel pivot point 502 (shown in fig. 5) may provide robot 100 with a range of motion beyond those components illustrated herein. The positions shown in the disclosed figures are merely illustrative and are not meant to indicate limitations in the range of motion of the components of the robot.

Fig. 2C illustrates robot 100 with an elevated shovel position and an elevated gripper position 200C. Gripper pad 116 and gripper pad arm 118 may be in raised gripper position 208 while blade 110 and blade arm 112 are in raised blade position 212. In this position, the robot 100 may be able to allow objects to fall from the shovel 110 and the gripper pad arm 118 to an area at the rear 214 of the robot 100.

As illustrated in fig. 27A-30C below, the load-bearing position may involve the placement of gripper pads 116, gripper pad arms 118, blade 110, and blade arm 112 in a relative configuration between lowered blade position and lowered gripper position 200a and the extremes of raised blade position and raised gripper position 200C.

Fig. 2D illustrates the robot 100 with the gripper pad 200D extended. By action of a servo or other actuator at pad pivot point 120, gripper pad 116 may be configured as a cantilevered gripper pad 216 to allow robot 100 to approach an object as wide as robot chassis 102 and blade 110 or wider than robot chassis 102 and blade 110. In some embodiments, gripper pads 116 may be capable of rotating almost 360 degrees to rest parallel with their associated gripper pad arms 118 and on the outside of their associated gripper pad arms 118 when fully extended.

Fig. 2E illustrates the robot 100 with the gripper pad 200E retracted. The closed gripper pad 218 may roughly define the receiving area 210 by its position relative to the blade 110. In some embodiments, gripper pad 116 can rotate farther than shown, almost three hundred sixty degrees, to rest parallel to and inside the side wall of blade 110.

Fig. 3A-3C illustrate a robot 100 such as introduced with reference to fig. 1A-2E. In such embodiments, the gripper pad arm 118 may be controlled by a servo or other actuator at the same connection point 302 with the chassis 102 as the dipper arm 112. It can be seen that robot 100 is disposed in lowered shovel position and lowered gripper position 300a, lowered shovel position and raised gripper position 300b, and raised shovel position and raised gripper position 300 c. The robot 100 may be configured to perform the algorithms disclosed herein.

The connection point shown between the boom 112/grabber pad arm 118 and the chassis 102 is an exemplary location and is not intended to limit the physical location of the connection point. Such connections may be made in a variety of locations suitable for the configuration of the chassis 102 and arms, as well as the application of the intended use.

Fig. 4A to 4C illustrate a robot 100 such as introduced with reference to fig. 1A to 2E. In such embodiments, the gripper pad arm 118 may be controlled by one or more servos (or other actuators) at a different connection point 402 to the chassis 102 than the connection point 402 to the control shovel arm 112. It can be seen that robot 100 is disposed in lowered shovel position and lowered gripper position 400a, lowered shovel position and raised gripper position 400b, and raised shovel position and raised gripper position 400 c. The robot 100 may be configured to perform the algorithms disclosed herein.

The different connection points 402 shown between the dipper arm and chassis and the gripper pad arm and chassis are exemplary locations and are not intended to limit the physical location of these connection points. Such connection may be made in a variety of locations suitable for the construction of the chassis and arms, as well as the application of the intended use.

Fig. 5 illustrates the robot 100 in a forward drop position 500, such as previously introduced. The arms of the robot 100 may be positioned to form the receiving area 210 as previously described.

Robot 100 may be configured with a blade pivot point 502, at which point blade 110 is connected to blade arm 112. The shovel pivot point 502 may allow the shovel 110 to tilt forward and downward while the shovel arm 112 is raised, allowing objects in the containment region 210 to slide out and store in the region of the front 202 of the robot 100.

Fig. 6 illustrates a robot 600 according to one embodiment. The robot 600 may be configured to perform the algorithms disclosed herein. The robot 600 may include a number of features previously introduced, which are not described in detail herein. In addition to these, robot 600 may include a raised rear boom pivot point 602, a split boom 604, a right telescopic gripper pad arm 606, a left telescopic gripper pad arm 608, a linear actuator 610, a wrist actuator 612, a right front drive wheel 614, a left front drive wheel 616, and rear casters 618.

The raised rear boom pivot point 602 may be connected to a split boom 604 to raise and lower the blade 110. This configuration may allow the front camera 126 to capture images without obstruction by the shovel arm. The groove or slot in the chassis 102 of the robot 600 may allow portions of the split shovel arm 604 to move unimpeded by the dimensions of the chassis 102.

The gripper pad arms may include a right telescoping gripper pad arm 606 and a left telescoping gripper pad arm 608. In this way, the gripper pad arm can extend (increase in length) and retract (decrease in length). The motion may be generated by a linear actuator 610 configured as part of the gripper pad arm. The wrist actuator 612 may be positioned at the pad pivot point 120, allowing the gripper pad to pivot and push the subject into the blade 110.

In one embodiment, the mobility system may include a right front drive wheel 614, a left front drive wheel 616, and a rear caster 618. The front drive wheels may provide motive force that allows the robot 600 to navigate in its environment, while the rear casters 618 may provide support to the rear portion of the robot 600 without limiting its range of motion. The front right drive wheel 614 and the front left drive wheel 616 may be independently actuated, allowing the robot 600 to rotate in place and when traversing the floor.

Fig. 7 illustrates a robot 700 according to one embodiment. The robot 700 may be configured to perform the algorithms disclosed herein. The robot 700 may include a number of features previously introduced, which are not described in detail herein. In addition to these, robot 700 may include a single gripper pad 702, a single telescoping gripper pad arm 704, a linear actuator 706, bearings 708, opposing pad arm pivot points 710, sliding joints 712, and opposing gripper pad arms 714.

The single gripper pad 702 may be raised and lowered by the gripper pad arm and may additionally be extended and retracted by the action of the single telescopic gripper pad arm 704 advanced by the linear actuator 706. The bearing 708 at the opposite pad arm pivot point 710 and the bearing 708 at the sliding joint 712 in the opposite gripper pad arm 714 may allow for the force of the linear actuator 706 to be transferred through the single gripper pad 702 to allow for motion symmetry in both gripper pads in the event that one of the arms is actively moving. In another embodiment, a single gripper pad 702 may be positioned by actuating right and left telescoping gripper pad arms 606 and 608 simultaneously, as illustrated in fig. 6.

Fig. 8 illustrates a robot 800 according to one embodiment. The robot 800 may be configured to perform the algorithms disclosed herein. The robot 800 may include a number of features previously introduced, which are not described in detail herein. In addition to these, the robot 800 may also include a raised rear boom pivot point 802 and a single boom 804.

The raised rear boom pivot point 802 may be connected to a single boom 804 to raise and lower the blade 110. This configuration may allow the front camera 126 to capture images without obstruction by the shovel arm. A groove or slot in the chassis 102 of the robot 800 may allow a single dipper arm 804 to move unimpeded by the dimensions of the chassis 102.

Fig. 9 illustrates a robot 900 according to one embodiment. The robot 900 may be configured to perform the algorithms disclosed herein. The robot 900 may include a number of features previously introduced, which are not described in detail herein. In addition to these, robot 900 may also include a single telescoping boom 902 and a linear actuator 904.

A single telescopic boom 902 may be capable of moving the blade 110 away from and toward the chassis 102 by extending and retracting powered by a linear actuator 904.

Fig. 10 illustrates a robot 1000 according to one embodiment. The robot 1000 may be configured to perform the algorithms disclosed herein. The robot 1000 may include a number of features previously introduced, which are not described in detail herein. In addition to these, robot 1000 may also include a spade gripper pad arm 1002 and a spade pad arm pivot point 1004.

Unlike connection to chassis 102 as seen in other embodiments disclosed herein, spade gripper pad arm 1002 may be connected to a spade pad arm pivot point 1004 that is positioned on spade 110. The actuator at the spade-style pad arm pivot point 1004 may allow the spade-style gripper pad arm 1002 to be raised and lowered relative to the spade 110 in addition to being raised and lowered along with the spade 110.

Fig. 11 illustrates a robot 1100 according to one embodiment. The robot 1100 may be configured to perform the algorithms disclosed herein. Robot 1100 may include a number of features previously introduced that are not described in detail herein. In addition to these, robot 1100 may also include a spade gripper pad 1102 and a spade pad pivot point 1104.

The spade-mounted gripper pad 1102 may be connected to the spade 110 at a spade-mounted pad pivot point 1104 instead of to the gripper pad arm. The wrist actuator at the spade-style pad pivot point 1104 may allow the spade-style gripper pad 1102 to pivot into the spade 110 and away from the spade 110 in order to move the subject into the spade 110.

Fig. 12 illustrates a robot 1200 according to one embodiment. The robot 1200 may be configured to perform the algorithms disclosed herein. The robot 1200 may include a number of features previously introduced, which are not described in detail herein. In addition to these, robot 1200 may also include a split shovel arm 1202, a raised rear shovel arm pivot point 1204, a shovel pivot point 1206, and a shovel pivot actuator 1208.

The robot 1200 may have a split shovel arm 1202 connected to the chassis 102 at two raised rear shovel arm pivot points 1204. The actuators at each raised rear shovel arm pivot point 1204 may be actuated simultaneously to raise and lower shovel 110.

Blade 110 may be connected to split blade arm 1202 at blade pivot point 1206. A shovel pivot actuator 1208 at the shovel pivot point 1206 may allow the shovel 110 to be lifted by the split shovel arm 1202 and tilted forward and downward into the forward drop position 500, such as illustrated in fig. 5.

Fig. 13 illustrates a robot 1300 according to one embodiment. Robot 1300 may be configured to perform the algorithms disclosed herein. Robot 1300 may include a number of features previously introduced that are not described in detail herein. In addition to these, robot 1300 may also include mobility system 104 equipped with tracks 1302.

These tracks 1302 may improve mobility and stability of the robot 1300 on some surfaces. The left and right tracks 1302 may each be separately actuated to allow the robot 1300 to rotate while traversing or while remaining in place.

Fig. 14 illustrates a robot 1400 according to one embodiment. Robot 1400 may be configured to perform the algorithms disclosed herein. Robot 1400 may include a number of features previously introduced that are not described in detail herein. In addition to these, robot 1300 may also include chassis 102, blade 110, single gripper pad arm 1402, single pad arm pivot point 1404, single gripper pad 1406, and single pad pivot point 1408.

The single gripper pad arm 1402 may be connected to the chassis 102 at a single pad arm pivot point 1404, allowing the single gripper pad arm 1402 to move relative to the robot 1400. The single gripper pad arm 1402 may have a single gripper pad 1406 connected to the single gripper pad arm 1402 at a single pad pivot point 1408, allowing the single gripper pad 1406 to move relative to the single gripper pad arm 1402. A servo, DC motor, or other actuator at the single pad arm pivot point 1404 and the single pad pivot point 1408 may advance the motion of the single gripper pad 1402 and the single gripper pad 1406 to maneuver the subject into the blade 110.

Fig. 15 illustrates a robot 1500 according to one embodiment. Robot 1500 may be configured to perform the algorithms disclosed herein. Robot 1500 may include, among other features, chassis 102, blade 110, blade arm 112, blade arm pivot point 114, gripper pad 116, gripper pad arm 118, pad pivot point 120, pad arm pivot point 122, right front camera 144, left front camera 146, and different connection points 402 of blade arm 112 and gripper pad arm 118, as previously described.

Fig. 16 illustrates a robot 1600 according to one embodiment. Robot 1600 may be configured to perform the algorithms disclosed herein. Robot 1600 may include, among other features, gripper pad 116, right telescoping gripper pad arm 606, left telescoping gripper pad arm 608, the same connection point 302 between each of the shovel arms 112 and these gripper pad arms, and linear actuators 610 for extending right telescoping gripper pad arm 606, left telescoping gripper pad arm 608, and gripper pad 116, either simultaneously or individually, as previously described.

Fig. 17 illustrates a robot 1700 according to one embodiment. The robot 1700 may be configured to perform the algorithms disclosed herein. Robot 1700 may include, among other features, a single gripper pad 702, a single telescoping gripper pad arm 704, a linear actuator 706, bearings 708, an opposing pad arm pivot point 710, and an opposing gripper pad arm 714, as previously described.

Fig. 18A and 18B illustrate a robot 1800 according to one embodiment. Robot 1800 may be configured to perform the algorithms disclosed herein. Robot 1800 may include, among other features, mobility system 104, lidar sensor 130, and crawler 1302, as previously described.

Features of the robots illustrated with respect to fig. 6-18B may be present in various combinations in particular embodiments. These illustrations are not intended to limit the configuration of the features described, as will be readily appreciated by one of ordinary skill in the art.

Fig. 19 depicts an embodiment of a robotic control system 1900 for implementing the components and process steps of the systems described herein. Some or all portions of the robot control system 1900 and its operational logic may be contained within physical components of the robot and/or within a cloud server in communication with the robot. In one embodiment, aspects of the robotic control system 1900 on the cloud server may control more than one robot at a time, allowing multiple robots to work cooperatively within the workspace.

The input device 1904 (e.g., an input device of a robot or a companion device such as a mobile phone or personal computer) includes a transducer that converts physical phenomena into signals (typically electrical, optical, or magnetic) internal to the machine. The signal may also be a wireless signal in the form of electromagnetic radiation in the Radio Frequency (RF) range, but potentially also a signal in the infrared or optical range. Examples of input devices 1904 are touch sensors that respond to touch or physical pressure from an object, or proximity of an object to a surface, mice that respond to movement through space or across a plane, microphones that convert vibrations in a medium (typically air) into device signals, scanners that convert optical patterns on two-or three-dimensional objects into device signals. Signals from the input device 1904 are provided to the memory 1906 via various machine signal conductors (e.g., bus or network interface) and circuits.

Memory 1906 is commonly referred to as a first level or second level memory device, providing information for storing signals received from input device 1904 (via substance or configuration of substance states), instructions and information for controlling the operation of central processing unit or CPU 1902, and signals from storage device 1910. Memory 1906 and/or storage 1910 may store computer-executable instructions and thereby form logic 1914, which logic 1914, when applied to and executed by CPU 1902, implements embodiments of the processes disclosed herein. Logic 1914 may include portions of a computer program executed by CPU 1902 or another processor, as well as configuration data. Logic 1914 may include one or more machine learning models 1916 for performing the disclosed actions. In one embodiment, portions of logic 1914 may also reside on a mobile or desktop computing device accessible to the user to facilitate direct control of the robot by the user.

The information stored in memory 1906 is typically directly accessible by the CPU 1902 of the device. The signals input to the device cause a reconfiguration of the internal material/energy states of the memory 1906, thereby essentially creating a new machine configuration, affecting the behavior of the robot control system 1900 by configuring the CPU 1902 with control signals (instructions) and data provided with the control signals.

The second or third level storage 1910 may provide slower but higher capacity machine memory capabilities. Examples of storage devices 1910 are hard disks, optical disks, mass flash memory or other non-volatile memory technology, and magnetic memory.

In one embodiment, memory 1906 may include virtual storage that is accessible through connection to a cloud server using network interface 1912, as described below. In such embodiments, some or all of logic 1914 may be stored and processed remotely.

The CPU 1902 may cause the configuration of the memory 1906 to be changed by a signal in the storage device 1910. In other words, the CPU 1902 may cause data and instructions to be read from the storage device 1910 in the memory 1906, then the data and instructions from the memory 1906 may affect the operation of the CPU 1902 as instructions and data signals, and the data and instructions may also be provided from the memory 1906 to the output device 1908. The CPU 1902 may change the contents of the memory 1906 by: the machine interface of memory 1906 is signaled to change the internal configuration, and then the signal is converted to memory device 1910 to change its material internal configuration. In other words, data and instructions may be backed up from memory 1906, which is typically volatile, to storage 1910, which is typically nonvolatile.

The output device 1908 is a transducer that converts the signals received from the memory 1906 into physical phenomena such as vibration in air, or patterns of light on a machine display, or patterns of vibration (i.e., haptic devices) or ink or other materials (i.e., printers and 3D printers).

The network interface 1912 receives signals from the memory 1906 and converts them, typically via a machine network, to electrical, optical or wireless signals to other machines. The network interface 1912 also receives signals from a machine network and converts them into electrical, optical, or wireless signals to the memory 1906. The network interface 1912 may allow the robot to communicate with cloud servers, mobile devices, other robots, and other network-enabled devices.

FIG. 20 illustrates a sensor input analysis 2000, according to one embodiment. The sensor input analysis 2000 may inform the robot 100 of the dimensions of its immediate environment 2002, as well as its own and the locations of other objects within the environment 2002.

The robot 100 as previously described includes a sensing system 106. The sensing system 106 may include at least one of the following: camera 124, IMU sensor 132, lidar sensor 130, odometer 2004, and actuator force feedback sensor 2006. These sensors may capture data describing the environment 2002 surrounding the robot 100.

Image data 2008 from camera 124 may be used for object detection and classification 2010. The object detection and classification 2010 may be performed by algorithms and models configured within the robot control system 1900 of the robot 100. In this way, the characteristics and types of objects in the environment 2002 may be determined.

Image data 2008, object detection and classification 2010 data, and other sensor data 2012 may be used for global/local map updates 2014. The global and/or local map may be stored by the robot 100 and may represent its knowledge of the dimensions and objects within its cleaning environment 2002. The graph may be used for navigation and policy determination associated with a cleaning task.

The robot may use a combination of cameras 124, lidar sensors 130, and other sensors to maintain a global or local area map of the environment and position itself within the global or local area map. Additionally, the robot may perform object detection and object classification, and may generate a visual re-identification fingerprint for each object. Robots can utilize stereo cameras and machine learning/neural network software architecture (e.g., semi-supervised or supervised convolutional neural networks) to efficiently classify the type, size, and location of different objects on a map of an environment.

The robot may determine the relative distance and angle to each object. The distances and angles may then be used to locate objects on the global or local region map. The robot may scan to both the front and back of the robot using both the forward and backward cameras.

Image data 2008, object detection and classification 2010 data, other sensor data 2012, and global/local map update 2014 data may be stored as observations, current robot state, current object state, and sensor data 2016. The observations, current robot state, current object state, and sensor data 2016 may be used by the robot control system 1900 of the robot 100 in determining navigation paths and task strategies.

Fig. 21 illustrates a main navigation, collection, and deposit process 2100, according to one embodiment. According to some examples, the method includes driving to a target object(s) at block 2102. For example, a robot 100 such as introduced with respect to fig. 1A may use a local or global map to navigate to a location near the target object(s) to drive to the target object(s) depending on the determined observations, current robot state, current object state, and sensor data 2016 as illustrated in fig. 20.

According to some examples, the method includes determining an object isolation policy at block 2104. For example, the robotic control system 1900 illustrated in fig. 1A may determine an object isolation policy to separate the target object(s) from other objects in the environment based on the location of the object(s) in the environment. The object isolation policy may be determined using a machine learning model or a rule-based approach, depending on the determined observations, current robot states, current object states, and sensor data 2016 as illustrated in fig. 20. In some cases, object isolation may not be required and the relevant box may be skipped. For example, in areas containing few items to be picked up and moved, or in areas where such items are not close to each other, furniture, walls or other obstacles, this may result in interference in picking up the target object, object isolation may not be required.

In some cases, there may be no effective isolation policy. For example, the robotic control system 1900 illustrated in fig. 1A may not be able to determine an effective isolation strategy. If it is determined at decision block 2106 that there is no valid quarantine policy, then the target object(s) may be marked as pickup failure at block 2120. The main navigation, collection, and deposit process 2100 may then proceed to block 2128 where the next target object(s) is determined.

If a valid quarantine policy is determined at decision block 2106, at block 2108, an object quarantine policy, such as that introduced with respect to robot 100 of FIG. 1A, may be performed to isolate the target object(s) from other objects. The quarantine policy may follow policy steps for quarantine, pick, and drop policies 2200 illustrated in fig. 22. The quarantine policy may be a reinforcement learning based policy that uses rewards and penalties in addition to observations, current robot state, current object state, and sensor data 2016, or a rule based policy that relies on determined observations, current robot state, current object state, and sensor data 2016 as illustrated in fig. 20. Reinforcement learning-based strategies that rely on rewards and penalties are described in more detail with reference to fig. 22.

The rule-based policy may use conditional logic to determine the next logic based on the observations, the current robot state, the current object state, and sensor data 2016 (such as developed in fig. 20). Each rule-based policy may have a list of available actions that it may consider. In one embodiment, a mobile collision avoidance system may be used to determine a range of motion that involves each action. Rules-based policies for object isolation may include:

Navigating the robot to a position facing the target object(s) to be isolated, but far enough to open the gripper pad arm and gripper pad and lower the shovel

Open the gripper pad arm and gripper pad, lower the gripper pad arm and gripper pad, and lower the shovel

Slightly turning the robot into position so that the target object(s) is centered in the front view

Open the gripper pad arm and gripper pad to be slightly wider than the target object(s)

Slowly drive forward until the gripper pad arm and the end of the gripper pad pass the target object(s) to be positioned

Slightly closing the gripper pad arm and gripper pad in a V-shape such that the gripper pad arm and gripper pad surround the target object(s)

Back-driving 100 cm, moving the target object(s) into the open space

According to some examples, the method includes determining whether the isolation was successful at decision block 2110. For example, the robotic control system 1900 illustrated in fig. 1A may determine whether the target object(s) were successfully isolated. If the quarantine policy is unsuccessful, the target object(s) may be marked as pickup failure at block 2120. The main navigation, collection, and deposit process 2100 proceeds to block 2128 where the next target object is determined. In some embodiments, instead of determining the next target object, a different policy may be selected for the same target object. For example, if the target object(s) cannot be quarantined by the current quarantine policy, a different quarantine policy may be selected and quarantine retried.

If the target object(s) were successfully isolated, the method includes determining a pick policy at block 2112. For example, the robotic control system 1900 illustrated in fig. 1A may determine a pick-up strategy. The pick-up strategy for the target object(s), as well as the location, may be determined using a machine learning model or rule-based method, depending on the determined observations, current robot state, current object state, and sensor data 2016 as illustrated in fig. 20.

In some cases, there may be no effective pick-up strategy. For example, the robotic control system 1900 illustrated in fig. 1A may not be able to determine an effective pick-up strategy. If it is determined at decision block 2114 that there is no valid pick policy, then the target object(s) may be marked as pick failed at block 2120, as previously mentioned. The pick-up strategy may need to take into account:

Initial default position of gripper pad arm and shovel before starting pick-up

Detection of the type of floor of the hard surface relative to the carpet, which may affect the pick-up strategy

Final shovel and gripper pad arm position for carrying

If a valid pick-up strategy is determined to be present at decision block 2114, then at block 2116, a pick-up strategy may be performed, such as with respect to the robot 100 introduced in FIG. 1A. The pick policy may follow policy step 2200 for the quarantine policy, pick policy, and drop policy illustrated in fig. 22. The pick-up strategy may be a reinforcement learning based strategy or a rule based strategy, depending on the determined observations, current robot state, current object state and sensor data 2016 as illustrated in fig. 20. The rule-based policy for object pick-up may include:

Navigating the robot to a position facing the target object(s), but far enough apart to open the gripper pad arm and gripper pad and lower the shovel

Forward drive until the target object(s) is in a "pick-up zone" relative to the edge of the blade

Determining the central position of the target object(s) relative to the shovel-either right, left or center

O if on the right side, the right gripper pad arm and gripper pad are closed first, then the left gripper pad arm and gripper pad are closed

O otherwise, first closing the left gripper pad arm and gripper pad, then closing the right gripper pad arm and gripper pad

Determining whether the target object(s) has been successfully pushed into the shovel

O if yes, pick up is successful

If not, the gripper pad arm and gripper pad are lifted and then retried at the appropriate portion of the strategy.

According to some examples, the method includes determining whether the target object(s) is picked up at decision block 2118. For example, the robotic control system 1900 illustrated in fig. 1A may determine whether the target object(s) are picked up. The pick-up success can be evaluated using the following steps:

Object detection within the area of the shovel and gripper pad arm (i.e., the containment area as previously illustrated) for determining whether an object is within the shovel/gripper pad arm/containment area

Force feedback from actuator force feedback sensor indicating that the object is retained by the gripper pad arm

Tracking the movement of the object(s) during the picking up of the object(s) to the area of the shovel, and keeping the state of these object(s) in memory (generally depending on the memory, since the object may no longer be visible when the shovel is in its carrying position)

Detecting an increased weight of the shovel during lifting, the increased weight indicating that the object is in the shovel

Determining whether an object is in a shovel using a classification model

Rechecking that the object is in the shovel using force feedback, added weight, and/or a dedicated camera while the robot is in motion

If the pick policy fails, the target object(s) may be marked as pick failed at block 2120. If the target object(s) were successfully picked up, the method includes navigating to a drop position at block 2122. For example, a robot 100 such as that introduced with reference to fig. 1A may navigate to a predetermined drop position. The drop location may be a designated area of the container, or the ground or floor. Navigation may be controlled by a machine learning model or a rule-based approach.

According to some examples, the method includes determining a fall strategy at block 2124. For example, the robotic control system 1900 illustrated in fig. 1A may determine a drop strategy. The drop strategy may need to take into account the bearer location determined for the pick-up strategy. The fall strategy may be determined using a machine learning model or a rule-based approach. The rule-based policy for object drop may include:

navigation of the robot to a position 100 cm from the side of the bin

Rotate the robot into position so that it is aligned facing the bin

Driven toward the cartridge maintaining alignment centered on the side of the cartridge

Stop three cm from the side of the bin

Verifying that the robot is correctly positioned with respect to the side of the bin

If so, lift the shovel upward and rearward to drop the target object(s) into the bin

O if not, drive away from bin and restart the process

The object drop strategy may involve navigating with a rear camera when a post-drop is attempted or with a front camera when a forward drop is attempted.

According to some examples, the method includes executing a fall policy at block 2126. For example, a drop strategy may be performed such as the robot 100 introduced with reference to fig. 1A. The drop policy may follow policy step 2200 for the quarantine policy, pick policy, and drop policy illustrated in fig. 22. The drop policy may be a reinforcement learning based policy or a rule based policy. Once the drop policy has been executed at block 2126, the method may continue with determining the next target object(s) at block 2128. For example, the robotic control system 1900 illustrated in fig. 1A may determine the next target object(s). Once the new target object(s) have been determined, the process may be repeated for the new target object(s).

Strategies such as the quarantine strategy, pick-up strategy, and drop strategy mentioned above may be simple strategies or may contain rewards and collision avoidance elements. These policies may follow general methods such as the quarantine policy, pick policy, and drop policy 2200 illustrated in fig. 22.

In some embodiments, the object isolation policy may include:

surrounding the object(s) and casting back using on-floor V-shaped gripper pad arms and gripper pads

Accurately gripping the object(s) and casting back with a V-shaped gripper pad arm and gripper pad

Loosely roll large objects away with raised gripper pad arms and gripper pads

De-compacting dense debris by loosely grabbing the stack and throwing back

Placing a single gripper pad arm/gripper pad on the floor between the object(s) and the grapple, then turning

Put the toys into a shovel and then drop them to separate them

Moving object(s) away from wall using single gripper pad arm/gripper pad

In some embodiments, the pick-up policy may include:

closing the gripper pad arm/gripper pad on the floor to pick up simple objects

Picking up stacks of small objects (similar to small plastic building blocks) by closing the gripper pad arms/gripper pads on the ground

Pick up small, scrollable objects (like balls) by tapping them with the gripper pad arm/gripper pad, thereby rolling them into the shovel

Picking up deformable object (similar to clothing) using gripper pad arm/gripper pad to repeatedly compress object(s) into shovel

Grasping these objects by grasping and compressing oversized, soft objects (similar to large filled animals) with the gripper pad arms/gripper pads

Gripping the ball by rolling the ball and holding the ball against the shovel with an elevated gripper pad arm/gripper pad

Picking up a flat object by passing the gripper pads sideways through the flat object (similar to a jigsaw), resulting in instability

Holding books and other large flat objects

Pick up the clothes with the gripper pad arm/gripper pad, lift them above the scoop, then drop them into the scoop

Rolling the ball by starting the first gripper pad arm movement and immediately starting the second gripper pad arm movement

In some embodiments, the drop policy may include:

Fall back into the bin

Drop forward into the bin

Forward release onto floor

Forward release against wall

Stacking books or other flat objects

Directly dropping large objects using gripper pad arms/gripper pads instead of relying on a shovel

Fig. 22 illustrates policy steps 2200 for quarantine policy, pick policy, and drop policy, according to one embodiment. According to some examples, the method includes determining action(s) according to a policy at block 2202. For example, the robotic control system 1900 illustrated in fig. 1A may determine action(s) according to a policy. The next action(s) may be based on the policy and observations, the current robot state, the current object state, and the sensor data 2016. The determination may be made by process 2300 illustrated in fig. 23 for determining an action according to a policy.

In one embodiment, the policy may contain rewards or penalties 2212 when determining the action(s) according to the policy at block 2202. These rewards or penalties 2212 may be primarily used to train the reinforcement learning model and, in some embodiments, may not be applicable to ongoing operation of the robot. Training of the reinforcement learning model may be performed using simulation or by recording model inputs/outputs/rewards/penalties during robot operation. The recorded data may be used to train a reinforcement learning model to select actions that maximize rewards and minimize penalties. In some embodiments, rewards or penalties 2212 for object pickup using reinforcement learning may include:

Small penalty added per second

Rewards when target object(s) first contact edge of shovel

Rewards when target object(s) are pushed completely into the shovel

Penalty when target object(s) are lost from the shovel

Penalty for collisions with obstacles or walls (exceeding the force feedback maximum)

Penalty for picking up non-target objects

Penalty when robot gets stuck or driven beyond object

In some embodiments, rewards or penalties 2212 for object isolation (e.g., moving the target object(s) to the right away from the wall) using reinforcement learning may include:

Small penalty added per second

Rewards when the right gripper pad arm is between the target object(s) and the wall

Rewards when the target object(s) is more than ten cm from the wall

Penalty for incorrect collision with target object(s)

Penalty in case the robot gets stuck or driven beyond the object

In some embodiments, rewards or penalties 2212 for object drop using reinforcement learning may include:

Small penalty added per second

Rewards when robot correctly stops in bin

Rewards when target object(s) successfully drop into bin

Penalty for collision of moving bins

Penalty when robot gets stuck or driven beyond object

In at least one embodiment, the techniques described herein may use a reinforcement learning approach, where the problem is modeled as a Markov Decision Process (MDP), which is represented as tuples (S, O, a, P, r, γ), where S is a set of states in the environment, O is an observation set, a is an action set,Is a function of the probability of a state transition,Is the reward function and γ is the discount factor.

In at least one embodiment, the goal of training may be to learn a deterministic strategy pi: o→a, such that taking action at time t, a _t＝π(o_t) maximizes the sum of discounted future rewards from state s _t:

In at least one embodiment, after taking action a _t, the environment transitions from state s _t to state s _t+1 by sampling from P. In at least one embodiment, the quality of taking action a _t in state s _t is determined by The measurement is called the Q function.

In one embodiment, the data from the mobile collision avoidance system 2214 may be used to determine action(s) according to a policy at block 2202. Each policy may have an associated list of available actions that it may consider. A policy may use a mobile collision avoidance system to determine the range of motion for each action involved in executing the policy. For example, a mobile collision avoidance system may be used to see if a shovel may be lowered to the ground without striking the gripper pad arms or gripper pads (if they are closed under the shovel), an obstacle (such as a nearby wall), or an object (similar to a ball) that may roll under the shovel.

According to some examples, the method includes performing an action(s) at block 2204. For example, a robot 100, such as introduced with reference to fig. 1A, may perform the action(s) determined from block 2202. The actions may be based on the observations, the current robot state, the current object state, and the sensor data 2016. The actions may be performed by movements of the robot motors and other actuators 2210 of the robot 100. The real world environment 2002 may be affected by the motion of the robot 100. Changes in the environment 2002 may be detected as described with respect to fig. 20.

According to some examples, the method includes checking for progress toward the target at block 2206. For example, the robotic control system 1900 illustrated in fig. 1A may examine the progress of the robot 100 toward a target. If the advance check determines that the goals of the policy have been met, or that a catastrophic error has been encountered at decision block 2208, execution of the policy will be stopped. If the objective is not met and no catastrophic error has occurred, the policy may return to block 2202.

FIG. 23 illustrates a process 2300 for determining an action according to a policy, according to one embodiment. The process 2300 for determining actions from policies may consider the policy type 2302 and may determine available actions to use based on the policy type 2302 at block 2304. Reinforcement learning algorithms or rule-based algorithms may utilize simple actions and predefined compound actions. Examples of controlling simple actions of a single actuator may include:

Move the left gripper pad arm to the new position (rotate up or down)

Move left gripper pad wrist to new position (rotate left or right)

Move the right gripper pad arm to the new position (rotate up or down)

Move the right gripper pad wrist to the new position (rotate left or right)

Lifting the blade to a new position (rotating up or down)

Changing the angle of the shovel (with a second motor or actuator for front drop)

Drive left wheel

Drive right wheel

Examples of predefined composite actions may include:

Driving a robot to a position/waypoint following a path

Rotate the robot left or right into position

Centering the robot with respect to the object(s)

Alignment of gripper pad arm with top/bottom/middle of object

Drive forward until the object abuts the edge of the blade

Closing both gripper pad arms, pushing the object(s) with a smooth motion

When gripping the object(s), lift the shovel and gripper pad arms together

Closing both gripper pad arms, pushing the object(s) with a quick tap and a slight release

Place the spade gently against the floor/carpet

Pushing the shovel down against the floor/pushing the shovel down onto the carpet

Closing the gripper pad arm until resistance/applied pressure is encountered and this position is maintained

Closing the gripper pad arm by vibration and left/right rotation to create a slight bounce of the unstable and flat object on the edge of the blade

At block 2308, process 2300 for determining actions according to policies may employ list 2306 of available actions determined at block 2304, and may determine a range of motion 2312 for each action. The range of motion 2312 may be determined based on the observations, the current robot state, the current object state, and sensor data 2016 available to the robot control system. The type of action 2310 may also be indicated to the mobile collision avoidance system 2214, and the mobile collision avoidance system 2214 may determine the range of motion 2312.

Block 2308 of process 2300 for determining actions according to policies may determine an observation list 2314 based on the determined range of motion 2312. Example observations list 2314 may include:

Objects detected and classified in the environment

Global or local environment map

State 1: the left arm position is turned 20 degrees

State 2: the right arm position is turned 150 degrees

State 3: the target object is 15 cm from the edge of the shovel

State 4: the center of the target object is deviated to the right 5 degrees

Action 1 maximum range: forward drive of up to 1 cm

Action 2 maximum range: back driving a maximum of 10cm

Action 3 maximum range: open the left arm by a maximum of 70 degrees

Action 4 maximum range: open the right arm by 90 degrees at maximum

Action 5 maximum range: the maximum of 45 degrees of the closed left arm

Action 6 maximum range: closed right arm max 0 degree

Action 7 maximum range: left turn is maximally 45 degrees

Action 8 maximum range: right turn of 45 degrees maximum

At block 2316, a reinforcement learning model may be run based on the observation list 2314. The reinforcement learning model may return action(s) 2318 appropriate to the strategy that the robot 100 is attempting to accomplish based on the strategy involved.

Fig. 24 illustrates a deposit process 2400 in accordance with an embodiment. The deposit process 2400 may be performed by a robot 100, such as introduced with respect to fig. 1A, as part of the algorithms disclosed herein. The robot may have the sensing system, control system, mobility system, gripper pad arm and shovel illustrated in fig. 1A-1D, or similar systems and features that perform equivalent functions well known in the art.

In block 2402, the robot may detect a destination where an object carried by the robot is to be deposited. In block 2404, the robot may determine a destination access path to the destination. The path may be determined to avoid an obstacle near the destination. In some embodiments, the robot may perform additional navigation steps to push the object out of the destination access path and away from the destination access path. The robot may also determine an object storage mode, wherein the object storage mode is one of a placement mode and a drop mode. Some neatly stackable objects (such as books, other media, narrow boxes, etc.) may be cleaned most neatly by careful stacking. Other objects may not be neatly stacked, but may be easily stored by falling into a bin. Based on the object properties, the robot can determine which object deposit mode is best suited for the object.

In block 2406, the robot may approach the destination via a destination access path. How the robot navigates the destination access path may be determined based on the object deposit pattern. If the carried object is to fall from the back of the chassis of the robot, the robot may traverse the destination access path in the reverse direction, stopping with the back of the chassis nearest the destination. Alternatively, for objects to be stacked or placed at the front of the shovel (i.e., at the region of the shovel opposite the chassis), the robot may travel forward along the destination access path so as to bring the shovel closest to the destination.

At decision block 2408, the robot may proceed in one of at least two ways, depending on whether the object is to be placed or dropped. If the object storage mode is intended as the placement mode, the robot may proceed to block 2410. If the object storage mode is intended as the drop mode, the robot may proceed to block 2416.

For objects to be placed via the placement mode, the robot may stop at block 2410, with the destination in front of the shovel and gripper pad. In block 2412, the robot may lower the shovel and gripper pad to a storage height. For example, if the book is stored on an existing stack, the storage height may be slightly higher than the top of the highest book in the stack, so that the book may be placed without damaging the stack or dropping the book from a height that makes it possible for the book to have sufficient momentum to slide out of the stack or destabilize the stack. Finally, at block 2414, the robot may push the object out of the containment area and onto the destination using its gripper pads. In one embodiment, the shovel may be tilted forward to drop the subject with or without the aid of a gripper pad to push the subject out of the shovel.

If in decision block 2408 the robot determines that it is to proceed in the object storage mode as the drop mode, the robot may proceed to block 2416. At block 2416, the robot may stop with the destination at the rear of the blade and gripper pads, and by virtue of this at the rear of the chassis of the robot (such as the robot illustrated initially in fig. 1A). In block 2418, the robot may raise the shovel and gripper pad to a storage height. In one embodiment, the object may be positioned such that raising the shovel and gripper pad arm from the load-bearing position to the storage height causes the object to drop from the containment region to the destination region. Otherwise, in block 2420, the robot may extend the gripper pad and allow the object to fall from the containment area such that the object rests at or in the destination area. In one embodiment, the shovel may be tilted forward to drop the subject with or without the aid of a gripper pad to push the subject out of the shovel.

The disclosed algorithm may include a capture process 2500 as illustrated in fig. 25. The capture process 2500 may be performed by a robot 100, such as introduced with respect to fig. 1A. The robot may have the sensing system, control system, mobility system, gripper pad arm and shovel illustrated in fig. 1A-1D, or similar systems and features that perform equivalent functions well known in the art.

The capture process 2500 may begin in block 2502, where a robot detects a starting position and properties of an object to be lifted. The starting position may be determined relative to a learned map of landmarks in the room that the robot is programmed to clear. Such graphs may be stored in a memory within the electrical system of the robot. These systems will be described in more detail with reference to fig. 19. The object properties may be detected based on input from a sensing system, which may include a camera, LIDAR, or other sensor. In some embodiments, data detected by such sensors may be compared to a database of common objects to determine properties such as deformability and size. In some embodiments, the robot may calculate object properties such as dimensions using known landmark properties. In some embodiments, machine learning may be used to improve attribute detection and analysis.

In block 2504, the robot may determine a proximity path to a starting position. The approach path may take into account the geometry of the surrounding space, obstacles detected around the object, and how the components of the robot may be configured when the robot approaches the object. The robot may further determine a grip height for initial contact with the object. The gripping height may take into account an estimated center of gravity for the object so that the gripper pad moves the object with the lowest likelihood that: causing it to slide down, down or around the object, or causing the object to deflect in some direction other than into the spade. The robot may determine a gripping pattern for movement of the gripper pads during object capture such that the object may be contacted from a certain direction and with a force applied at intervals that are optimized to guide and push the object into the shovel. Finally, the robot may determine the load bearing position of the gripper pads and the shovel, which secures the object in the containment area for transport after the object is captured. This location may take into account properties such as the size of the object, the weight of the object, and the center of gravity of the object.

In block 2506, the robot may extend its gripper pad outwardly and forwardly with respect to the gripper pad arm and raise the gripper pad to a gripping height. This may allow the robot to be as close to the object as possible without having to make room for the extension after access. Alternatively, the robot may perform some portion of the approach with the arm folded close to the chassis and the shovel to prevent the obstacle from being impacted along the approach path. In some embodiments, the robot may first navigate on the approach path and deploy the arm and the shovel to clear the object out of and away from the approach path. In block 2508, the robot may eventually approach the object via the approach path, stopping when the object is positioned between the gripper pads.

In block 2510, the robot may perform the grab mode determined in block 2502 to capture objects within the containment region. The receiving area may be an area roughly described by the size of the blade and the placement of the gripper pad arm relative to the blade. It is understood that the area in which the object to be transported may reside during transport with minimal likelihood of being transferred or dislodged or dropped from the shovel and gripper pad arms. In decision block 2512, the robot may confirm that the object is within the containment zone. If the object is within the containment area, the robot may proceed to block 2514.

In block 2514, the robot may apply light pressure on the object with the gripper pad to hold the object stationary in the containment area. In some embodiments, the pressure may be downward to hold objects protruding above the top of the shovel down against the sides and surface of the shovel. In other embodiments, the pressure may be applied horizontally to hold the object within the shovel against the back of the shovel. In some embodiments, pressure may be against the bottom of the shovel to prevent a gap from forming, which may allow the subject to slide out of the front of the shovel.

In block 2516, the robot may raise the shovel and gripper pad to the load-bearing position determined in block 2502. Then, at block 2518, the robot may carry the object to the destination. The robot may follow a transition path between the starting position and the destination where the object is to be deposited. To deposit the object at the destination, the robot may follow a depositing process 2400 illustrated in fig. 24.

At decision block 2512, if an object is not detected within the containment area, or it is determined that the object is partially or unstably located within the containment area, at block 2520 the robot may extend the gripper pad from the shovel 2946 and forward relative to the gripper pad arm, and return the gripper pad to the gripping height. The robot may then return to block 2510. In some embodiments, if simply releasing and retrying to capture the object is not feasible, the robot may back away from the object at block 2522. This may occur if the object has been repositioned or moved when the object was initially attempted to be captured. In block 2524, the robot may re-determine the approach path to the object. The robot may then return to block 2508.

Fig. 26 illustrates the beginning of a process diagram 2600 according to one embodiment of the deposit process 2400 and the capture process 2500 described above. In step 2602, the robot may drive to a target object or group of objects. The robot may use the local map or global map to navigate to the vicinity of the target object or group of objects.

In step 2604, the robot may adjust the approach angle and move the obstacle. The optimal approach angle may be determined for a particular object, such as approaching the book from the direction of the spine. The gripper pad arm may be used to push the obstacle away and the robot may be driven to adjust its approach angle.

In step 2606, the robot may adjust its arm height based on the type of object or group of objects. The strategy for picking up a target object or group of objects includes arm height and may vary depending on the object to be picked up. For example, a basketball may be pushed from its top so that it rolls. The stuffed animal can be pushed from its middle so that it will slip and not fall sideways and will not become more difficult to push or to flap over the gripper pad arm. The book can be pushed from its side to a position very close to the floor. The music toy may be pushed against the floor with an arm.

At step 2608, the robot may drive such that the arms are aligned through the object or group of objects. The object or group of objects may be in contact with the shovel or bucket and may be located in an area inside the two gripper pad arms.

At step 2610, the robot may use its arm to push the object or group of objects onto the shovel or bucket. The arm may be used intelligently in accordance with the grasping mode as previously described. In some cases, both arms may be gradually pushed together, but may adjust as the subject rolls and moves.

At step 2612, the robot may determine whether the object is picked up. A camera sensor or other sensor may be used to see or detect whether an object or group of objects has been successfully pushed onto or into the shovel or bucket. If the object is not picked up, the robot may release the arm upward and open in step 2614. The arm may first be released slightly so that the object is not squeezed and then moved over and over the object or group of objects so that the object or group of objects is not pushed farther from or farther from the blade or bucket. This allows the robot to advance step by step through pick up if the initial action is insufficient. From there, the robot may return to step 2604. At step 2612, if the object is detected within the shovel, the robot may proceed to step 2616.

In step 2616, the robot may apply light pressure on top of the object or against a shovel or bucket. Based on the object type, the robot may thereby apply pressure to the object in order to hold the object or group of objects within the shovel or bucket. For example, the robot may hold the top of a basketball firmly, squeeze the stuffed animal, push down on a book, or push against a shovel or bucket to hold a small set of objects (such as marble or plastic building blocks).

At step 2618, the robot may lift the object or group of objects while continuing to hold with the gripper pad arm. The blade or bucket and arm may be lifted together to a predetermined angle (such as forty-five degrees) to carry an object or group of objects without rolling out or moving out of the blade or bucket. When the blade and arm are raised to a desired angle, the arm may continue to apply pressure to hold the subject in the blade.

At step 2620, the robot may drive to a destination location to place the object. The robot may use the local or global map to navigate to the destination location in order to place an object or group of objects. For example, this may be a container intended to hold objects, a stack of books, or a designated floor section where an object or group of objects may not be on a route.

At step 2622, the robot may move the shovel or bucket holding the object or group of objects upward and over the destination. The height or position of the shovel may be adjusted to align with the destination location. For example, the scoop or bucket may be lifted over the container or aligned with an area over the top of an existing book stack.

At step 2624, the robot may place the object or group of objects at the destination location using its gripper pad arm. The arm may be opened to drop the object or group of objects into the container, or may be used to push the object forward out of the shovel or bucket. For example, basketball may be dropped into a container over the back of the robot and books may be carefully pushed forward onto the stack of existing books. Finally, the process ends at step 2626, where the object is successfully dropped or placed at the destination.

Fig. 27A-27D illustrate a process 2700 for a stackable object, according to one embodiment. Fig. 27A shows a side view of a robot performing steps 2702-2710, while fig. 27B shows a top view of the performance of these same steps. Fig. 27C illustrates a side view of steps 2712-2720, and fig. 27D illustrates a top view of these steps. The stackable object may be a book, a housing containing a Compact Disc (CD), digital Video Disc (DVD) or other medium, a narrow box such as a jigsaw box, or some other object that can be easily and neatly stacked.

As illustrated in fig. 27A and 27B, the robot may first drive to a stackable object 2722 located at a starting position 2724, as shown at step 2702. The robot may drive to the stackable object 2722 following the approach path 2726. As shown in steps 2704 and 2712, the robot may adjust its gripper pad arm to a gripping height 2728 based on the type of object. For stackable objects like books, this may be just above the top of the book. At step 2706 and step 2706, the robot may drive such that its arm is aligned with the object 2730. At step 2708 and step 2708, the robot may employ a grasping mode 2732, using its arm to push the book onto the shovel or bucket. At steps 2710 and 2710, using the gripper pad arm, the robot may apply light pressure 2734 to the top of the book to hold the book securely within the spade or on top.

As shown in fig. 27C and 27D, at step 2712, the robot may lift the book while continuing to hold the book with its gripper pad arm, thereby maintaining the book in the in-shovel load position 2736. At step 2714, the robot may drive following a destination access path 2740 to a destination 2738 where the book is intended to be placed. At step 2716, the robot may adjust the shovel and gripper pad arms to position the book at storage height 2742. For stackable objects such as books, this may position the book on the level of the area above the top of the existing stack. At step 2718, the robot may push the book using its arm and place the book at the destination (i.e., at the top of the stack) using a placement mode 2744. At step 2720, the book may drop or be stored at its destination in this manner.

This process 2700 for stackable objects may be performed by any of the robots disclosed herein, such as the robots illustrated in fig. 1A-1D, fig. 2A-2E, fig. 3A-3C, fig. 4A-4C, fig. 5, fig. 6, fig. 7, fig. 8, fig. 9, fig. 10, fig. 11, fig. 12, fig. 13, fig. 14, fig. 15, fig. 16 and fig. 17, and fig. 18A and fig. 18B.

Fig. 28A-28D illustrate a process 2800 for a large, slightly deformable object, according to one embodiment. Fig. 28A shows a side view of a robot performing steps 2802-2810, while fig. 28B shows a top view of the performance of these same steps. Fig. 28C illustrates a side view of steps 2812-2820, and fig. 28D illustrates a top view of these steps. The large, slightly deformable object may be an object such as a basketball that protrudes beyond the size of the shovel and may respond to pressure with very little deformation or shape change.

As illustrated in fig. 28A and 28B, at step 2802, the robot may first drive to a large, slightly deformable object 2822 (such as a basketball) located at a starting position 2824 following a proximity path 2826. At step 2804, the robot may adjust its gripper pad arm to a gripping height 2828 based on the type of object. For a large, slightly deformable object 2822 such as a basketball, this may be near or above the top of the basketball. At step 2806, the robot may drive such that its arm is aligned past the object 2830. At step 2808, the robot may employ a grip mode 2732 to push or roll the basketball onto a shovel or bucket using its arm. At step 2810, using the gripper pad arm, the robot may apply light pressure 2834 to the top of the basketball to securely hold the basketball within the shovel or on top.

As shown in fig. 28C and 28D, at step 2812 the robot may lift the basketball while continuing to hold the basketball with its gripper pad arm, thereby maintaining the ball in a carrying position 2836 within the shovel. Next, at step 2814, the robot may drive to a destination 2838 where the basketball is intended to be placed following a destination access path 2840. At step 2816, the robot may adjust the shovel and grabber pad arms to position the basketball at a storage height 2842. For objects such as basketball, this may position the blade and ball in an area above the robot, tilted or aimed toward the container. At step 2818, the robot may open its arm to release the object into the destination container using the drop mode 2844. Then, at step 2820, the basketball may be dropped from the shovel 2846 and left in its destination container.

The process 2800 for a large, slightly deformable object may be performed by any of the robots disclosed herein, such as the robots illustrated in fig. 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6,7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17, and 18A and 18B.

Fig. 29A-29D illustrate a process 2900 for a large, highly deformable object, according to one embodiment. Fig. 29A shows a side view of a robot performing steps 2902-2910, while fig. 29B shows a top view of the performance of these same steps. Fig. 29C illustrates a side view of steps 2912-2920, and fig. 29D illustrates a top view of these steps. The large, highly deformable object may be an object that protrudes beyond the dimensions of the spatula (such as a stuffed animal, a bean bag toy, an empty backpack, etc.), and may respond to pressure with significant deformation or shape changes.

As illustrated in fig. 29A and 29B, at step 2902, the robot may first follow the approach path 2926 to a large, highly deformable object 2922 (such as a stuffed animal) located at the starting position 2924. At step 2904, the robot may adjust its gripper pad arm to a gripping height 2928 based on the type of object. For large, highly deformable objects (such as filled animals), this may be near the vertical center of the object, or even at the estimated center of gravity. At step 2906, the robot may drive such that its arms are aligned past the object 2930. At step 2908, the robot may employ a grasping mode 2932 to push the stuffed animal onto the shovel or bucket using its arm. At step 2910, using the gripper pad arm, the robot may apply light pressure 2934 to the top of the stuffed animal to securely hold the stuffed animal within or on top of the shovel.

As shown in fig. 29C and 29D, at step 2912, the robot may lift the stuffed animal while continuing to hold the stuffed animal with its gripper arms, thereby maintaining the stuffed animal in a carrying position 2936 within the shovel. Next, at step 2914, the robot may drive following a destination access path 2940 to a destination 2938 where the filling animal is intended to be placed. At step 2916, the robot may adjust the shovel and gripper pad arms to position the stuffed animal at storage height 2942. For objects such as stuffed animals, this may position the spatula and stuffed animal in an area above the robot, tilted or aimed toward the container. At step 2918, the robot may open its arm to release the object into the destination container using the drop mode 2944. The stuffed animal may then roll out of the shovel 2946, slide out of the shovel 2946, or fall out of the shovel 2946 and remain in its destination container at step 2920.

The process 2900 for a large, highly deformable object may be performed by any of the robots disclosed herein, such as the robots illustrated in fig. 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6,7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17, and 18A and 18B.

Fig. 30A-30D illustrate a process 3000 for a small, easily scattered object, according to one embodiment. Fig. 30A shows a side view of a robot performing steps 3002-3010, while fig. 30B shows a top view of the performance of these same steps. Fig. 30C illustrates a side view of steps 3012-3020, and fig. 30D illustrates a top view of these steps. Small, easily scattered objects may be small and light objects (such as small plastic building blocks, marbles, grains, etc.), which may easily scatter when in contact with the gripper pad arm of the robot, or may slip out of the shovel during transport without taking proper care.

As illustrated in fig. 30A and 30B, at step 3002, the robot may first be driven to a small, easily scattered object 3022 (such as a set of plastic building blocks) located at a starting position 3021 following a proximity path 3026. At step 3004, the robot may adjust its gripper pad arm to a gripping height 3028 based on the type of object. For small, easily scattered objects, this may be near or in contact with the floor. At step 3006, the robot may drive such that its arms are aligned past object 3030. At step 3008, the robot may employ a grasping mode 3032 to push an object onto a shovel or bucket using its arm. The grabbing pattern 3032 for such objects may apply less force or use less sweeping motion rather than continuous pressure. At step 3010, the robot may close its arm 3034 across the front of the shovel and may apply slight pressure to the shovel to prevent the subject from rolling or sliding out.

As shown in fig. 30C and 30D, at step 3012, the robot may lift the building block while continuing to block the blade front opening with its gripper pad arm, thereby maintaining the object within the blade in the load-bearing position 3036 within the blade. Next, at step 3014, the robot may drive to a destination 3038 where the object is intended to be placed following a destination access path 3040. At step 3016, the robot may adjust the shovel and gripper pad arms to position the subject at storage height 3042. For objects such as small plastic building blocks, this may position the shovel in an area above the robot, tilted or aimed towards the container. At step 3018, the robot may open its arm to release the objects captured by them into the destination container using the drop mode 3044. Then, at step 3020, the pieces may roll out of scoop 3046, slide out of scoop 3046, or fall out of scoop 3046 and remain in their destination container.

The process 3000 for small, easily scattered objects may be performed by any of the robots disclosed herein, such as the robots illustrated in fig. 1A-1D, 2A-2E, 3A-3C, 4A-4C, 5, 6,7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17, and 18A and 18B.

Fig. 31 depicts a robotic system 3100 in one embodiment. The robotic system 3100 receives inputs from one or more sensors 3102 and one or more cameras 3104 and provides these inputs for processing by positioning logic 3106, mapping logic 3108, and sensing logic 3110. The output of the processing logic is provided to a robotic system 3100 path planner 3112, a pick-up planner 3114 and a motion controller 3116, which in turn drives motors and servo controllers 3118 of the system.

The cameras may be disposed in a forward stereoscopic arrangement, and may also include one or more rearward cameras. Alternatively, a single forward-facing camera may be utilized, or a single forward-facing camera may be utilized with a single backward-facing camera. In some cases, other camera arrangements (e.g., one or more side or diagonal cameras) may also be utilized.

One or more of positioning logic 3106, mapping logic 3108, and sensing logic 3110 may be positioned and/or executed on a mobile robot, or may be executed in a computing device in wireless communication with the robot, such as a cell phone, laptop, tablet, or desktop. In some embodiments, one or more of positioning logic 3106, mapping logic 3108, and sensing logic 3110 may be positioned and/or executed in the "cloud" (i.e., on a computer system coupled to the robot via the internet or other network).

The perception logic 3110 is engaged by the image segmentation activation 3144 signal and detects objects in the field of view of the camera 3104 using any one or more of well-known image segmentation and object recognition algorithms. The sense logic 3110 may also provide calibration and object 3120 signals for mapping purposes. Positioning logic 3106 uses any one or more of well-known algorithms to position the mobile robot in its environment. The positioning logic 3106 outputs a local-to-global transform 3122 reference frame transform and the mapping logic 3108 combines this with the calibration and object 3120 signals to generate an environment map 3124 for the pick-up planner 3114 and an object tracking 3126 signal for the path planner 3112.

In addition to the object tracking 3126 signal from mapping logic 3108, path planner 3112 also utilizes the current state 3128 of the system from system state settings 3130, synchronization signals 3132 from pick-up planner 3114, and movement feedback 3134 from motion controller 3116. The path planner 3112 transforms these inputs into navigational waypoints 3136 that drive the motion controller 3116. Pick-up planner 3114 transforms local perception with image segmentation 3138 input from perception logic 3110, 3124 input from mapping logic 3108, and synchronization signal 3132 from path planner 3112 into steering actions 3140 (e.g., of a robotic grip, shovel) to motion controller 3116. An embodiment of the algorithm utilized by the path planner 3112 and pick planner 3114 is described in more detail below.

In one embodiment, a simultaneous localization and mapping (SLAM) algorithm may be used to generate a global map and simultaneously localize the robot on the map. Many SLAM algorithms are known in the art and commercially available.

The motion controller 3116 converts the navigation waypoint 3136, steering action 3140 and local perception 3138 signals with image segmentation into target movement 3142 signals to the motor and servo controller 3118.

Fig. 32 depicts a robotic process 3200 in one embodiment. In block 3202, robotic process 3200 wakes up the sleeping robot at the base station. In block 3204, robotic process 3200 uses cameras to navigate the robot around the environment of the robot to map the type, size, and location of toys, clothing, obstacles, and other objects. In block 3206, robotic process 3200 operates a neural network to determine a type, size, and position of an object based on images from left/right stereo cameras. In an open loop block 3208, robotic process 3200 is performed for each class of objects having a corresponding container. In block 3210, the robotic process 3200 selects a particular object in the category to pick up. In block 3212, the robotic process 3200 performs path planning. In block 3214, the robotic process 3200 navigates near and towards the target object. In block 3216, robotic process 3200 actuates the arms to move the other objects away and push the target object onto the front blade. In block 3218, robotic process 3200 tilts the front blades upward to retain them on the blade (creating a "bowl" configuration of blades). In block 3220, the robotic process 3200 actuates the arms to close at the front to prevent the object from under the wheel when the robot navigates to the next position. In block 3222, the robotic process 3200 performs path planning and navigates near the container for the current object class for collection. In block 3224, the robotic process 3200 aligns the robot with a side of the container. In block 3226, robotic process 3200 lifts the blade up and back to lift the target object up and over the sides of the container. In block 3228, the robotic process 3200 returns the robot to the base station.

In a less complex mode of operation, the robot may opportunistically pick up objects in its field of view and drop them into the container without first creating a global map of the environment. For example, the robot may simply explore until it finds an object to pick up, and then explore again until it finds a matching container. This approach works effectively in a single-space environment where the area to be explored is limited.

Fig. 33 also depicts a robotic process 3300 in one embodiment, wherein the robotic system sequentially passes through an embodiment of a state space diagram 3400 as depicted in fig. 34.

The sequence begins with the robot sleeping (sleep state 3402) and charging at the base station (block 3302). The robot is activated, for example, according to a schedule, and enters a search mode (environment search state 3404, activation action 3406, and schedule start time 3408). In the environment exploration state 3404, the robot scans the environment using cameras (and other sensors) to update its environment map and locate its own position on the map (block 3304, exploration is done for configured interval 3410). The robot may transition from the environment exploration state 3404 back to the sleep state 3402 without more objects 3412 to be picked up or with a low battery 3414.

From the environment exploration state 3404, the robot may transition to the object organization state 3416, where the robot operates to move items on the floor to organize them by category 3418. The transition may be triggered by the robot determining that the object is too close on the floor 3420 or that the path to one or more objects is blocked 3422. If none of these trigger conditions are met, the robot may transition directly from the environment exploration state 3404 to the object pick-up state 3424 if the environment map includes at least one drop container 3426 for the category of objects and there is an unobstructed item 3428 for pick-up in the category of containers. Likewise, the robot may transition from the object tissue state 3416 to the object pick-up state 3424 under these latter conditions. Without the object ready to pick 3430, the robot may transition from the object organization state 3416 back to the environment exploration state 3404.

In the environment exploration state 3404 and/or the object organization state 3416, image data from the camera is processed to identify different objects (block 3306). The robot selects a particular object type/category to pick up, determines the next waypoint to navigate to, and determines the location of the target object and type to pick up based on the environmental map (blocks 3308, 3310, and 3312).

In the object pick state 3424, the robot selects a target location adjacent to the target object(s) (block 3314). It uses a path planning algorithm to navigate itself to a new location while avoiding obstacles. The robot actuates the left push arm and the right push arm to create an opening large enough that the target object can pass through, but not so large that other unwanted objects are collected as the robot is driven forward (block 3316). The robot drives forward such that the target object is between the left push arm and the right push arm, and the left push arm works with the right push arm to push the target object onto the collection shovel (block 3318).

The robot may continue in object pick state 3424 to identify other target objects of the selected type to pick based on the environmental map. If other such objects are detected, the robot selects a new target location adjacent to the target object. It uses a path planning algorithm to navigate itself to a new location while avoiding obstacles while carrying the previously collected target object(s). The robot actuates the left and right push arms to create an opening large enough that the target object can pass through, but not so large that other unwanted objects are collected as the robot is driven forward. The robot is driven forward with the next target object(s) between the left push arm and the right push arm. Likewise, the left push arm and the right push arm work together to push the target object onto the collection shovel.

With all identified objects in the category picked up 3432, or if the shovel reached the upper capacity limit 3434, the robot transitions to an object drop state 3436 and uses the environmental map to select a target location adjacent to the bin for the type of object collected and uses the path planning algorithm to navigate itself to the new location while avoiding the obstacle (block 3320). The robot is retracted toward the bin to a rest position where the rear of the robot is aligned with the back of the bin (block 3322). The robot rotationally lifts the blade (block 3324) both above and rearward of the rigid arm at the rear of the robot. This lifts the target object over the top of the bin and dumps it into the bin.

In the event that there are more items to pick 3438 or it has an incomplete environmental map 3440, the robot may transition from the object drop state 3436 back to the environmental exploration state 3404. The robot resumes exploration and the process may repeat for each other type of object in the environment that has an associated collection bin (block 3326).

In the event that there are no more subjects 3412 to pick up, or the battery power is low 3414, the robot may instead transition from the subject drop state 3436 back to the sleep state 3402. Once the battery is fully recharged, or at the next active or predetermined pick interval, the robot resumes exploration and the process may be repeated for every other type of object in the environment that has an associated collection bin (block 3326).

Fig. 35 depicts a robot control algorithm 3500 for a robotic system in one embodiment. Robot control algorithm 3500 begins by selecting one or more categories of objects for organization (block 3502). Within the selected one or more categories, a set of target categories and starting locations for the determined path is identified (block 3504). Any of a variety of well-known clustering algorithms may be utilized to identify groupings of objects within one or more categories.

A path is formed to the starting target location that includes zero or more waypoints (block 3506). The movement feedback is provided back to the path planning algorithm. Waypoints may be selected to avoid static and/or dynamic (moving) obstructions (objects not in the target group and/or class). The movement controller of the robot is engaged to follow the waypoint to the target group (block 3508). The target group is evaluated as it is being implemented, including additional qualifications for determining whether it can be safely organized (block 3510).

The perception system of the robot is engaged (block 3512) to provide image segmentation for determining an activation sequence for the tissue group generated for the manipulator (e.g., arm) and positioning system (e.g., wheel) of the robot (block 3514). The ordering of the activations is repeated until the target group is organized, or the organization fails (failure results in regression to block 3510). Engagement of the sensing system may be triggered by proximity to the target set. Once the target group is organized, and provided there is sufficient battery life left for the robot and there are more groups to organize in one or more categories, these actions are repeated (block 3516).

In response to the low battery life, the robot navigates back to the docking station to charge (block 3518). However, if there is sufficient battery life, and under the condition that one or more categories are organized, the robot enters an object pick-up mode (block 3520) and picks up one of the organized groups for return to the drop container. Entering the pick-up mode may also be conditioned on an environmental map comprising at least one drop container for the target object, and the presence of unobstructed objects in the target group for pick-up. In the absence of a ready pick-up of the object group, the robot continues to explore the environment (block 3522).

Fig. 36 depicts a robot control algorithm 3600 for a robotic system in one embodiment. The target objects in the selected object class are identified (item 3602), and the target position of the robot is determined as the neighboring position of the target object (item 3604). The path to the target object is determined as a series of waypoints (item 3606), and the robot navigates along the path while avoiding the obstacle (item 3608).

Once the neighboring location is reached, the target object is evaluated to determine if safe manipulation is possible (item 3610). The robot is operated to lift the object using a manipulator arm (e.g., a shovel) of the robot under conditions that can safely manipulate the target object (item 3612). The target object and nearby objects may now be analyzed with the perception module of the robot to better control the manipulation (item 3614).

Once the target object is on the blade or manipulator arm thereof, it is fixed (item 3616). The object drop mode is initiated (item 3618) if the robot has no capacity for more objects, or it is the last object in the selected category(s). Otherwise, the robot may begin the process again (3602).

Fig. 37 illustrates a robot control algorithm 3700 according to one embodiment. At block 3702, left and right cameras, or some other configuration of a robot camera, such as those disclosed herein, may provide inputs that may be used to generate scale-invariant keypoints within the robot's workspace.

"Scale-invariant keypoints" or "visual keypoints" in this disclosure refer to unique visual features that can be maintained across different perspectives (such as photographs taken from different regions). This may be an aspect in the captured image of the robot's workspace that may be used to identify features of the region or objects within the region when the features or objects are captured in other images taken from different angles, at different scales, or using different resolutions than the original capture.

The scale invariant keypoints may be detected by a robot or augmented reality robot interface mounted on the mobile device based on images taken by a camera of the robot or a camera of the mobile device. The scale invariant keypoints may help a robot or augmented reality robot interface on the mobile device determine geometric transformations between camera frames displaying matching content. This may help to confirm or fine tune the estimation of the position of the robot or mobile device within the robot's workspace.

The scale invariant keypoints may be detected, transformed, and matched for use by algorithms well known in the art such as, but not limited to, scale Invariant Feature Transform (SIFT), speeded Up Robust Features (SURF), oriented robust binary features (ORB), and SuperPoint.

At block 3704, objects located in the workspace of the robot may be detected based on inputs from the left and right cameras, defining a starting position of the objects and classifying the objects into categories. At block 3706, a re-identification fingerprint may be generated for the object, wherein the re-identification fingerprint is used to determine visual similarity of the object to the object detected in the future. The objects detected in the future may be the same objects that were re-detected as part of an update or transformation of the global region map, or may be similar objects that are similarly located at a future time, where re-identification fingerprints may be used to help classify the objects more quickly.

At block 3708, the robot may be positioned within a workspace of the robot. Inputs from at least one of a left camera, a right camera, a light detection and ranging (LIDAR) sensor, and an Inertial Measurement Unit (IMU) sensor may be used to determine the robot position. The robot's workspace may be mapped to create a global region map that includes scale-invariant keypoints, objects, and starting positions of objects. At block 3710, objects within the workspace of the robot may be re-identified based on at least one of the starting location, the category, and the re-identification fingerprint. At block 3712, each object may be assigned a persistent unique identifier.

At block 3714, the robot may receive the camera frame from the augmented reality robot interface installed as an application on the mobile device operated by the user and may update the global region map with the starting location and scale invariant key points using camera frame-to-global region map transforms based on the camera frame. In the camera frame to global region map transformation, the global region map may be searched to find a set of scale invariant keypoints that match those detected in the moving camera frame by using a specific geometric transformation. The transformation can maximize the number of matching keypoints and minimize the number of non-matching keypoints while maintaining geometric consistency.

At block 3716, a user indicator may be generated for the object, wherein the user indicator may include a next target, a target order, a hazard, being too large, vandalism, confusion, and blocking the travel path. At block 3718, the global region map and object details may be transmitted to the mobile device, wherein the object details may include at least one of: visual snapshots of objects, categories, starting locations, persistent unique identifiers, and user indicators. As supported by the communication 134 module introduced in fig. 1C and the network interface 1912 introduced in fig. 19, this information may be communicated using wireless signaling such as bluetooth or Wifi.

The updated global region map, objects, starting locations, scale-invariant keypoints, and object details may be displayed on the mobile device using an augmented reality robot interface. At block 3720, the augmented reality robot interface may accept user input to the augmented reality robot interface, wherein the user input indicates an object property override, including changing an object type, next put, no put, and modify user indicators. The object property overlay may be transmitted from the mobile device to the bot and may be used to update the global region map, user indicators, and object details at block 3722. Returning to block 3718, the robot may retransmit its updated global region map to the mobile device to resynchronize the information.

Fig. 38 illustrates a robotic system 3800 according to one embodiment. The robotic system 3800 may include a robot 100, a charging station 3802, a plurality of destination bins 3804 in which objects 3806 may be placed, and logic 3814 to allow the robot 100 to perform the disclosed methods, each destination bin 3804 being associated with at least one object class (such as object class 3808, object class 3810, and object class 3812). The robot 100, charging station 3802, object 3806, and destination bin 3804 may be located in an area that may be considered a workspace 3816 of the robot.

The robot 100 may use the sensors and cameras illustrated in fig. 1C and 1D to detect features of the robot's workspace 3816. These features may include scale invariant keypoints 3818, such as walls, corners, furniture, and the like. The robot 100 may also detect objects 3806 on the floor of the robot's workspace 3816, and these objects 3806 may be based on the destination bins 3804 in which the categories are placed, the robot 100 may determine the categories based on user input, recognition of similarity to objects treated in the past, machine learning, or some combination of these. The robot 100 may also use its sensors and cameras to locate itself within the robot's workspace 3816. The robot 100 may synthesize all of this data into a global region map 3820, as described with respect to fig. 37.

In one embodiment, the robotic system 3800 may also include a mobile device 3822 having an installed augmented reality robot interface application 3824, and the capability to provide a camera frame 3826. The robotic system 3800 may include a user having a mobile device 3822 such as a tablet or smartphone. The mobile device 3822 can have an installed augmented reality robot interface application 3824 that functions in accordance with the present disclosure. The augmented reality robot interface application 3824 may provide the camera frames 3826 using a camera configured as part of the mobile device 3822. The camera frame 3826 may include a ground plane 3828, which may be identified and used to position the mobile device 3822 within the robotic system 3800 such that information about the workspace 3816 of the robot detected by the robot 100 may be transformed according to the camera frame-to-global region map transformation 3830 for allowing the robot 100 and the mobile device 3821 to remain synchronized with respect to the objects 3806 in the workspace 3816 of the robot and the user indicators and object attribute overlays that may be attached to these objects 3806.

In one embodiment, the global region map 3820 may be a top-down two-dimensional representation of the robot workspace 3816. The global region map 3820 may undergo a camera frame to global region map transformation 3830 such that information detected by the robot 100 may be represented in the augmented reality robot interface application 3824 from the point of view of the user. The global region map 3830 may be updated to include the mobile device location 3832, the robot location 3834, the object start location 3836, and the object drop location 3838. In one embodiment, the global area map 3820 may identify furniture or other objects 3806 as obstacles 3840. During that stage of pickup, object 3806 other than the target object that 100 is currently considering may be considered an obstacle 3840. In one embodiment, the augmented reality robot interface application 3834 may also show a mobile device location 3832 and a robot location 3834, which are not indicated in this illustration.

Various functional operations described herein may be implemented in logic that is referred to using nouns or noun phrases that reflect the described operations or functions. For example, the association operation may be performed by a "correlator" or a "correlator". Likewise, switching may be performed by a "switch", selected by a "selector", and so forth. "logic" refers to machine memory circuitry and non-transitory machine-readable media including machine-executable instructions (software and firmware), and/or circuitry (hardware) by which material and/or material energy configurations include control and/or program signals, and/or settings and values (such as resistance, impedance, capacitance, inductance, current/voltage ratings, etc.) that may be used to affect operation of a device. Magnetic media, electronic circuitry, electrical and optical memory (both volatile and non-volatile) and firmware are examples of logic. Logic explicitly excludes pure signals or software itself (but does not exclude machine memory containing software and thereby forming the material arrangement).

Within this disclosure, different entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or referred to as being "configured" to perform one or more tasks or operations. This expression-an entity-is configured for [ performing one or more tasks-is used herein to refer to a structure (i.e., something of an entity, such as an electronic circuit). More specifically, the expression is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure may be said to be "configured to" perform certain tasks even if the structure is not currently operated. For example, "credit allocation circuitry configured to allocate credits to a plurality of processor cores" is intended to encompass integrated circuits having circuitry that performs this function during operation, even if the integrated circuit is not currently in use (e.g., a power supply is not connected to the integrated circuit). Thus, an entity described or detailed as "configured to" perform certain tasks refers to some physical east-west such as a device, circuitry, memory storing program instructions executable to perform the tasks, and the like. The phrase is not used herein to refer to something that is intangible.

The term "configured for" is not intended to mean "configurable for". For example, an unprogrammed Field Programmable Gate Array (FPGA) is not considered "configured to" perform certain specific functions, although it may be "configurable to" perform that function after programming.

In the appended claims, the detailed structure "configured to" perform one or more tasks explicitly means 35u.s.c. ≡112 (f) that is not intended to refer to the claim elements. Accordingly, claims that do not otherwise include means for performing a function in this disclosure should not be construed in accordance with 35U.S. c. ≡112 (f).

As used herein, the term "based on" is used to describe one or more factors that affect a determination. This term does not exclude the possibility that additional factors may influence the determination. That is, the determination may be based solely on specified factors, or on specified factors as well as other unspecified factors. Consider the phrase "determine a based on B". This phrase specifies that B is a factor for determining a or affecting the determination of a. This phrase does not exclude that the determination of a may also be based on some other factor, such as C. This phrase is also intended to cover embodiments in which a is determined based solely on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on".

As used herein, the phrase "responsive to" describes one or more factors that trigger an impact. This phrase does not exclude the possibility that additional factors may affect or otherwise trigger the effect. That is, the impact may be responsive only to these factors, or may be responsive to specified factors as well as other unspecified factors. Consider the phrase "execute a in response to B". This phrase specifies that B is a factor that triggers the execution of a. This phrase does not exclude that executing a may also be responsive to some other factor, such as C. This phrase is also intended to cover embodiments in which a is performed in response to B only.

As used herein, the terms "first," "second," and the like are used as labels for their preceding nouns, and do not imply any sort of order (e.g., spatial, temporal, logical, etc.) unless otherwise indicated. For example, in a register file having eight registers, the terms "first register" and "second register" may be used to refer to any two of the eight registers, and not just logical registers 0 and 1, for example.

The term "or" when used in the claims is intended to be inclusive or rather than exclusive or. For example, the phrase "at least one of x, y, or z" means any one of x, y, and z, and any combination thereof.

As used herein, recitation of "and/or" with respect to two or more elements should be interpreted to mean only one element or combination of elements. For example, "element a, element B, and/or element C" may include element a only, element B only, element C only, element a and element B, element a and element C, element B and element C, or elements A, B and C. In addition, "at least one of the element a or the element B" may include at least one of the element a, at least one of the element B, or at least one of the element a and at least one of the element B. Further, "at least one of the element a and the element B" may include at least one of the element a, at least one of the element B, or at least one of the element a and at least one of the element B.

The subject matter of the present disclosure is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Furthermore, although the terms "step" and/or "block" may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when individual steps are explicitly described.

Having thus described the illustrative embodiments in detail, it will be apparent that modifications and variations are possible without departing from the scope of the disclosure as claimed. The scope of the disclosed subject matter is not limited to the depicted embodiments, but is set forth in the following claims.

Claims

1. A method, comprising:

receiving a starting position and properties of a target object to be lifted by a robot, the robot comprising a robot control system, a shovel, a gripper pad arm with a gripper pad, and at least one wheel or one track for mobility of the robot;

Determining an object isolation policy comprising at least one of: using reinforcement learning based policies, rule based policies including rewards and penalties, relying on observations, current object states and sensor data;

executing the object isolation policy to isolate the target object from other objects;

determining a pick-up strategy, comprising:

a path of approach for the robot to the target object;

A grip height for initial contact with the target object;

a capture mode for movement of the gripper pad while capturing the target object; and

A load-bearing position of the gripper pad and the shovel, the load-bearing position securing the target object in a containment area on the robot for transport, the containment area including at least two of the gripper pad arm, the gripper pad, and the shovel;

executing the pick-up strategy, comprising:

extending the gripper pad outwardly and forwardly relative to the gripper pad arm and raising the gripper pad to the gripping height;

Approaching the target object via the approach path, stopping when the target object is positioned between the gripper pads;

performing the capture mode to allow capture of the target object within the containment area; and

Confirming that the target object is within the accommodation area;

under the condition that the target object is in the accommodation area:

applying pressure to the target object with the gripper pad to hold the target object stationary in the containment region; and

Raising at least one of the shovel and the gripper pad holding the target object to the load-bearing position; and

Under the condition that the target object is not in the accommodation area:

The pick-up strategy is changed using at least one of: different reinforcement learning based policies, different rule based policies, and depending on different observations, current object states, and sensor data; and

The changed pick-up strategy is performed.

2. The method of claim 1, further comprising:

Navigating to a drop location at the destination;

determining a fall-off strategy using a machine learning model or a rule-based method; and

Executing the drop policy, comprising:

Determining a destination approaching path and an object storage mode, wherein the object storage mode is one of a falling mode and a placing mode; and

Approaching the destination via the destination approaching path;

Under the condition that the object storage mode is the placement mode:

Stopping with the destination in front of the shovel and gripper pad;

lowering the shovel and the gripper pad to a storage height; and

Performing at least one of:

using the gripper pad to push the target object out of the receiving area and into the dropped position; and

Tilting the shovel forward, thereby allowing the target object to fall out of the containment area and into the dropped position; and

Under the condition that the object storing mode is the falling mode:

stopping with the destination at the rear of the shovel and the gripper pad;

raising the shovel and the gripper pad to the storage height; and

Extending out of the gripper pad and allowing the target object to fall from the receiving area and into the falling position.

3. The method of claim 2, wherein the rule-based policy for the drop policy comprises at least one of:

Navigating the robot to a position immediately adjacent to a side of the bin;

rotating the robot into position to align the robot facing the bin;

driving the robot toward the bin to maintain alignment centered on the side of the bin;

stopping at a short distance from the sides of the cartridge;

navigating with a rear camera if the user falls after attempting;

navigating with a front camera if a fall before an attempt is made; and

Verifying that the robot is properly positioned relative to the sides of the bin;

on condition that the robot is correctly positioned, at least one of the following is performed:

Lifting the shovel upwardly and rearwardly to drop the target object into the bin; and

Lifting the shovel upward and tilting the shovel forward to drop the target object into the bin; and

Under the condition that the robot is not positioned correctly:

the drop strategy is driven and re-executed remotely from the bin.

4. The method of claim 2, wherein the rewards and penalties for executing the drop policy include at least one of:

Penalty added for each second exceeding the maximum time;

rewards when the robot correctly stops the storage bin;

rewards when the target object successfully falls into the bin;

A penalty for a collision to move the bin;

a penalty for collision with an obstacle or wall exceeding a force feedback maximum; and

A penalty in case the robot is stuck or driven beyond the target object.

5. The method of claim 1, wherein the rewards and penalties for executing the object isolation policy include at least one of:

Penalty added for each second exceeding the maximum time;

rewards when the correct gripper pad arm is between the target object and the wall;

rewards when the target object is more than a predetermined distance from the wall;

A penalty for incorrectly colliding with the target object;

A penalty in case the robot is stuck or driven beyond the target object.

6. The method of claim 1, wherein the rewards and penalties for executing the pick-up policy include at least one of:

Penalty added for each second exceeding the maximum time;

rewards when the target object first contacts the edge of the shovel;

Rewards when the target object is pushed completely into the shovel;

a penalty when the target object is lost from the shovel;

A penalty for collision with an obstacle or wall exceeding a force feedback maximum;

Punishment for picking up non-target objects; and

A penalty in case the robot is stuck or driven beyond the target object.

7. The method of claim 1, wherein the rule-based policy for object isolation comprises at least one of:

Navigating the robot to a position facing a target object to be isolated, but far enough apart to open the gripper pad arm and the gripper pad and lower the shovel;

opening the gripper pad arm and the gripper pad, lowering the gripper pad arm and the gripper pad, and lowering the shovel;

slightly turning the robot into position so that the target object is centered in front view;

opening the gripper pad arm and the gripper pad such that the gripper pad arm and the gripper pad are slightly wider than the target object;

slowly driving forward until the gripper pad arm and the end of the gripper pad are positioned past the target object;

Slightly closing the gripper pad arm and the gripper pad in a V-shape such that the gripper pad arm and the gripper pad surround the target object; and

A short distance is driven backwards to move the target object into the open space.

8. The method of claim 1, further comprising: assessing target object pick-up success, including at least one of:

Detecting the target object within the containment area of the shovel and the gripper pad arm to determine if the target object is within the containment area;

Receiving force feedback from an actuator force feedback sensor, the force feedback indicating that the target object is retained by the gripper pad arm;

Tracking the movement of the target object during picking up of the target object to the area of the shovel and retaining the state of the target object in memory;

Detecting an increased weight of the shovel during lifting of the target object, the increased weight being indicative of the target object being in the shovel;

determining whether the target object is in the shovel using a classification model; and

Rechecking that the target object is in the shovel while the robot is in motion using at least one of: the force feedback, the added weight, and a dedicated camera.

9. The method of claim 1, wherein the reinforcement learning strategy and the rule-based strategy comprise controlling actions of individual actuators, the actions comprising at least one of:

moving the left gripper pad arm to a new position by rotating up or down;

moving the left gripper pad wrist to a new position by rotating left or right;

moving the right gripper pad arm to a new position by rotating up or down;

Moving the right gripper pad wrist to a new position by rotating left or right;

lifting the shovel to a new position by rotating up or down;

Changing the angle of the shovel with a second motor or a second actuator, thereby producing a front drop of the target object;

driving a left wheel or a left crawler on the robot; and

And driving a right wheel or a right crawler on the robot.

10. The method of claim 1, wherein the reinforcement learning strategy and the rule-based strategy comprise a composite action that controls an actuator, the composite action comprising at least one of:

Driving the robot to a position or waypoint following a path;

rotating the robot left or right into position;

Centering the robot relative to the target object;

Aligning the gripper pad arm with a top or bottom or middle portion of the target object;

Driving forward until the target object abuts against an edge of the shovel;

closing both of the gripper pad arms and pushing the target object with a smooth motion;

Lifting the shovel and the gripper pad arm together while grasping the target object;

Closing both of the gripper pad arms and pushing the target object with a quick tap and a slight release;

placing the spade gently against a floor;

Pushing the shovel downward against the floor;

closing the gripper pad arm until resistance is encountered and position is maintained; and

The gripper pad arm is closed by vibration and left or right steering to create a slight bounce of the unstable and flat target object on the edge of the blade.

11. A robotic system, comprising:

A robot, the robot comprising:

A shovel;

A gripper pad arm having a gripper pad;

At least one wheel or one track for mobility of the robot;

A processor; and

A memory storing instructions that, when executed by the processor, allow operation and control of the robot;

A base station;

A plurality of bins storing objects;

a robot control system in at least one of the robot and a cloud server; and

Logic for:

Receiving a starting position and an attribute of a target object to be lifted by the robot;

Determining an object isolation policy, the object isolation policy comprising at least one of: using reinforcement learning based policies, rule based policies including rewards and penalties, relying on observations, current object states and sensor data;

determining a pick-up strategy, comprising:

a path of approach for the robot to the target object;

A grip height for initial contact with the target object;

executing the pick-up strategy, comprising:

Confirming that the target object is within the accommodation area;

under the condition that the target object is in the accommodation area:

Raising at least one of the shovel and the gripper pad holding the target object to the load-bearing position;

under the condition that the target object is not in the accommodation area:

The changed pick-up strategy is performed.

12. The robotic system of claim 11, further comprising logic to:

Navigating to a drop location at the destination;

Determining a fall-off strategy using a machine learning model or a rule-based method;

Executing the drop policy, comprising:

Approaching the destination via the destination approaching path;

Under the condition that the object storage mode is the placement mode:

Stopping with the destination in front of the shovel and gripper pad;

lowering the shovel and the gripper pad to a storage height; and

Performing at least one of:

Under the condition that the object storing mode is the falling mode:

stopping with the destination at the rear of the shovel and the gripper pad;

raising the shovel and the gripper pad to the storage height; and

13. The robotic system of claim 12, wherein the rule-based policy for the drop policy comprises at least one of:

Navigating the robot to a position immediately adjacent to a side of the bin;

rotating the robot into position to align the robot facing the bin;

stopping at a short distance from the sides of the cartridge;

navigating with a rear camera if the user falls after attempting;

navigating with a front camera if a fall before an attempt is made; and

Under the condition that the robot is not positioned correctly:

the drop strategy is driven and re-executed remotely from the bin.

14. The robotic system of claim 12, wherein the rewards and penalties for executing the drop strategy include at least one of:

Penalty added for each second exceeding the maximum time;

rewards when the robot correctly stops the storage bin;

rewards when the target object successfully falls into the bin;

A penalty for a collision to move the bin;

A penalty in case the robot is stuck or driven beyond the target object.

15. The robotic system of claim 11, wherein the rewards and penalties for executing the object isolation policy include at least one of:

Penalty added for each second exceeding the maximum time;

A penalty for incorrectly colliding with the target object;

A penalty in case the robot is stuck or driven beyond the target object.

16. The robotic system of claim 11, wherein the rewards and penalties for executing the pick-up strategy include at least one of:

Penalty added for each second exceeding the maximum time;

rewards when the target object first contacts the edge of the shovel;

Rewards when the target object is pushed completely into the shovel;

a penalty when the target object is lost from the shovel;

Punishment for picking up non-target objects; and

A penalty in case the robot is stuck or driven beyond the target object.

17. The robotic system of claim 11, wherein the rule-based policy for object isolation comprises at least one of:

18. The robotic system of claim 11, further comprising logic for evaluating a target object pick success, comprising at least one of:

Tracking movement of the target object during picking up of the target object to the area of the shovel and retaining a state of the target object in the memory;

19. The robotic system of claim 11, wherein the reinforcement learning strategy and the rule-based strategy include controlling actions of individual actuators, the actions including at least one of:

moving the left gripper pad arm to a new position by rotating up or down;

moving the left gripper pad wrist to a new position by rotating left or right;

moving the right gripper pad arm to a new position by rotating up or down;

Moving the right gripper pad wrist to a new position by rotating left or right;

lifting the shovel to a new position by rotating up or down;

driving a left wheel or a left crawler on the robot; and

And driving a right wheel or a right crawler on the robot.

20. The robotic system of claim 11, wherein the reinforcement learning strategy and the rule-based strategy comprise a composite action that controls the actuator, the composite action comprising at least one of:

Driving the robot to a position or waypoint following a path;

rotating the robot left or right into position;

Centering the robot relative to the target object;

Driving forward until the target object abuts against an edge of the shovel;

placing the spade gently against a floor;

Pushing the shovel downward against the floor;