WO2014177969A1

WO2014177969A1 - Method and apparatus for automatically handling larvae grafting operations in beekeeping

Info

Publication number: WO2014177969A1
Application number: PCT/IB2014/060798
Authority: WO
Inventors: Marco Balsi; Salvatore Esposito
Original assignee: Garuleu S.R.L.
Priority date: 2013-05-02
Filing date: 2014-04-17
Publication date: 2014-11-06
Also published as: ITMI20130718A1; EP2908624A1

Abstract

An apparatus adapted to automatically handle larvae grafting operations in beekeeping is described, which comprises: - means for containing at least one bee brood comb (3), said comb comprising cells suitable for receiving bee larvae; - means for containing at least one support (4) for artificial cells suitable for receiving bee larvae; - a movable system (5) comprising: an image sensor (8); a manipulator (7) adapted to pick said bee larvae from the cells of said at least one comb (3) and to place them into said artificial cells; handling means (6) adapted to move said image sensor (8) and manipulator (7) in the apparatus; - control means adapted to control the movement "of said~haridiing means "(6), so that said' image sensor (8) will identify the positions of those larvae which are suited to being picked from the cells of said at least one comb (3), and said manipulator (7) will act in accordance with said identified larvae.

Description

METHOD AND APPARATUS FOR AUTOMATICALLY HANDLING LARVAE GRAFTING OPERATIONS IN BEEKEEPING

DESCRIPTION

Field of the invention

The present invention relates to a method and an apparatus for automatically handling larvae grafting operations for royal jelly production and queen bee rearing.

Background art

Royal jelly production represents an important integration of the income of an apiary farm. At present, however, royal jelly production is strongly limited because it requires employing a large number of skilled workers, although the large demand for a high-quality product like the Italia one would absorb any production increase that might be attained. Stimulation of the royal jelly production process is now effected only manually through a slow and delicate operation called "larvae grafting". It includes the following main steps:

1. manual extraction of new-born bee larvae from a brood comb, by means of a larvae picking tool: this operation is slow and must be carried out by a very skilled operator;

2. transferring the larvae and "grafting" them into artificial queen cell cups, e.g. contained in a bar; this operation is referred to as "larvae grafting";

3. placing the bar of queen cell cups into a box for royal jelly production or into a normal hive, where young orphan nurse bees will feed the larvae in order to produce new queens;

4. extracting the bar of queen cell cups after approximately 72 hours from deposition;

5. extracting the larvae from the bar of queen cell cups;

6. aspirating the royal jelly produced.

The larvae grafting operation is currently carried out by highly skilled operators without using any technologic equipment. The larvae grafting operation takes most of the specialized work time needed during the above-described process for the production of royal jelly.

For example, for breeding 50 production units by two experienced and qualified beekeepers, the yearly labour requirement will be approximately 2,000 hours, all concentrated within the spring-summer period, and the labour portion allocated to larvae grafting operations is large, corresponding to approximately 40% of the total time. Therefore, said larvae grafting operations are not optimized as concerns process times and procedures, leading to the necessity of introducing automatic steps useful for reducing the times and costs of the required operations.

Summary of the invention

It is one object of the present invention to provide a method and an apparatus allowing automatic handling of larvae grafting operations, thus overcoming the drawbacks of manual operations, first and foremost the laboriousness thereof, that must be carried out by highly skilled operators, while at the same time ensuring, thanks to the controlled environmental conditions in which the process takes place, the utmost hygiene and the least impact on the brood in the comb.

The basic idea of the present invention is to automatically recognize the larvae to be transferred in the^" brood comb^" through the use of image processing^" techniques, to automatically insert them into artificial cells, and to automatize the transfer by means of robotic systems.

It is one object of the present invention to provide a machine comprising a box into which one or more brood combs and one or more bars or other suitable supports with artificial cells (also referred to as cell cups) are introduced, whether manually or by means of an automatic handling system.

The machine includes an image acquisition system comprising a video camera mounted on a suitable handling system, and a robotic transfer system carrying an automatic larvae picking tool. The machine is connected to a processing and control system comprising a computing unit that carries out the image processing activity, i.e. the recognition of the position of the suitable larvae, and controls the robotic handling and transfer system, through interaction with the operator.

It is one object of the present invention to provide an apparatus adapted to automatically handle larvae grafting operations in beekeeping, which comprises:

- means for containing at least one bee brood comb, said comb comprising cells suitable for receiving bee larvae;

- means for containing at least one support for artificial cells suitable for receiving bee larvae;

- a movable system comprising: an image sensor; a manipulator adapted to pick said bee larvae from the cells of said at least one comb and to place them into said artificial cells; handling means adapted to move said image sensor and manipulator in the apparatus;

- control means adapted to control the movement of said handling means, so that said image sensor will identify the positions of those larvae which are suited to being picked from the cells of said at least one comb, and said manipulator will act in accordance with said identified larvae.

It is a particular object of the present invention to provide a method and an apparatus for automatic handling of larvae grafting operations as set out in the claims, which are an integral part of the present description.

Brief description of the drawings

Further objects and advantages of the present invention will become apparent from the following detailed description of a preferred embodiment (and variants) thereof and from the annexed drawings, which are only supplied by way of non-limiting example, wherein: Figure 1 is a general block diagram of the system according to the present invention;

Figures 2 and 3 are two functional block views, from the top and from the rear, respectively, of an example of embodiment of the machine according to the present invention;

Figure 4 shows an axonometric view and a side view, respectively, of a detail of the manipulator block of Figures 2 and 3;

Figure 5 shows an operational flow chart of the method according to the invention;

Figures 6 and 7 show some examples of images relating to the recognition and positioning of the larva into the comb cell.

In the drawings, the same reference numerals and letters identify the same items or components.

Detailed description of some embodiments of the invention

With reference to Figure 1, the automatic larvae grafting machine according to the invention can be divided into two functional blocks:

• Functional block 1; Larvae identification;

• Functional block 2: Automatic larvae handling system.

The functional block 1 is the intelligence of the system. This block can be further exploded and subdivided into two sub-blocks, i.e. a hardware block and a software block, respectively.

The hardware section includes: - a video camera or photo camera or another image sensor, also in the spectrum extended beyond the visible, in a fixed configuration to frame the entire comb or, preferably, of the "eye-in-hand" type, i.e. movable on a mechanical support and capable of framing small areas;

- an embedded computer or PC for image acquisition and processing. The same computer will control the automation of the handling system (functional block 2).

The software section includes image processing applications having the following essential functionalities:

^•S image improvement (equalization, noise reduction, brightness and contrast adjustment);

extraction of significant features, such as edges, closed shapes, regular patterns (in particular, the cell matrix);

morphological operations for extracting recognition candidates;

automatic decision based on systems equipped with learning functions, fuzzy logic, expert systems, etc.;

^•S possible interaction with an operator.

The functional block 2 implements the larvae picking solution by means of high-precision robotic solutions, such as those listed below.

^•S mechanical arm or pantograph system or two-axis sliding system, or another suitable handling system;

^•S servomechanisms for inserting, in the optimal position relative to the position of the larva within the cell, a spring-type larvae picking tool, as described in detail below, or, preferably, a similar tool specially designed for use in this machine.

With reference to Figures 2, 3 and 4, the following will describe an example of embodiment of the machine according to the invention. It appears as a container that contains at least one comb 3, at least one bar containing cell cups 4, which in this example is arranged beside the comb 3, the movable system 5 comprising the image sensor device and a manipulator.

The machine is preferably wholly enclosed into a thermostat-controlled and hygienically protected container 9 for the purpose of preserving the quality of the product and the life of the larvae.

The movable system 5 comprises a two-axis handling system 6, on which the manipulator 7 and the image sensor device 8 are installed.

In Fig. 3, which shows a rear cross-section of the system, one can see, in particular, that the handling system 6 has two axes and allows positioning the manipulator 7 for picking the larva and then grafting it into a cell cup (artificial queen cell). In the solution described herein, the cell cups are fixed on a bar 4 acting as a support, but in an alternative solution they may be transported on the same movable support and then fixed on a bar.

Fig. 4 shows an enlarged view of the manipulator 7, which is based on the operating principle of a spring-type larvae picking tool. It comprises a head 10, to which a flexible reed 1 1 is connected for picking the larva from the comb, and a piston 12 for subsequently depositing the picked larva into the cell cup.

In particular, once the cell with the larva that will have to be transferred into the cell cup has been identified, the manipulator is positioned above the cell and goes down alongside the cell; the flexible reed will follow the bottom of the cell, passing under the royal jelly contained therein. In this manner, the larva will stay attached to the reed because of the adhesion of the royal jelly, and extraction is thus complete. At this point, the manipulator is brought into the position of the first free cell of the cell cup, and the piston on the manipulator allows dropping the larva into the empty cell to complete the grafting operation.

Preferably, the larvae picking tool is inserted by following the angle of the cell with respect to the comb plane, e.g. 5°-7°, and can be rotated relative to the cell axis, if necessary, so that it can enter the cell and extract the larva without damaging it. The larvae picking tool is then extracted with the larva and re-positioned into an artificial queen cell, where it is operated in order to deposit the larva.

Other possible embodiments of the manipulator may make use of a teaspoon-type or spring-type larvae picking tool.

In an alternative solution, the manipulator has several flexible tongues on all sides, so as to ensure that the larva will be picked up regardless of its position in the cell.

In the preferred embodiment, the video camera and the manipulator are moved integrally with each other on a shelf, their maximum travel allowing them to reach the whole comb and the cell cup bars positioned beside it.

In its turn, the arm of the larvae picking tool is moved relative to the shelf and is positioned with high precision by means of a fine adjustment system, its travel being limited to the visual field of the camera, which is relatively narrow compared with the comb surface.

In the system as a whole, the automatic larvae grafting solution can operate sequentially on multiple frames (combs) at a time.

In the simplest embodiment, the machine acts upon one side only of a single comb. The mechanism can be doubled or anyway adapted for working on both sides of the comb; also, a comb handling system can be implemented, which allows loading the machine with a certain quantity of combs, which are then automatically positioned in succession in the work area.

The same handling system can be used for positioning the bars carrying the artificial queen cells.

Moreover, the system can be equipped with means for automatically handling the combs and the cell cup supports, as well as with suitable spaces for loading and removing combs and cell cups without interrupting the operation of the machine.

The operational flow of the machine which is the subject of the method of the invention will now be described with reference to the flow chart shown in Figure 5.

This description will refer to a typical configuration of the machine, which uses a video camera that is movable on a two-axis handling system capable of framing small areas

("eye-in-hand"), and wherein the comb is loaded into the machine by an operator.

After the comb has been inserted into the machine (Step 51, Comb insertion), the comb cells are scanned by a video camera directly connected to the robotic support used for picking (Step 52, Positioning).

When the comb is in the machine, the video camera can frame comb blocks having a size of approximately 8x6 cells. Once the image has been acquired, it is processed (Step 53, Image processing) through the procedures described below for improving the image, highlighting the larvae, and automatically recognizing them. The result of this processing will be the identification of the larva suited to grafting, i.e. its position relative to the cell (Step 54, Larva identified?).

Figures 6 and 7 show an example of larvae recognition, which is based on the use of "edge- detection" procedures.

In particular, a per se known "active contour" algorithm is used as described, for example, in: M. Kass, A. Witkin and D. Terzopoulos, "Snakes: active contour models", International Journal of Computer Vision, 1987, or as described in: D. L. Vilarino et al., "Pixel -Level Snakes", Proc. Int. Conf. Pattern Recognition, 2000, by providing as inputs the image of the cell (Fig. 6a) and a first processed version thereof (Fig. 6b), wherein the edges are detected through a combined application of the anisotropic diffusion algorithms, e.g. as described in Pietro Perona and Jitendra Malik, "Scale-space and edge detection using anisotropic diffusion", IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990, and the Canny algorithms, e.g. as described in J.Canny, "A Computational Approach to Edge Detection" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-8, n. 6, Nov. 1986.

It is thus possible to obtain the identification of the larva and its position relative to the cell.

In particular, it is possible to extract morphological information (texture) about both the cell (Fig! 6c) and the larva (Fig. 6d). In fact, by using binary images one can extract information about the shape, position and dimensions of the larva with respect to the cell. In particular, in order to improve the recognition of the larvae, they can be distinguished from other objects having similar color and size based on their shape. In particular, their oblong shape can be recognized by imposing a minimum threshold for the ratio between the maximum and minimum transversal dimensions. Their curvature can also be recognized by means of mathematical morphology operations, such as a comparison between the original shape and the same shape subjected to a closing operation, possibly acting upon the image skeleton.

Figure 6a shows a photograph of a larva in a comb cell. Figures 6b, 6c, 6d show the results of the identification algorithm; in particular, Fig. 6b shows the guiding information obtained, Fig. 6c shows the identification of the active region of the cell, and Fig. 6d shows the final result: identification of the position of the larva.

As highlighted in Figures 7a, 7b, 7c, information is extracted about the perimeter of the objects and the mass centre thereof. In Fig. 7a one can see the cell, in Fig. 7b one can see the larva, and in Fig. 7c one can see both. Knowing the distance between the two centres, the mechanical arm can be positioned appropriately for picking the larva.

It must be pointed out that, in order to properly identify the larva, it is sufficient to evaluate the space occupied by it within the cell. This operation is wholly equivalent to evaluating the percentage of the area occupied by the larva within the cell.

In particular, only larvae with a cell occupation percentage below a suitable threshold will be picked. In this case, the threshold is, for example, 2.5%. As an alternative, only those larvae within a given range of cell occupation percentage may be picked, e.g. between 1.9% and 2.5%. As a further alternative, one may only pick larvae recognized on the basis of parameters describing their shape.

Referring back to Figure 5, once the larva has been recognized, the information about its position relative to the cell is sent to the handling system (Step 55, Transmission of coordinates), and will be used for handling the manipulator in order to extract the larva (Step 56) and graft it into a cell cup (Step 57, Grafting).

All operations are carried out for each 8x6 block until the whole comb has been inspected (Step 58, Inspection complete?).

The control part of the present invention can advantageously be implemented through a computer program, which comprises coding means for implementing one or more steps of the method when said program is executed by a computer. It is therefore understood that the protection scope extends to said computer program as well as to computer-readable means that comprise a recorded message, said computer-readable means comprising program coding means for implementing one or more steps of the method when said program is executed by a computer.

The above-described example of embodiment may be subject to variations without departing from the protection scope of the present invention, including all equivalent designs known to a man skilled in the art.

The elements and features shown in the various preferred embodiments may be combined together without however departing from the protection scope of the present invention. The advantages deriving from the application of the present invention are apparent.

The automation process of the invention ensures a significant production increase, due to the fact that, the number of workers being the same, the farm will be able to handle at least twice the number of families for royal jelly production. The increased income will allow not only to quickly amortize the cost of the machine, but also to increase occupation possibilities. Furthermore, the automatic larvae grafting system may be used consortially within appropriate production areas.

In this way, in addition to increasing the profitability of the investment, small-to-medium enterprises will have the possibility to begin producing royal jelly in order to differentiate production and consolidate their business. The advantages of this solution clearly consist of a reduction of the time required for larvae grafting by skilled operators and process accuracy, resulting in the possibility for the farm to increase production with the same personnel (also taking into account that this specialized operation is often carried out by the owner of a small-to-medium apiary farm, who can hardly be replaced by other workers). Finally, more profit and better sustainability for the apiary farm can be attained through diversification, along with a possible occupational increase.

The device of the invention is not intended to replace skilled labor, but its application can contribute to modifying and improving the organization of the work necessary for royal jelly production.

From the above description, those skilled in the art will be able to produce the object of the invention without introducing any further construction details.

Claims

1. An apparatus adapted to automatically handle larvae grafting operations in beekeeping, comprising:

- means for containing at least one bee brood comb (3), said comb comprising cells suitable for receiving bee larvae;

- means for containing at least one support (4) for artificial cells suitable for receiving bee larvae;

- a movable system (5) comprising: an image sensor (8); a manipulator (7) adapted to pick said bee larvae from the cells of said at least one comb (3) and to place them into said artificial cells; handling means (6) adapted to move said image sensor (8) and manipulator (7) in the apparatus;

- control means adapted to control the movement of said handling means (6), so that said image sensor (8) will identify the positions of those larvae which are suited to being picked from the cells of said at least one comb (3), and said manipulator (7) will act in accordance with said identified larvae.

2. An apparatus according to claim 1, wherein said image sensor (8) is a video camera or a photo camera adapted to frame the entire comb or an "eye-in-hand" unit capable of framing comb areas, which is also adapted to identify said larvae on the basis of edge detection, segmentation, texture extraction and mathematical morphology procedures allowing to detect the perimeters and the centres of the cells and of the larvae in the cells.

3. An apparatus according to claim 2, wherein said larvae suited to being picked are larvae with a cell occupation percentage below a threshold or anyway within a given cell occupation percentage range, and/or larvae recognized on the basis of parameters describing the shape thereof.

4. An apparatus according to claim 1, wherein said manipulator (7) comprises a head (10) to which one or more flexible reeds (11) are connected for picking the larvae from the comb, and a piston (12) for subsequently depositing the picked larvae into said artificial cells.

5. An apparatus according to claim 1, wherein said manipulator (5) comprises a teaspoon-shaped or brush-type head.

6. An apparatus according to claim 1, comprising means for automatically moving the combs and the supports of the artificial cells, and spaces for loading and removing combs and artificial cells without interrupting the operation of the machine.

7. An apparatus according to claim 1, comprising a dual larvae recognition and handling system, so as to be able to simultaneously work on both sides of a comb.

8. An apparatus according to any one of the preceding claims, characterized in that it is enclosed into a thermostat-controlled and hygienically protected container (9) for the purpose of preserving the quality of the product and the life of the larvae.

9. A method for automatically managing larvae grafting operations in beekeeping, adapted to use an apparatus according to any one of the preceding claims, comprising the steps of:

- placing at least one bee brood comb (3) into said containing means, said comb comprising cells suitable for receiving bee larvae;

- placing at least one support (4) for artificial cells suitable for receiving bee larvae into said containing means;

- controlling the movement of said movable system (5) comprising: an image sensor (8); a manipulator (7) adapted to pick said bee larvae from the cells of said at least one comb (3) and to place them into said artificial cells; handling means (6) adapted to move said image sensor (8) and manipulator (7) in the apparatus, so that said image sensor (8) will identify the positions of those larvae which are suited to being picked from the cells of said at least one comb (3), and said manipulator (7) will act in accordance with said identified larvae.