JP2016057813A - Commodity management system and commodity management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a commodity management system and a commodity management method capable of surely identifying each commodity in a shopping basket.SOLUTION: A representative commodity management system 140 comprises: a server 116 for managing commodity information of each commodity; and a shopping basket 100 to which a communication appliance 104 for communicating with the server by radio communication is attached, and the commodity is taken in and out from the basket by a shopper. The communication appliance comprises: an imaging part 106 for imaging a moving image in the shopping basket; and a radio communication part 108 for sending the moving image imaged by the imaging part to the server. The server comprises: an object recognition part 166 which performs object recognition of each commodity put into the shopping basket, on the basis of the moving image, and tracks a position of each commodity; and an image recognition part 168 which performs image recognition of each commodity on the basis of the moving image to identify each commodity. The object recognition part continues to track the position of the commodity, at least until the image recognition part identifies the commodity.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a merchandise management system and merchandise management method for managing merchandise in a store such as a supermarket.

  In a store such as a supermarket, a shopper selects a desired product from a product shelf as appropriate and places the product in a shopping cart. Further, if the product once entered is not purchased, the shopper takes out the product from the shopping basket and returns it to the product shelf before the checkout at the cash register (clearing office).

  After that, at the cash register, the store clerk receives the shopping cart from the shopper and scans a barcode or the like attached to the product in the shopping cart, so that the total price of the product is displayed on the display or the like. The shopper then settles by paying the total amount. As described above, the total amount of commodities in the shopping cart is generally calculated for the first time at a cash register operated by a store clerk.

  On the other hand, there is known a technique for automatically calculating the total amount of products in the shopping cart regardless of the cash register (for example, Patent Document 1). Patent Document 1 describes an automatic product clearing system having a product cart (shopping basket) provided with a plurality of cameras and a product clearing apparatus that automatically clears the product.

  In this product automatic settlement system, first, when a product is stored in a shopping basket, the product is photographed from a plurality of directions by a plurality of cameras, and the plurality of photographed image information is transmitted to the product clearing apparatus as product information of the product. . Next, the merchandise clearing apparatus receives the plurality of pieces of image information of the merchandise transmitted as the merchandise information, and recognizes the merchandise based on the plurality of pieces of image information and time data at which the images are taken.

  Subsequently, the merchandise clearing apparatus refers to pre-registered merchandise information based on the merchandise recognition result and notifies the shopping cart of information including the merchandise name and the price of the merchandise. The notified product information is displayed on the display operation section of the shopping basket. In this way, in Patent Document 1, each time a product is stored in the shopping cart, the total amount of all the products to be purchased is calculated by the product clearing apparatus by repeating these processes.

Japanese Patent No. 3704499

  In the technology described in Patent Document 1, when a product is stored in a shopping basket, a plurality of image information obtained by photographing the product from a plurality of directions by a plurality of cameras, and time data when the image is photographed. Recognize the product based on. Japanese Patent Application Laid-Open No. H10-228707 describes that when a shopper takes out a product from the shopping basket in order to cancel the purchase of the product stored in the shopping cart, the product is recognized by the same process. (Paragraph 0042).

  However, the process of identifying a product by image recognition is complicated in the calculation process and takes a long time. In Patent Literature 1, when a product is stored in a shopping basket or when a product is taken out from the shopping cart, it is necessary to recognize an image from a plurality of pieces of image information in an instant, and the product can be reliably identified. It becomes difficult.

  In view of such a problem, an object of the present invention is to provide a merchandise management system and a merchandise management method that can more reliably identify merchandise in a shopping cart.

  In order to solve the above-described problems, a typical configuration of a product management system according to the present invention includes a server that manages product information of a product and a communication device that performs wireless communication with the server, and a product is input by a shopper. A product management system having a shopping basket to be taken out, wherein the communication device includes an imaging unit that captures a moving image in the shopping basket, and a wireless communication unit that transmits the moving image captured by the imaging unit to a server, The server recognizes the product that has been put into the shopping cart based on the video, and recognizes the product by recognizing the product image based on the video and the object recognition unit that tracks the position of the product. The object recognition unit continues to track the position of the product until at least the product is identified by the image recognition unit.

  Object recognition (object recognition) is lighter in arithmetic processing and finishes in a shorter time than processing for identifying a product by image recognition. Therefore, according to the above configuration, object recognition can be performed based on the moving image in the shopping basket transmitted from the communication device, and image recognition can be continued while tracking the position of the object. For this reason, the time required for image recognition can be secured and the product can be identified more reliably.

  The server may further include a control unit that interrupts the image recognition when the product is taken out of the shopping basket before the image recognition of the product by the image recognition unit is completed. Here, the object recognition unit continues to track the position of the object until the image recognition is completed. On the other hand, since the product taken out from the shopping basket is not a target for settlement, image recognition is not necessary. Therefore, as described above, when a product before completion of image recognition is taken out of the shopping basket, the image recognition of this product is interrupted, thereby reducing the load associated with image recognition.

  The server further includes a product master table in which product information is stored, and an in-car product table created by the control unit for each shopping cart with reference to the product master table. The object ID is registered in the in-car product table, and when the product is identified by image recognition, the product information of the product is read from the product master table, and the product information is registered in the in-car product table.

  Thereby, the product information of the products remaining in the shopping cart can be registered in the in-car product table. For this reason, the server can manage the product name, the price, the weight, and the like of the product in the shopping cart by referring to the in-car product table. When a product is taken out from the shopping basket, a flag indicating that may be set in the record of the product, or the record of the product may be deleted.

  The communication device may further include a lamp that lights up when product information is registered in the in-car product table for all products in the shopping cart. Thus, the shopper can check out at the unmanned cash register when the lamp is lit, and can check out at the manned cash register when the lamp is off. Therefore, it is possible to smoothly guide the shopper to an appropriate cash register.

  The control unit may calculate a motion vector based on a change in the position of the product due to object recognition using a moving image, and determine whether the product has been taken out from the shopping cart based on the motion vector. As described above, since the object recognition is performed based on a moving image instead of a still image, the control unit can calculate a motion vector. For example, the control unit can determine that the product has been taken out from the shopping basket when the product disappears from the moving image with the upward motion vector.

  The object recognition unit may continue the object recognition even after the product is identified. Thereby, when a product is taken out from the shopping basket, the product can be specified without performing image recognition. That is, there is no need to instantly recognize an image when a product is taken out from the shopping basket. This makes it possible to appropriately grasp the increase / decrease in the products in the shopping basket.

  The representative configuration of the merchandise management method according to the present invention uses a shopping cart in which merchandise information of merchandise is managed and a communication device that wirelessly communicates with the server is provided and merchandise is input or removed by a shopper. A product management method, wherein a communication device captures a video in a shopping cart and transmits the video to a server, and the server performs object recognition and image recognition of the product put in the shopping cart based on the video. And at least until the product is identified by image recognition, the position of the product is continuously tracked by object recognition.

  The components corresponding to the technical idea in the product management system described above and the description thereof can also be applied to the method.

  ADVANTAGE OF THE INVENTION According to this invention, the merchandise management system and merchandise management method which can identify the goods in a shopping basket more reliably can be provided.

It is a figure which shows roughly the external appearance of the shopping basket used for the merchandise management system in this embodiment. It is a figure which shows roughly the shop where the merchandise management system in this embodiment is applied. It is a functional block diagram which illustrates the composition of the merchandise management system in this embodiment. It is a flowchart which shows each process of the merchandise management method in this embodiment. It is a figure for demonstrating the process when goods are thrown into the shopping basket of FIG. It is a figure which shows the goods table in a car produced when goods are thrown into the shopping basket of FIG. It is a figure for demonstrating the process at the time of goods being taken out from the shopping basket of FIG. It is a figure which shows the goods table in a basket produced when goods are taken out from the shopping basket of FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  FIG. 1 is a diagram schematically showing the appearance of a shopping basket used in the product management system in the present embodiment. FIG. 1A is a perspective view of a shopping basket. FIG. 1B is a front view of FIG. FIG. 2 is a diagram schematically illustrating a store to which the product management system according to the present embodiment is applied. FIG. 2A shows the inside of a supermarket as an example of a store. FIG. 2B is a perspective view showing the unmanned cash register of FIG.

  As shown in FIG. 1A, the shopping basket 100 is provided with a communication device 104 on a side surface 102 thereof. The communication device 104 is a board that can be attached to or detached from the shopping basket 100, and includes, for example, an imaging unit 106, a wireless communication unit 108, and a lamp 110 as shown in FIG. The imaging unit 106 is a moving image camera, and is located above the upper edge 112 of the shopping basket 100 as shown in FIG. 1A, and images a moving image in the shopping basket 100 from above. Note that the shopping basket 100 is covered with a light-shielding net 114 that uses both linen and the like to achieve both light-shielding properties and air permeability, so that imaging can be performed satisfactorily.

  The wireless communication unit 108 transmits the moving image captured by the imaging unit 106 to the server 116 (see FIG. 3). For example, the lamp 110 is turned on when the unmanned cash register 118 shown in FIG. 2 can be settled, and is turned off when the manned cash register 120 is cleared. That is, the lamp 110 guides the shopper to the unmanned cash register 118 or the manned cash register 120 depending on whether or not the lamp 110 is lit.

  As shown in FIG. 2A, in a store such as a supermarket 130, a shopper 132 has a shopping basket 100 arranged near the entrance 134, selects a product from various product shelves 136a to 136d, and selects the product. Put it in the shopping basket 100. Further, if the product once entered is not purchased, the shopper 132 takes out the product from the shopping basket 100 and returns it to the product shelves 136a to 136d before settlement. After repeating such an operation, the shopping basket 100 stores products to be settled.

  In the present embodiment, the product information on all the products in the shopping basket 100 is managed in the server 116 so that the total price of the products is automatically calculated and the product that can be cleared at the unmanned cash register 118. A management system 140 (see FIG. 3) was adopted.

  As shown in FIG. 2B, the unmanned cash register 118 includes a display unit 142, an IC reader 144, a weighing table 146, and a wired communication unit 148 as an example. The display unit 142 displays, for example, the total amount of items in the shopping basket 100, the number of items, and the like. The IC reader 144 reads a car ID (described later) and the like. The weighing platform 146 checks the weight of the entire shopping basket 100. The wired communication unit 148 performs wired communication with the server 116. The unmanned cash register 118 has a bill slot 150a, a coin slot 150b, and a card slot 150c for a credit card, and the shopper 132 performs settlement using these, and when the settlement is completed, a receipt is printed from the receipt printer 152. The

  The unmanned cash register 118 further includes a speaker 154 for performing appropriate voice guidance and a button 156 for calling a store clerk. For example, if there is a question about the total price or the number of items displayed on the display unit 142, the shopper 132 can call the store clerk by pressing the button 156. Further, in the unattended cash register 118, the gate 158 opens when the liquidation is completed. After passing through the gate 154, the shopper 132 packs a product in a bag in the bagging space 160, for example, and then exits the store from the exit 162.

  Hereinafter, the merchandise management system 140 will be described. FIG. 3 is a functional block diagram illustrating the configuration of the product management system 140 according to this embodiment. FIGS. 3A and 3B are functional block diagrams illustrating configurations of the shopping basket 100 and the server 116, respectively.

  As shown in FIG. 3A, the communication device 104 of the shopping basket 100 includes a control unit 164 that controls these in addition to the imaging unit 106, the wireless communication unit 108, and the lamp 110. As illustrated in FIG. 3B, the server 116 includes an object recognition unit 166, an image recognition unit 168, a control unit 170, and a memory 172. The memory 172 includes a product master table 174 and an in-car product table 176. Further, the server 116 includes a wireless communication unit 178 that performs wireless communication with the shopping basket 100 and a wired communication unit 180. The wired communication unit 180 performs wired communication with the unmanned cash register 118 via the communication network 182.

  The product master table 174 is a list table such as a product list in the supermarket 130, and stores product information, that is, product names, prices, weights, and the like. The in-car product table 176 is a table created for each shopping basket 100 by the control unit 170 of the server 116 referring to the product master table 174. Here, the server 116 receives the car ID included in the data transmitted from the wireless communication unit 108 of the shopping basket 100 via the wireless communication unit 178. The control unit 170 of the server 116 can create the in-car product table 176 for each shopping basket 100 using the received car ID.

  Next, the operation of the product management system 140 will be described. FIG. 4 is a flowchart showing each process of the merchandise management method in the present embodiment. Here, it is assumed that in the shopping cart 100, the image capturing unit 106 continues to capture moving images in the shopping cart 100, and the wireless communication unit 108 continues to transmit moving images to the server 116.

  First, when a product is inserted, the lamp 110 is turned off in the shopping basket 100 (step S100). Since the lamp 110 is turned off, it can be confirmed that the shopper 132 can only be paid at the manned cash register 120 instead of the unmanned cash register 118 at this stage. Next, the control unit 170 of the server 116 controls the object recognition unit 166 and the image recognition unit 168 to start object recognition processing and image recognition processing based on the moving image captured by the imaging unit 106 (step S102). .

  Here, the object recognition process (object recognition) is a process for recognizing the size and shape of the object. For example, the position and movement of the object in the shopping basket 100 based on the coordinates (position) of the vertex of the object. To figure out. The image recognition process is a process for identifying a product by grasping, for example, a feature point of an object. The object recognition process is lighter in arithmetic processing than the image recognition process, and the process is completed in a short time. Therefore, in the merchandise management system 140, the object recognition unit 166 performs a process of continuing to track the position of the merchandise until at least the merchandise recognition unit 168 identifies the merchandise.

  That is, after step S102, the control unit 170 of the server 116 first registers the object ID in the in-car product table 176 after the object recognition of the product by the object recognition unit 166 (step S104). Subsequently, the control unit 170 of the server 116 continues the image recognition process by the image recognition unit 168 while tracking the position of the object by the object recognition by the object recognition unit 166 (step S106).

  Next, the control unit 170 of the server 116 determines whether or not the product has been taken out from the shopping basket 100 before the image recognition of the product by the image recognition unit 168 is completed (step S108). The determination in step S108 can be performed based on the motion vector calculated by the control unit 170 based on the change in the position of the product due to the object recognition using the moving image. As an example, when the product disappears from the moving image with an upward motion vector, the control unit 170 can determine that the product has been taken out from the shopping basket 100.

  If the product has moved out of the shopping cart 100 in step S108 (Yes), the control unit 170 sets a return flag (described later) in the in-car product table 176, and interrupts image recognition for the object (step S110). . As described above, the object recognition unit 166 continues to track the position of the object until the image recognition is completed. On the other hand, since the product taken out from the shopping basket 100 is not a target for settlement, image recognition is not necessary. Therefore, when a product before completion of image recognition is taken out from the shopping basket 100 as in the process of step S110, by interrupting image recognition of this product, the load associated with image recognition can be reduced.

  On the other hand, if the product has not moved out of the shopping cart 100 in step S108 (No), the control unit 170 of the server 116 determines whether the image recognition for the object has been completed and the product has been identified (step S112). ). When the image recognition is completed (step S112, Yes), the control unit 170 registers the product information in the in-car product table 176 (step S114).

  Hereinafter, with reference to FIG. 5 and FIG. 6, the processing of each step will be specifically described. FIG. 5 is a view for explaining processing when the product 184 is put into the shopping basket 100 of FIG. Note that the left side of FIG. 5 shows a state where milk is introduced as the product 184. The right side of FIG. 5 shows how the product 184 is recognized as the object A, the coordinates of the vertices T1, T2, T3, and T4 are grasped and the object A is tracked. Further, a region B shown on the right side of FIG. 5 illustrates a two-dimensional image obtained from one imaging unit 106. FIG. 6 is a diagram showing the in-car product table 176 created when the product 184 is put into the shopping basket 100 of FIG.

  First, as shown in FIG. 5A, when a product 184 is put into the shopping cart 100, each vertex T1, T2, T3, T4 of the object A in the region B is obtained by the object recognition processing of the object recognition unit 166. The coordinates of are grasped. As shown in FIG. 6A, the control unit 170 registers these coordinates together with the object ID “A” in the in-car product table 176.

  The in-car product table 176 includes items of product name, price, weight, and return in addition to the items of object ID and coordinates. In FIG. 6A, the coordinates item shows only the coordinates (Xa, Ya) of the vertex T1, and the coordinates of the other vertices T2, T3, T4 are not shown. In the “return” item, a return flag is set when the product 184 is returned from the shopping basket 100 as in step S108. Here, since the product 184 is put in the shopping basket 100 and the image recognition of the product 132 has not been completed yet, the items of product name, price, weight, and return as product information are blank. Yes.

  Next, as shown in FIG. 5 (b), when the introduced product 184 moves to the bottom of the shopping basket 100, the object recognition unit 166 continues to perform object recognition processing, and the object A in the region B The coordinates of each vertex T1, T2, T3, T4 are grasped. Note that the object recognition unit 166 calculates a motion vector C based on a change in the position of the object A due to object recognition. Based on the motion vector C, the control unit 170 can grasp that the product 184 has moved toward the bottom of the shopping basket 100. In FIG. 6B, only the coordinates (Xb, Yb) are shown as the coordinates after the movement of the vertex T1 in the coordinate item. At this point, since the image recognition of the product 184 has not yet been completed, the items of product name, price, weight, and return are blank.

  Furthermore, as shown in FIG. 5C, when the image recognition of the product 184 located at the bottom of the shopping basket 100 is completed and the product 184 is identified, the control unit 170 refers to the product master table 174, In the in-car product table 176, product information of product name “XXX milk”, price “180 yen”, and weight “500 g” is registered. Since the product 184 is located in the shopping basket 100, the item to be returned is blank.

  Returning to FIG. 4, the description will be continued. When the product information is registered in the in-car product table 176 in step S112, the fact is transmitted from the wireless communication unit 178 of the server 116 to the shopping cart 100. Then, the control unit 164 of the shopping basket 100 receives the fact by the wireless communication unit 108 and turns on the lamp 110 (step S116). When the lamp 110 is turned on, the shopper 132 can confirm that the unmanned cash register 118 can be settled.

  Next, when the image recognition is not completed even if the image recognition is continued for a predetermined time in step S112 and the image recognition is impossible (No), the lamp 110 remains off and the process proceeds to step S118 (the image about the product due to the time-out). End recognition).

  The control unit 170 of the server 116 continues tracking by object recognition even after the product 184 is identified by the image recognition unit 168, and determines whether or not the product 184 has been taken out from the shopping basket 100 (step S118). When the product 184 moves out of the shopping cart 100 in step S118 (Yes), the control unit 170 proceeds to step S110 and sets a return flag in the in-car product table 176. The above steps will be specifically described below with reference to FIGS.

  FIG. 7 is a view for explaining processing when the product 184 is taken out from the shopping basket 100 of FIG. Note that the left side of FIG. 7 shows a state where milk as the product 184 moves on the bottom of the shopping basket 100 and is further taken out from the shopping basket 100. The right side of FIG. 7 shows how the product 184 is recognized as the object A, the coordinates of the vertices T1, T2, T3, and T4 in the region B are grasped and the object A is tracked. FIG. 8 is a diagram showing the in-car product table 176 created when the product 184 is taken out from the shopping basket 100 of FIG.

  First, as shown in FIG. 7A, when the product 184 moves on the bottom of the shopping basket 100, each vertex T1, T2, T3, T4 of the object A in the region B is detected by the object recognition processing of the object recognition unit 166. The coordinates of are grasped. The control unit 170 registers the coordinates after movement of the object A in the item of coordinates in the in-car product table 176. In FIG. 8A, only the coordinates (Xc, Yc) after the movement of the vertex T1 are shown in the coordinate item, and the coordinates of the other vertices T2, T3, T4 are not shown. Here, although the product 184 has moved into the shopping basket 100 but has not been removed from the shopping basket 100, each item of return is blank.

  Next, as shown in FIG. 7B, even when the product 184 that has moved to the bottom of the shopping basket 100 is to be taken out from the shopping basket 100, the object recognition unit 166 continues to perform object recognition processing. The coordinates of the vertices T1, T2, T3, and T4 of the object A in the region B are grasped. Here, the object recognition unit 166 calculates the motion vector D based on the change in the position of the object A due to the object recognition. Based on the motion vector D, the control unit 170 can grasp that the product has moved upward from the bottom of the shopping basket 100. In FIG. 8B, only the coordinates (Xd, Yd) are shown as coordinates after the movement of the vertex T1 in the coordinate item. At this point, although the product 184 is moving upward in the shopping basket 100, it has not been removed from the shopping basket 100. For this reason, each item of return in the in-car product table 176 is blank.

  Further, as shown in FIG. 7C, when the product 184 disappears from the moving image B with the upward motion vector D, the control unit 170 determines that the product 184 has been taken out from the shopping basket 100. And the control part 170 sets the return flag which shows that to the return item of the goods table 176 in a car, as shown in FIG.8 (c). At this time, instead of setting the return flag, the record itself may be deleted.

  Returning to FIG. 4, the description will be continued. When the product 184 has not moved out of the shopping cart 100 in step S118 (No), the control unit 170 determines whether or not the settlement at the unattended cash register 118 has been completed (step S120). When the liquidation is not completed in step S120 (No), the process returns to step S118, and when the liquidation is completed (Yes), the above processes are terminated. The above-described processes are executed for each product that has been put into the shopping cart 100. That is, if the lamp 110 is lit in step S116, it means that product information is registered in the in-car product table 176 for all products in the shopping cart 100. Accordingly, it is possible to smoothly guide the shopper 132 to an appropriate cash register.

  The unattended cash register 118 reads the car ID with the IC reader 144 at the time of settlement, and reads the in-car product table 176 corresponding to the car ID from the server 116 with the wired communication unit 148. Then, the unattended cash register 118 calculates the total amount of all the products in the shopping basket 100 and displays it on the display unit 142. The unmanned cash register 118 measures the total weight of the shopping basket 100 placed on the weighing table 146 and compares it with the total weight registered in the in-car product table 176. If the weights are almost the same, it can be liquidated. If they do not match, the settlement at the manned cash register 120 is guided.

  Note that when the settlement is completed, the in-car product table 176 on the server 116 may be reset. Further, the imaging by the imaging unit 106 may be continued until the shopper 132 returns the shopping basket 100.

  As described above, according to the product management system 140 in the present embodiment, object recognition is performed based on the video in the shopping basket 100 transmitted from the communication device 104, and image recognition is continued while tracking the position of the object. it can. Therefore, it is possible to secure the time required for product image recognition and more reliably identify the product.

  Further, the server 116 interrupts the image recognition when the product is taken out from the shopping basket 100 before the image recognition of the product by the image recognition unit 168 is completed, so that the load associated with the image recognition can be reduced.

  Further, the object recognition unit 166 of the server 116 continues the object recognition even after the product is identified. For this reason, when a product is taken out from the shopping basket 100, the product can be specified without performing image recognition. That is, when the product is taken out from the shopping basket 100, it is not necessary to recognize the image in an instant, so that the increase or decrease of the product in the shopping basket 100 can be properly grasped.

  In the above embodiment, the position of an object is tracked using a two-dimensional image obtained from one imaging unit 106. However, the present invention is not limited to this, and a three-dimensional image can be obtained using a plurality of video cameras. The position of the object may be tracked.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  The present invention can be used in a product management system and a product management method for managing products in a store such as a supermarket.

DESCRIPTION OF SYMBOLS 100 ... Shopping basket, 102 ... Side surface, 104 ... Communication apparatus, 106 ... Imaging part, 178 ... Wireless communication part, 110 ... Lamp, 112 ... Upper edge, 114 ... Shading net, 116 ... Server, 118 ... Unmanned cash register, DESCRIPTION OF SYMBOLS 120 ... Manned cash register, 130 ... Supermarket, 132 ... Shopper, 134 ... Entrance, 136a-136d ... Merchandise shelf, 140 ... Merchandise management system, 142 ... Display unit, 144 ... IC reader, 146 ... Weighing table, 148, 180 ... Wired communication section, 150a ... bill slot, 150b ... coin slot, 150c ... card slot, 152 ... receipt printer, 154 ... speaker, 156 ... button, 158 ... gate, 160 ... bagging space, 162 ... outlet, 164, 170 ... Control unit, 166 ... Object recognition unit, 168 ... Image recognition unit, 172 ... Memory, 174 ... Product master table , 176 ... basket in the product table, 182 ... communication network, 184 ... Products

Claims (7)

A product management system comprising a server that manages product information of a product, and a shopping device in which a communication device that performs wireless communication with the server is provided and a product is put in or taken out by a shopper,
The communication device is:
An imaging unit for imaging a video in the shopping basket;
A wireless communication unit that transmits the moving image captured by the imaging unit to the server;
The server
An object recognition unit that performs object recognition of a product put in the shopping basket based on the video and tracks the position of the product;
An image recognition unit for recognizing the product based on the video and identifying the product;
The product management system, wherein the object recognition unit keeps tracking the position of the product until at least the product is identified by the image recognition unit.   2. The product according to claim 1, further comprising a control unit that interrupts image recognition when a product is taken out of the shopping basket before completion of image recognition of the product by the image recognition unit. Management system. The server further includes a product master table storing the product information, and a car product table created by the control unit for each shopping basket with reference to the product master table,
The controller is
Register the object ID in the product table in the car after the object recognition of the product,
3. The product management system according to claim 2, wherein when the product is identified by image recognition, the product information of the product is read from the product master table, and the product information is registered in the in-car product table. .   The product management system according to claim 3, wherein the communication device further includes a lamp that is turned on when the product information is registered in the in-car product table for all products in the shopping cart.   The control unit calculates a motion vector based on a change in the position of the product by object recognition using a moving image, and determines whether the product has been taken out from the shopping basket based on the motion vector. The merchandise management system according to any one of claims 2 to 4, wherein the merchandise management system is characterized.   The product management system according to claim 1, wherein the object recognition unit continues object recognition even after the product is identified. A product management method using a shopping cart in which a server that manages product information of a product and a communication device that performs wireless communication with the server is provided and a product is input or removed by a shopper,
The communication device captures a moving image in the shopping basket and transmits the moving image to the server.
The server
Based on the video, performs object recognition and image recognition of the product put in the shopping basket,
A merchandise management method characterized by continuing to track the position of the merchandise by the object recognition until at least the merchandise is identified by the image recognition.

Images