TWI847642B - Intelligent commodity supply monitoring system - Google Patents

Abstract

An intelligent commodity supply monitoring system includes a database, a camera module, a computing module, and a communication module. The database stores a plurality of commodity storage area features and commodity features. The camera module captures the display and sales area and generates the image thereof. The computing module marks a plurality of the commodity storage area images and the corresponding commodity images on the display and sales area images. When there are no commodity images displayed in the commodity storage area image, according to the commodity storage area images having no commodity images, the computing module obtains the matching commodity storage area features and the commodity features from the database, and generates a replenishment signal. The communication module sends the replenishment signal, the commodity display and sales area images, and the corresponding commodity features obtained from the database to the message interface of a social communication platform, such that the supply of commodities for display and sale is effectively controlled, allowing the staff to timely replenish commodities.

Description

Intelligent commodity supply monitoring system

The present invention relates to a monitoring system, and more particularly to an intelligent commodity supply monitoring system capable of real-time monitoring of commodity display supply.

Currently, shopping malls or stores sell a variety of goods, and salespeople need to manually display each type of goods at the corresponding location on the shelf. In order to ensure the supply of goods, salespeople will manually check the shelves regularly to see the current supply status of each product.

However, each product has a different sales speed. When a product has been sold out and it is not yet time for the store clerk to restock it, the product will be unsaleable. If a consumer needs to buy this product, the consumer needs to find a store clerk to ask whether the product is in stock. If the clerk cannot be found immediately, the consumer will have a bad shopping experience. Therefore, how to provide consumers with a pleasant shopping experience is the direction considered by the relevant technical field.

To solve the above problems, the present invention provides an intelligent commodity supply monitoring system, which effectively controls the supply of commodities for display and sale through image monitoring, and can timely notify the store staff to replenish the commodities when the supply of commodities is insufficient.

An embodiment of the present invention provides an intelligent commodity supply quantity monitoring system, which includes: a database, which stores a plurality of storage area features and a commodity feature set in each storage area feature; a camera module, which can shoot the commodity display area to generate an image of the display area; a computing module, which is coupled to the database and the camera module, the computing module receives the display area image transmitted by the camera module, and performs an image detection process on the display area image to mark the plurality of storage area images and a commodity image in each storage area image in the display area image, wherein when the computing module determines that there is any A product image is not displayed in a storage area image, and the computing module obtains one of the matching storage area features and the corresponding product features from the database according to the storage area image corresponding to the non-appearing product image, and the computing module generates a replenishment signal; and a communication module is coupled with the computing module and a social communication platform, the communication module receives the replenishment signal transmitted by the computing module, the display area image and the corresponding product features obtained from the database, and the communication module transmits the replenishment signal, the display area image and the corresponding product features obtained from the database to the message interface of the social communication platform through a service communication technology.

Through the above, the present invention utilizes image monitoring and image processing to confirm whether the supply of goods on display is sufficient. When the supply of goods is insufficient, a replenishment signal is generated, and the store clerk is notified to replenish the goods through service communication technology. In this way, insufficient supply of goods can be avoided, which may cause trouble to consumers during consumption, and the supply of goods can be accurately and efficiently monitored.

In order to facilitate the description of the central idea of the present invention in the above invention content column, specific embodiments are used for illustration. Various objects in the embodiments are depicted according to the proportions, sizes, deformations or displacements suitable for the description, rather than being drawn according to the proportions of the actual components, which should be noted in advance.

Referring to FIGS. 1 to 6 , the present invention provides an intelligent commodity supply quantity monitoring system 100, which comprises:

A database 10 stores a plurality of storage area features and a product feature set in each storage area feature; wherein each storage area feature and each product feature are established in the database 10 through computer vision combined with deep learning.

It should be particularly noted that the characteristics of each storage area and each product established in the database 10 are images obtained by photographing the actual storage area A1 and the actual product P1, and each storage area characteristic and each product characteristic are captured in an initial state and then established in the database 10 through computer vision (Computer Vision) combined with deep learning (Deep Learning); wherein, the relationship between each storage area characteristic and the product characteristic can be established in the database 10 by the user (store clerk) using a setting method to establish each product characteristic corresponding to each storage area characteristic; or after the actual storage area A1 and the actual product P1 are placed, they are acquired and established in the database 10 through image capture and processing. The present invention does not limit the establishment method of each storage area characteristic and each product characteristic.

A setting module 20 is coupled to the database 10. The setting module 20 can capture the characteristics of each storage area of the database 10 and provide the setting of each storage area characteristic as an urgent replenishment area U1 or a normal replenishment area U2. The setting module 20 stores the set characteristics of each storage area back to the database 10, wherein the urgent replenishment area U1 is a storage area for commodities that need immediate replenishment, and the normal replenishment area U2 is a storage area for commodities that can be replenished on a regular basis (for example, every few hours/every day/every few days/weekly/monthly, etc.). The user (store clerk) can set it according to the needs.

A camera module 30 is provided, which can photograph the merchandise display area C to generate an image E of the display area. It should be noted that a plurality of camera modules 30 can be installed in a merchandise display area C, and the amount of merchandise display area C that each camera module 30 can capture can be set by the user (store clerk). A single camera module 30 will be used for explanation below.

A computing module 40 is coupled to the database 10 and the camera module 30. The computing module 40 receives the display area image E transmitted by the camera module 30. The computing module 40 performs an image detection process on the display area image E to mark a plurality of storage area images A2 and a product image P2 in each storage area image A2 in the display area image E. The computing module 40 can determine whether there is a product image P2 in each storage area image A2 (for determining whether the supply of the product is insufficient). The computing module 40 performs image detection processing and marks the edges of each storage area image A2 with a plurality of marked frames F, and performs image color block processing on each marked frame F to determine whether there is a color change of different color blocks in each marked frame F. In the embodiment of the present invention, the computing module 40 presents each marked frame F of each storage area image A2 belonging to the urgent replenishment area U1 and each storage area image A2 belonging to the normal replenishment area U2 in different states. The image detection processing is Canny edge detection.

Please refer to FIG. 1 to FIG. 3 . When the calculation module 40 determines that each frame F has a plurality of color blocks with color changes, that is, each storage area image A2 displays a product image P2, it means that the actual supply of the product P1 is sufficient.

Please refer to Figures 1, 4 to 5. When the computing module 40 determines that there is a color change of the non-color block in any frame F (indicating that the bottom of the actual storage area A1 has been exposed), that is, the product image P2 is not displayed in any storage area image A2 (indicating that the supply of the actual product P1 is insufficient), at this time, the computing module 40 will obtain the matching storage area characteristics and the corresponding product characteristics from the database 10 according to the storage area image A2 corresponding to the non-product image P2, and the computing module 40 generates a replenishment signal.

In other embodiments of the present invention, the computing module 40 is provided with a replenishment threshold, and the replenishment threshold is when the color of a single color block has filled 50% to 90% of the frame F. The present invention does not limit the replenishment threshold, and the user can set it according to demand. When the computing module 40 determines that there is a color change of a color block in any frame F, and any part of the color change of the color block is a single color, the computing module 40 determines that any product image P2 does not fill the storage area image A2 (indicating that the actual supply of the product P1 is close to insufficient), and generates a replenishment signal.

Furthermore, the computing module 40 can generate different replenishment signals according to the urgent replenishment area U1 or the normal replenishment area U2, wherein the replenishment signal is divided into an immediate replenishment signal and a periodic replenishment signal; the computing module 40 obtains one of the matching storage area features from the database 10 according to the storage area image A2 corresponding to the product image P2 that does not appear, and obtains the matching storage area feature as the urgent replenishment area U1 or the normal replenishment area U2, and the computing module 40 can generate an immediate replenishment signal corresponding to the urgent replenishment area U1; or generate a periodic replenishment signal corresponding to the normal replenishment area U2.

A communication module 50 is coupled to the computing module 40 and a social communication platform. The communication module 50 receives the replenishment signal, the display area image E and the corresponding product features obtained from the database 10 transmitted by the computing module 40. The communication module 50 transmits the replenishment signal, the display area image E and the corresponding product features obtained from the database 10 to the message interface of the social communication platform through a service communication technology. Among them, the reminder signal can be a text message, for example: XX product supply is insufficient; in the embodiment of the present invention, the social communication platform is LINE; the service communication technology is LINE Notify.

Please refer to FIG. 1 and FIG. 6 . When a user (store clerk) receives a replenishment signal, a sales area image E, and corresponding product features on the social communication platform of the terminal device D, and learns from the replenishment signal and the sales area image E that the supply of the actual product P1 in the actual storage area A1 is insufficient, and learns from the product features which actual product P1 needs to be replenished, the user (store clerk) can then take the products to be replenished to the actual storage area A1 to replenish the actual product P1.

Furthermore, the communication module 50 is provided with a transmission time. The communication module 50 transmits the replenishment signal, the display area image E and the corresponding product features to the message interface of the social communication platform at intervals of the transmission time. The transmission time is divided into a first transmission time and a second transmission time, and the first transmission time is shorter than the second transmission time. The immediate replenishment signal corresponds to the first transmission time, and the regular replenishment signal corresponds to the second transmission time. For example, the first transmission time can be received and the immediate transmission time can be adjusted according to demand, but the present invention is not limited to this.

It should be particularly noted that the database 10, the setting module 20, the calculation module 40 and the communication module 50 can be integrated into software or an application and installed on the terminal device D, wherein the user (employee) can use the setting module 20 through the terminal device D to set the characteristics of each storage area as an urgent replenishment area U1 or a normal replenishment area U2.

In summary, the intelligent product supply monitoring system 100 of the present invention utilizes image monitoring and image processing to confirm whether the supply of the displayed products is sufficient. When the supply of the products is insufficient, a replenishment signal is generated, and the store clerk is notified to replenish the products through service communication technology. In this way, insufficient product supply can be avoided, which may cause troubles to consumers during consumption, and the product supply can be monitored accurately and efficiently.

The above embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. Any modifications or changes that do not violate the spirit of the present invention are within the scope of protection of the present invention.

100: Intelligent product supply monitoring system 10: Database 20: Setting module 30: Photographic module 40: Computing module 50: Communication module A1: Actual storage area A2: Storage area image P1: Actual product P2: Product image U1: Urgent replenishment area U2: Normal replenishment area C: Product display area E: Display area image F: Marking frame D: Terminal device

Figure 1 is a schematic diagram of the structural block of the present invention. Figure 2 is a schematic diagram of the present invention's camera module set up in the commodity exhibition area (I), indicating that the supply of each commodity is sufficient. Figure 3 is a schematic diagram of the present invention's display area image implementation example (I), indicating that the supply of each commodity is sufficient. Figure 4 is a schematic diagram of the present invention's camera module set up in the commodity exhibition area (II), indicating that the supply of one of the commodities is insufficient. Figure 5 is a schematic diagram of the present invention's display area image implementation example (II), indicating that the supply of one of the commodities is insufficient. Figure 6 is a schematic diagram of the present invention's store clerk receiving a replenishment signal to replenish the goods.

100: Intelligent commodity supply monitoring system

10: Database

20: Setting module

30: Photography module

40: Computation module

50: Communication module

D: Terminal device

Claims (5)

An intelligent commodity supply quantity monitoring system includes: a database storing a plurality of storage area characteristics and a commodity characteristic set in each storage area characteristic; a photography module capable of photographing a commodity display area to generate an image of the display area; a setting module coupled to the database, the setting module capable of extracting each storage area characteristic of the database and providing a method of setting each storage area characteristic as an urgent replenishment area or a normal replenishment area, the setting module storing each storage area characteristic that has been set back to the database; an operation module coupled to the database and the photography module, the operation module having a replenishment threshold, the operation module receiving the information transmitted by the photography module, The computing module performs an image detection process on the image of the sales area, and uses a plurality of marking frames to mark along the edges of each of the storage area images, so as to mark a plurality of storage area images and a product image in each storage area image in the sales area image. When the computing module determines that the product image is not displayed in any of the storage area images, the computing module obtains one of the matching storage area features and the corresponding product features from the database according to the storage area image corresponding to the storage area image in which the product image does not appear, and the computing module generates a replenishment signal. The computing module performs image color block processing on each of the marking frames to determine the product image in each of the marking frames. Whether there is a color change of different color blocks, when the calculation module determines that there is no color change of the color block in any of the marked frames, it is determined that the product image is not displayed in the storage area image, and the calculation module determines that there is a color change of the color block in any of the marked frames, and any part of the color change of the color block is a single color, and the single color color block area in the marked frame has reached the replenishment threshold, then the calculation module determines that any of the product images does not fill the storage area image, so as to generate the replenishment signal; the replenishment signal is divided into an immediate replenishment signal and a periodic replenishment signal; the calculation module obtains the matching other from the database according to the storage area image corresponding to the non-appearance of the product image The computing module generates the immediate replenishment signal corresponding to the urgent replenishment area; the computing module generates the regular replenishment signal corresponding to the regular replenishment area; and a communication module is coupled with the computing module and a social communication platform. The communication module receives the replenishment signal, the display area image and the corresponding product characteristics obtained from the database sent by the computing module. The communication module transmits the replenishment signal, the display area image and the corresponding product characteristics obtained from the database to the message interface of the social communication platform through a service communication technology. As described in claim 1, the intelligent commodity supply monitoring system, wherein the image detection processing is Canny edge detection. As described in claim 1, the intelligent commodity supply quantity monitoring system, wherein the computing module presents the marked frames of each of the storage area images belonging to the urgent replenishment area and each of the storage area images belonging to the normal replenishment area in different forms. As described in claim 1, the intelligent commodity supply quantity monitoring system, wherein the communication module is provided with a transmission time, and the communication module transmits the replenishment signal, the display area image and the corresponding commodity features obtained from the database to the message interface of the social communication platform at intervals of the transmission time. As described in claim 4, the intelligent commodity supply quantity monitoring system, wherein the transmission time is divided into a first transmission time and a second transmission time, the first transmission time is shorter than the second transmission time; the immediate replenishment signal corresponds to the first transmission time, and the periodic replenishment signal corresponds to the second transmission time.