JP7704414B2 - Self-checkout machine - Google Patents

Description

The present invention relates to a self-checkout device that allows customers to pay for products they purchase themselves.

Self-checkout machines equipped with a scanner that allows customers to scan product information to pay for the products they purchase have been known for some time.

For example, Patent Document 1 describes the invention of a self-checkout terminal as follows. The self-checkout terminal described in Patent Document 1 is equipped with a display, a barcode scanner, and the like. It is also described that a table for temporarily placing products on it and a shopping basket stand are provided. It is also described that a coin insertion slot, a bill receipt/receive slot, a coin refund slot, and a coin withdrawal slot are provided below the table.

In the self-checkout terminal described in Patent Document 1, the customer registers products with barcodes attached by operating a barcode scanner or the like, and the self-checkout terminal registers the products as items to be paid for based on the information read.

JP 2015-153324 A

In the self-checkout terminal described in Patent Document 1, the display is set up so as to face the customer (user), and furthermore, the table, coin slot, etc. are set up below the display. Therefore, when a customer is making a payment, the customer has to move their eyes significantly between the display showing the payment method and the table and coin slot where the products are scanned and cash is inserted, which can lead to operational errors.

The present invention aims to solve the above problems by providing a self-checkout device that reduces operational errors and improves user convenience.

The invention described in claim 1, which has been made to solve the above problem, is a self-checkout device that includes a first display surface that is horizontally arranged, and a detection unit that is arranged outside the display area of the first display surface and detects operations on an operation unit displayed on the first display surface, and is characterized in that an operation guide for the user is displayed on the first display surface.

As described above, according to the present invention, an operation guide for the user is displayed on a horizontally arranged first display surface, and a detection unit that detects operations on the operation unit displayed on the first display surface is provided outside the display area of the first display surface. Therefore, it is possible to use the horizontal first display surface as a table on which products are placed, and by displaying the operation guide on the table, the operation guide can be viewed while keeping the line of sight on the product, with almost no movement of the line of sight. This reduces operation errors and improves user convenience.

1 is a schematic configuration diagram of a self-checkout device according to one embodiment of the present invention, seen from the side; 2 is a schematic configuration diagram of the self-checkout device shown in FIG. 1 as seen from above. FIG. 1 is a diagram showing an example of how a self-checkout device is used. FIG. 13 is a diagram (part 1) showing changes in the operation guide during a purchasing operation. FIG. 2 is a diagram showing changes in the operation guide during a purchase operation (part 2). FIG. 1 is an explanatory diagram of commodity detection by a video camera. FIG. 1 is an explanatory diagram of commodity detection by a motion sensor.

The self-checkout device according to one embodiment of the present invention will be described with reference to Figs. 1 to 5. Fig. 1 is a schematic diagram of the self-checkout device 1 according to one embodiment of the present invention as viewed from the side. Fig. 2 is a schematic diagram of the self-checkout device 1 as viewed from above.

The self-checkout device 1 includes a housing 2, monitors 3 and 4, a facial recognition camera 5, a video camera 6, a scanner 7, a printer 8, a motion sensor 9, a projector 10, a payment terminal 11, an automatic change dispenser 12, a PC 13, and a SUBPC 14.

The housing 2 is composed of a box-shaped first housing section 2a and a flat second housing section 2b standing on the top surface of the first housing section. The first housing section 2a has a monitor 3 and a payment terminal 11 on the top surface section 2a1. The top surface section 2a1 also has a banknote insertion port 12a1, a banknote withdrawal port 12a2, a coin insertion port 12b1, a coin withdrawal port 12b2, a coin return port 12b3, etc. (see Figures 4 and 5) that are provided in the automatic change dispenser 12.

The monitor 3 is provided on the upper surface 2a1 of the housing 2. That is, the monitor 3 functions as a first display surface provided horizontally. In this embodiment, the monitor 3 is configured as a screen for the projector 10. As described below, the monitor 3 is strong enough to be used as a table or the like to be used during checkout operations such as scanning products, in addition to being a display surface. Therefore, since the monitor 3 is used as a table or the like, it does not function as a touch panel itself, as there is a risk of malfunction due to products or the like placed on it. The function replacing the touch panel is realized by using a motion sensor 9, as described below.

In addition, the monitor 3 displays an operation guide for using the self-checkout device 1. The operation guide will be described later.

The payment terminal 11 is a terminal device that handles payment functions such as credit cards and electronic money.

In addition, an automatic change dispenser 12, a PC 13, and a SUBPC 14 are provided within the first housing unit 2a.

The automatic change machine 12 is composed of a banknote handling device 12a and a coin handling device 12b (see FIG. 2), and performs cash settlement under the control of the PC 13. As shown in FIG. 4 and FIG. 5 described later, the automatic change machine 12 also has a banknote input port 12a1, a banknote withdrawal port 12a2, a coin input port 12b1, a coin withdrawal port 12b2, a coin return port 12b3, etc. In other words, the automatic change machine 12 has a cash input port and a change discharge port.

The PC 13 is configured as a small computer and performs all the settlement processing for purchasing products. In other words, the PC 13 can function as a cash register with a POS (Point of Sale) function by using a built-in POS application program, etc.

SUBPC14 is configured as a small computer and performs processes such as automatic customer recognition, automatic motion analysis, and voice recognition. SUBPC14 is configured as a separate PC (computer) from PC13, but they may be combined into a single PC. Separating PC13 and SUBPC14 makes it easy to add processes such as automatic customer recognition, automatic motion analysis, and voice recognition to an existing system.

Automatic customer recognition identifies the gender and age of customers from facial images captured by the face recognition camera 5 using well-known methods. Automatic motion analysis recognizes the movements (face direction, hand movements, etc.) of customers (users) captured by the video camera 6 using well-known methods. Voice recognition recognizes the voices uttered by customers. The results of these automatic customer recognition, automatic motion analysis, and voice recognition processes are sent to the robot assistant processing unit 22, which will be described later.

The second housing 2b is provided with a monitor 4, a face recognition camera 5, a video camera 6, a scanner 7, a printer 8, a motion sensor 9, and a projector 10 (see also FIG. 2). In other words, these devices are provided outside the display area of the monitor 3.

Monitor 4 is installed approximately vertically so that the display surface faces directly (faces) customer U. That is, monitor 4 is installed above monitor 3 (first display surface) and facing customer U (user). In this embodiment, monitor 4 is configured as a liquid crystal display. Furthermore, monitor 4 does not have a function as a touch panel. Furthermore, monitor 4 displays a human image (human-shaped guide image) that serves as a robot assistant, either live-action or created using CG (Computer Graphics), etc. The specific content displayed on monitor 4 will be described later.

The facial recognition camera 5 is a camera for capturing a facial image of the customer U to recognize the gender and age, and is a camera module having a CCD (Charge Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

The video camera 6 has a video camera for recognizing the actions of the customer U and a microphone for recognizing the voice emitted by the customer U. The video camera 6 does not necessarily have to be configured with the video camera and the microphone integrated, and may be separate.

The scanner 7 irradiates the product with a laser scanning line or uses a CCD scanner to read various codes such as barcodes attached to the product. The scanner 7 can also scan cards with barcodes attached, such as membership cards and point cards.

The printer 8 prints a receipt containing information such as the name of the purchased product, the unit price of the product, the deposit amount, and the change amount.

The motion sensor 9 detects the shape and movement direction of the customer's fingers. In this embodiment, the motion sensor 9 is composed of a camera, and detects the movement of the fingers by performing image analysis on the captured image. In other words, the motion sensor 9 functions as a detection unit that detects operations on buttons and the like (operation unit) displayed on the monitor 3 (first display surface).

The SUBPC 14 determines which of the images (operation units) such as buttons displayed (projected) on the monitor 3 has been operated based on the detection result of the motion sensor 9. Then, based on the determination result, the SUBPC 14 executes a process (such as settlement) corresponding to the operation, changes the image displayed on the monitor 3, or changes the video displayed on the monitor 4.

In addition, in this embodiment, when the PC 13 determines based on the identified fingertip position that the fingertip has come within, for example, 3 cm of the surface of the monitor 3, a mark is displayed at the position on the monitor 3 where the fingertip is presumed to be pointing. Then, when it determines that the fingertip has come within, for example, 1 cm of the surface of the monitor 3, the PC 13 determines that the button at that position has been touched (operated). These distance and other determinations can be made based on the identification results of the motion sensor 9.

That is, when the fingertip approaches within 3 cm (within a second distance that is longer than the first distance) of the surface of the monitor 3 (first display surface), the PC 13 (determination unit) displays a mark at a position on the operation guide where the fingertip is estimated to be pointing. Also, when the fingertip approaches within 1 cm (within the first distance) of the surface of the monitor 3 (first display surface), the PC 13 (determination unit) determines the operation of the operation guide.

The projector 10 projects an operation guide onto the monitor 3 under the control of the PC 13. That is, the projector 10 functions as a projection unit that projects an operation guide onto the monitor 3 (first display surface). Note that the scanner 7, printer 8, motion sensor 9, and projector 10 are not limited to being installed in the second housing unit 2b, and may be installed in the first housing unit 2a (excluding the projection portion on the top surface unit 2a1).

The cloud service 20 is composed of a server device and the like connected to the self-checkout device 1 via a network such as the Internet. In other words, the self-checkout device 1 (PC 13 and SUBPC 14) has a communication function. The cloud service 20 has POSDATA 21 and a robot assistant processing unit 22.

POSDATA21 stores sales data and other data collected by multiple POS terminals, including the self-checkout device 1.

The robot assistant processing unit 22 predicts the next action of the customer U from the learned scenario based on the results of the automatic customer recognition, automatic action analysis, and voice recognition sent from the SUBPC 14, and sends it to the SUBPC 14. The SUBPC 14 displays an image on the monitor 4 based on the received predicted action.

Next, the operation guide displayed on monitor 3 and the robot assistant image displayed on monitor 4 will be described with reference to Figs. 3 to 5. Fig. 3 is a diagram showing an image of how the self-checkout device 1 is used. In Fig. 3, some of the devices shown in Figs. 1 and 2, such as the scanner 7 and printer 8, are omitted from the illustration.

As shown in FIG. 3, when a customer U purchases a product P, such as a book, a robot assistant image R is displayed on monitor 4. In addition, an operation guide G is displayed on monitor 3. The customer U performs the payment procedure by following the operation guide G displayed on monitor 3, and can receive assistance from the robot assistant image R as necessary.

Figures 4 and 5 show how the operation guide G changes during a purchase operation. First, in the standby state, the SUBPC 14 detects that the customer U is approaching the self-checkout device 1 using the face recognition camera 5 or the like, and the PC 13 receives the detection result from the SUBPC 14 and displays on the monitor 3 text, images, etc., such as a greeting and guidance for scanning the product P.

Next, as the product scanning state, the PC 13 displays a screen on the monitor 3 explaining how to scan. At this time, if the user wishes to proceed to payment, a button or the like indicating payment is displayed, and the button can be touched. As described above, the touch operation is performed by detecting the position of fingers, etc., using the motion sensor 9. Therefore, it is acceptable to place products, etc. on parts of the monitor 3 other than the buttons.

Next, to select a payment method, the PC 13 displays a screen on the monitor 3 explaining the payment methods. In FIG. 4, there are multiple options for the payment method, but if there is no option, such as cash only, this display may be omitted and the screen may transition to the next screen.

Next, when cash is selected as the payment method, a screen explaining how to insert cash is displayed on monitor 3 (Figure 5). In the payment screen of Figure 5, the actual positions of coin slot 12b1 and bill slot 12a1 are indicated using arrows and text information as operation guide G. When viewed from above, coin slot 12b1 and bill slot 12a1 are located on the same plane as monitor 3 (located on the periphery of monitor 3). In other words, operation guide G displays arrows and text information indicating the position of the device to be operated. In this embodiment, operating a device also includes actions such as inserting coins, which cause the target device to perform some kind of operation.

Next, to complete the settlement process, arrows are used as operation guide G to indicate the positions of the actual receipt printer (printer 8), banknote withdrawal port 12a2, and coin withdrawal port 12b2. The receipt printer and scanner 7 are also located on the same plane as the monitor 3 when viewed from above (they are located at the end of the monitor 3).

The robot assistant image R can be, for example, a live-action image of a person (CG is also acceptable, but live-action images are preferable as they provide a more familiar appearance). A number of different types of person images are recorded in advance according to response patterns. The robot assistant image R can be switched to a person's image (gender, age) corresponding to the customer U recognized from the image captured by the face recognition camera 5. The robot assistant image R also assists the customer U to perform a smooth payment process based on the customer U's actions recognized from the image captured by the video camera 6 and the customer U's speech recognized from the voice picked up by the microphone of the video camera 6.

Specifically, for example, if customer U moves too quickly, the system can give guidance such as "Please operate slowly." Also, for example, if it takes time to select a payment method, the system can give guidance such as "Are you paying in cash?", and if customer U responds in the affirmative, such as "Yes," the system can give guidance such as "Please touch the area that says cash." Alternatively, if customer U is having difficulty operating the system, the system can give guidance such as "Shall I page an operator?" Then, if customer U responds in the affirmative, such as "Yes," the image on monitor 4 can be switched to an operator. After that, the system can communicate with a real operator via videophone.

According to the present embodiment described above, the self-checkout device 1 includes a horizontally installed monitor 3 and a motion sensor 9 that is installed outside the display area of the monitor 3 and detects operations on the operation guide G displayed on the monitor 3, and the operation guide G for the customer U is displayed on the monitor 3. In this way, since the operation guide G for the customer U is displayed on the horizontally installed monitor 3, it is also possible to use the horizontal monitor 3 as a table on which the product P is placed, and by displaying the operation guide G on the table, the operation guide G can be viewed while keeping the line of sight fixed on the product P, with almost no movement of the line of sight. This reduces operation errors and improves user convenience.

In addition, the operation guide G uses arrows and text information to indicate the location of the equipment to be used (operated), such as the coin insertion slot 12b1 and the banknote insertion slot 12a1, so that the user can intuitively understand which equipment to use, improving convenience.

The devices to be used (operated) include a scanner 7 that reads product information from the product to be purchased, a printer 8, a banknote insertion slot 12a1, a banknote withdrawal slot 12a2, a coin insertion slot 12b1, a coin withdrawal slot 12b2, and a coin return slot 12b3, which are installed on the same plane as the monitor 3 when viewed from above. This makes it easy to indicate each of the above-mentioned devices using arrows or the like displayed on the monitor 3, and allows the user to be guided in an easy-to-understand manner.

In addition, since the device is equipped with a projector 10 that projects an operation guide G onto the monitor 3, the monitor 3 can be configured as a screen, making it easy to double as a table on which products can be placed.

In addition, monitor 4 is provided above monitor 3 and facing customer U, and monitor 4 displays a human-shaped guide image, so that even though it is a self-checkout, it can provide a sense of familiarity similar to that of a face-to-face checkout where a cashier is in charge. In this way, the human-shaped guide image can be considered a substitute for a cashier, and it is desirable to display it in front of customer U. Therefore, by installing a separate monitor 4 rather than displaying it on monitor 3, convenience for customer U can be improved.

The self-checkout device 1 also includes a motion sensor 9 that detects the position of the hand or finger operating the monitor 3, and a PC 13 that determines the operation of the operation guide G based on the detection result of the motion sensor 9. In this way, it is possible to achieve the same functionality as a touch panel without requiring a touch panel. Therefore, the monitor 3 can also serve as a table.

In addition, the PC 13 determines whether or not the user is operating the operation guide G when the fingertip approaches, for example, within 1 cm of the surface of the monitor 3, so it can determine whether or not the user is operating the operation guide G immediately before the finger touches the screen. In addition, the user can experience the same operational sensation as with a touch panel.

In addition, when a fingertip approaches, for example, within 3 cm of the surface of the monitor 3, the PC 13 displays a mark at the position on the monitor 3 where the fingertip is estimated to be pointing, so that the customer U, etc. can be informed of where the finger is about to point.

In the above embodiment, a monitor 4 is provided, but this is not essential. If a monitor 4 is not provided, the customer U may be assisted by outputting only audio instead of the robot assistant image R.

In addition, the monitor 3 does not have to be configured as a screen for the projector 10. For example, the monitor 3 can be configured as a liquid crystal display and a strong transparent cover can be provided over the liquid crystal display, allowing it to be used as a table on which merchandise, etc. can be placed.

The operation guide G may also be displayed while avoiding the products placed on the monitor 3. That is, the operation guide G is displayed while avoiding the products on the first display surface. This can prevent the operation guide G from being hidden by the products or being reflected by the products when displayed using a projector or the like, making the display difficult to see. The operation guide G may be shifted up, down, left, or right, or may be displayed in a reduced size, or multiple display patterns of the operation guide G may be prepared depending on the product position.

In order to display the operation guide G while avoiding the product, it is necessary to detect the product on the monitor 3. An example of a method for detecting a product will be described with reference to Figs. 6 and 7. In Figs. 6 and 7, a video camera 6 and a motion sensor 9 are used. The video camera 9 is capable of capturing images within the range of area A1 indicated by the dashed line extending from the video camera 6 in Fig. 6. Therefore, the position of the product can be detected by recognizing the product with the video camera 6. The product may be recognized by learning the object shape, but may also be detected by detecting at least a non-flat part.

In addition, by using the motion sensor 9 in addition to the video camera 6, the accuracy of product detection can be improved. The motion sensor 9 can recognize the range of area A2, which is an area including each device of the self-checkout device 1 (monitor 3, banknote slot 12a1, etc.) below the dashed line extending from the motion sensor 9 in FIG. 7. As described above, the motion sensor 9 can recognize which position on the monitor 3 has been operated by finger movements, and can also recognize how the product has moved, such as being placed on the monitor 3 from a shopping basket, etc. Therefore, by using the detection results of the motion sensor 9, the position of the product can be recognized with high accuracy. Then, the operation guide G can be displayed avoiding the recognized product.

Furthermore, the present invention is not limited to the above-described embodiment. In other words, a person skilled in the art can implement various modifications according to conventional knowledge without departing from the gist of the present invention. As long as such modifications still have the configuration of the self-checkout device of the present invention, they are of course included in the scope of the present invention.

1 Self-checkout device 3 Monitor (first display surface)
4 Monitor (second display surface)
7. Scanner (equipment to be operated)
8. Printer (receipt printer)
9. Motion sensor (detection part)
10 Projector (projection unit)
12 Automatic change machine 12a1 Bill insertion slot (equipment to be operated)
12a2 Banknote dispensing port (equipment to be operated)
12b1 Coin slot (equipment to be operated)
12b2 Coin dispensing port (equipment to be operated)
12b3 Coin return slot (equipment to be operated)
13 PC (judgment section)
14. SUBPC
G Operation guide R Robot assistant image (human-shaped guide image)
U Customer (user)

Claims (8)

A first display surface provided horizontally;
a detection unit that is provided outside a display area of the first display surface and detects an operation on an operation unit displayed on the first display surface,
An operation guide for a user is displayed on the first display surface.
the operation guide includes a display showing a position of a device to be operated by an arrow and text information,
A self-checkout device characterized in that the device to be operated is installed on the same plane as the first display screen when viewed from above . 2. The self-checkout apparatus according to claim 1 , wherein the devices to be operated include a scanner for reading product information from a product to be purchased, a cash insertion slot, a change ejection slot, and a receipt printer. The self-checkout apparatus according to claim 1 or 2 , wherein the operation guide is displayed so as to avoid the product on the first display surface. The self-checkout device according to any one of claims 1 to 3 , further comprising a projection unit that projects the operation guide onto the first display surface. a second display surface provided above the first display surface and facing the user;
The self-checkout apparatus according to claim 1 , wherein the second display surface displays a guide image of a person. The detection unit detects a position of a fingertip of the user,
The self-checkout device according to any one of claims 1 to 5 , further comprising a determination unit that determines whether or not the operation of the operation guide is performed based on a detection result of the detection unit. The self-checkout device according to claim 6 , wherein the determination unit determines that the operation guide is being operated when the fingertip approaches within a first distance from the surface of the first display surface. The self-checkout device of claim 7, characterized in that the determination unit displays a mark at a position on the operation guide where it is estimated that the fingertip is pointing when the fingertip approaches within a second distance, which is longer than the first distance, from the surface of the first display surface.

Images