JP7595375B2 - Product sales data processing system, mobile terminal device, and computer - Google Patents

Description

The present invention relates to a product sales data processing system, a mobile terminal device, and a computer.

There is a known system in which customers themselves register products using a mobile terminal equipped with a camera (see, for example, Patent Document 1).

JP 2018-147252 A

However, because customers themselves must register products using a mobile device, there is a risk of problems occurring, such as customers thinking they have scanned the barcode of a product but not actually being able to do so, or maliciously putting products into their shopping carts without having the barcode scanned.

The present invention was made in light of these circumstances, and aims to make it possible to appropriately deal with problems that arise when customers register products themselves.

One aspect of the present invention that solves the above-mentioned problems is a product sales data processing system in which products are registered by a mobile terminal device and a settlement device performs settlement processing corresponding to the products registered by the mobile terminal device, the product sales data processing system comprising: an imaging means that is capable of capturing images of the situation related to the registration of products by customers; a detection means that detects a predetermined registration failure situation in which the registration of a product is deemed to be insufficient while the product is being registered according to a transaction; and a display control means that displays an image corresponding to the registration failure situation detected by the detection means based on the image obtained by the imaging means when the settlement device settles the products registered according to the transaction.

1 is a diagram illustrating an example of the overall configuration of a product sales data processing system according to an embodiment of the present invention. 1 is a diagram showing an example of the state near the entrance of a store in which a product sales data processing system according to an embodiment of the present invention has been introduced. 1A to 1C are diagrams illustrating examples of how a mobile terminal is set in a shopping cart according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a settlement device according to the present embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a mobile terminal according to the present embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a transaction management device according to the present embodiment. 4A to 4C are diagrams illustrating an example of customer information, store information, and cart information according to the embodiment. FIG. 13 is a diagram showing an example of a definition of a reserved product according to the embodiment; A sequence diagram showing an example of processing procedures executed by a mobile terminal, a settlement device, and a transaction management device according to this embodiment in response to the accounting of one transaction in self-service operation. 11A to 11C are diagrams showing examples of a start-up screen, a product registration initial screen, a product registration update screen, and a code display screen according to the present embodiment. 11A to 11C are diagrams illustrating an example of a registered product presentation screen, a product registration screen, and a guide image according to the embodiment. 11 is a flowchart illustrating an example of a processing procedure executed by a mobile terminal and a transaction management device according to the present embodiment in relation to product registration, including processing for dealing with a registration failure situation. FIG. 13 is a diagram showing an example of a settlement screen according to the embodiment. FIG. 13 is a diagram showing an example of a reserved product processing screen according to the embodiment; FIG. 13 is a diagram showing an example of a reserved product processing screen according to the embodiment; 1 is a diagram showing an example of an enlarged captured image area according to the embodiment; FIG. 13 is a diagram showing an example of a reserved product processing screen according to the embodiment; 1 is a diagram showing an example of an enlarged captured image area according to the embodiment; 13 is a flowchart showing an example of a processing procedure executed by the settlement device according to the embodiment in relation to displaying a reserved product processing screen. FIG. 11 is a diagram showing an example of a captured image area according to a first modified example of the embodiment. FIG. 11 is a diagram showing an example of a settlement screen according to a second modified example of the present embodiment.

<Embodiment>
[Example of overall configuration of product sales data processing system]
FIG. 1 shows an example of the overall configuration of a product sales data processing system 1 according to the present embodiment.
The product sales data processing system 1 in the figure includes a management device 10 (e.g., a store controller), a transaction management device 20 (cloud server), a registration settlement device 30, a settlement device 40, a monitoring device 50 (e.g., a personal computer, etc.), and a mobile terminal 60.
Although the registered settlement device 30 is illustrated, the number of registered settlement devices 30 installed in one store may be two or more. Although two settlement devices 40 are illustrated in the figure, the number of settlement devices 40 in one store may be one or three or more. Although one monitoring device 50 is illustrated in Fig. 1, the number of monitoring devices 50 in one store may be two or more. Furthermore, two or more management devices 10 may be provided in one store.
The product sales data processing system 1 in the figure can be introduced into various stores (supermarkets, convenience stores, drug stores, home centers, home appliance retailers, etc.).

The product sales data processing system 1 in the figure is capable of handling a store clerk-assisted operation in which a store clerk registers products using a registration and settlement device 30. The store clerk-assisted operation includes an operation in which the store clerk subsequently uses the registration and settlement device 30 to settle the registered products, and an operation in which a customer uses a settlement device 40 to settle the registered products.
The sales system shown in the figure is also capable of supporting self-service operation in which a customer registers an item using a mobile terminal 60 and also performs payment using a payment device 40. In the following explanation, a case in which self-service operation is performed will be taken as an example.

The management device 10, the registration settlement device 30, the settlement device 40, and the monitoring device 50 are installed in the store. The management device 10, the registration settlement device 30, the settlement device 40, and the monitoring device 50 are connected to each other so that they can communicate with each other via a LAN 19 (which may be wired or wireless). The management device 10 is capable of communicating with the transaction management device 20.

The management device 10 is a device that manages various devices that are mainly installed in a store. For example, the management device 10 has a communication function with an external device, and receives a product file (product master) from an external device (e.g., a headquarters server not shown, which may be the transaction management device 20) and supplies it to the registered settlement device 30, etc., or receives transaction information from the settlement device 40, etc. and transmits it to an external device (e.g., the transaction management device 20, etc.).

The transaction management device 20 is a server that provides various services. For example, the transaction management device 20 manages various information corresponding to self-operation (e.g., transaction information (cart information), etc.).

The registration and settlement device 30 is a device that executes the registration process and the settlement process. The registration and settlement device 30 is used by a store clerk (the registration process and the settlement process are executed by the operation of the store clerk).

The settlement device 40 is a device that executes the settlement process. The settlement device 40 is used by the customer. In other words, the settlement device 40 executes the settlement process in response to operations by the customer. Note that, if the registered items at the time of settlement include items that require the assistance of a store clerk (hereafter referred to as "on hold items"), a store clerk is called, and the store clerk operates the settlement device 40 to cause the settlement device 40 to execute processes such as canceling or registering the on hold items.

The monitoring device 50 is a device that mainly monitors various devices within the LAN 19. The monitoring device 50 monitors the operating status of other devices (e.g., the registration settlement device 30, the settlement device 40, etc.), displays various information on the other devices, and outputs various commands to the other devices.

The mobile terminal 60 is a terminal used by a customer under self-operation. In this embodiment, the mobile terminal 60 is used while attached to a shopping cart 70 under self-operation.

The portable terminal 60 has a communication function, an imaging function (camera), and a recognition function for recognizing various codes (barcodes, two-dimensional codes, etc.). For example, the portable terminal 60 recognizes barcodes attached to products. In other words, the portable terminal 60 reads product identification information (product codes). The portable terminal 60 also recognizes barcodes (markers 73, described below) in which specific information is encoded. In other words, the portable terminal 60 reads the specific information. The portable terminal 60 may also extract feature points in an image captured by the imaging function (an image acquired as a through image) and recognize objects (various codes).

The mobile terminal 60 is a smartphone or the like owned by the customer. The mobile terminal 60 may be owned by the store and loaned to the customer when shopping. Although two mobile terminals 60 are shown in the figure, the number of mobile terminals 60 used for product registration in the store varies depending on the number of customers.

A shopping application having a function for registering products in accordance with the self-service operation of this embodiment is installed on the mobile terminal 60.

Figure 2 shows an example of the state near the entrance of a store that has introduced the product sales data processing system 1 of this embodiment. As shown in Figure 2, near the entrance (or the entrance if the entrance and exit are separate), a rental shopping cart 70, rental shopping baskets 80, and sales shopping bags 90 are prepared. When a customer visits the store, they place the shopping basket 80 in the shopping cart 70, place the shopping bag 90 in the shopping basket 80, etc., and proceed inside the store. At this time, if the customer wishes to do their own shopping, they place the mobile terminal 60 in the shopping cart 70.

The customer may bring the shopping bag 90 that he or she purchased this time as his or her own bag the next time he or she visits the store, or may bring a bag other than the shopping bag 90 as his or her own bag. In addition, the customer may purchase a plastic bag.

Although not shown in the figure, the shopping cart 70 is also provided with a bag for reserved items (described below) in addition to the shopping basket 80.

[Example of setting a mobile terminal in a shopping cart]
An example of how the mobile terminal 60 is set in the shopping cart 70 will be described with reference to Fig. 3. Fig. 3(A) is a side view of the portion of the shopping cart 70 where the mobile terminal 60 is set. Fig. 3(B) is a top view of the shopping cart 70.

As shown in FIG. 3(A), a holder 71 for setting the mobile terminal 60 is attached to the shopping cart 70. The customer sets the mobile terminal 60 in the holder 71. When the mobile terminal 60 is set in the holder 71, it is in a state where the imaging unit 606 can capture an image of the entire shopping bag 90 or the personal bag, as shown in FIG. 3(B).

[About the settlement device]
Fig. 4 is a diagram for explaining the settlement device 40. Fig. 4(A) is a diagram showing an example of the appearance of the settlement device 40 (a front view seen from the customer side). Fig. 4(B) is a diagram showing an example of the configuration of the settlement device 40. In Fig. 4(A) and Fig. 4(B), the same parts are given the same reference numerals.

Below, an example of the configuration of the settlement device 40 shown in FIG. 4(B) will be described with reference to FIG. 4(A). The settlement device 40 comprises a CPU 401, a ROM 402, a RAM 403, a hard disk 404, a display unit 405, a scanner unit 406, a card payment unit 408, a change dispenser 409, a printing unit 413, an audio output unit 414, a communication unit 415, and an imaging unit 416 (camera). These can communicate with each other via a bus. The settlement device 40 may further include a key operation unit and an alarm unit (for example, a lamp on a sign pole, etc.).

The CPU 401 is a central processing unit, and controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 .
The ROM 402 is a read-only memory, and stores various types of information used by the CPU 401, including programs.

RAM 403 is a read/write memory, and stores various information as a main storage device. For example, RAM 403 stores information read from ROM 402 or hard disk 404, information acquired from the outside, information generated during processing, etc.

The hard disk 404 stores various information as an auxiliary storage device. For example, the hard disk 404 may store programs executed by the CPU 401 instead of the ROM 402. Also, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information obtained from the outside, information generated during processing, etc. Note that a storage device such as an SSD (Solid State Drive) may be provided instead of the hard disk 404.

The display unit 405 is a touch display that displays various information and accepts various inputs.

The scanner unit 406 optically reads various information. For example, the scanner unit 406 scans a code (barcode, two-dimensional code, etc.) displayed on the display unit of the mobile terminal 60 and reads the information required for the settlement process. The scanner unit 406 may also scan a code printed on an accounting ticket (registered trademark) emitted by another device (for example, the registered settlement device 30) and read the information required for the settlement process. The scanner unit 406 may also scan a code attached to a product and read the product code. The scanner unit 406 may also scan a code attached to a store clerk's name tag and read the store clerk code.

The card payment unit 408 is a payment mechanism that uses various cards (credit cards, prepaid cards such as transportation cards, cash cards for debit payments, point cards for point payments, etc.). The card payment unit 408 is equipped with a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient that it is equipped with at least the card recognition unit. The card recognition unit may recognize various cards (e.g., point cards, membership cards, shareholder cards, etc.) for the purpose of granting benefits, discounts, etc.

The change dispenser 409 (cash settlement unit) is a cash settlement mechanism that has an input port for banknotes and coins and an output port for banknotes and coins, calculates the amount of money inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. When banknotes or coins are inserted into the input port, they are detected by a sensor (detecting that they have been inserted, detecting the number of coins by denomination, etc.).

The printing unit 413 prints and issues various media (receipts, bills, etc.).
The audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like.
The communication unit 415 transmits and receives information to and from other devices (the management device 10, the registered settlement device 30, the other settlement devices 40, the monitoring device 50, etc.).

The imaging unit 416 is configured to capture, for example, an image of the hands of a customer operating the change machine.

[Registered payment devices]
The registration and settlement device 30 may be configured to have a display unit and a scanner unit on each side (clerk side, customer side) and a settlement unit (card settlement unit, cash settlement unit) on one side (customer side). The registration and settlement device 30 configured in this manner may be a device that can be switched to any of a plurality of operation modes (a mode in which the registration process is executed by the operation of the clerk and the settlement process is executed by the operation of the customer, a mode in which the registration process is executed by the operation of the clerk and the customer and the settlement process is executed by the operation of the customer, etc.) depending on, for example, the operating status. Furthermore, if the registration and settlement device 30 is configured to execute the settlement process by the operation of the customer, it can be used for self-operation. That is, the customer can make a transaction by having the mobile terminal 60 execute the registration process and have the registration and settlement device 30 execute the settlement process.
However, in the following explanation, an example will be given in which the settlement device 40 is used for self-service operation.

[Example of configuration of a mobile terminal]
5 shows an example of the configuration of the portable terminal 60. The portable terminal 60 in the figure includes a CPU 601, a storage unit 602, a RAM 603, a display unit 604, an operation unit 605, an imaging unit 606, and a network-compatible communication unit 607.

The CPU 601 is a central processing unit that controls the operation of the mobile terminal 60 by reading and executing programs stored in the storage unit 602 .
The storage unit 602 is an auxiliary storage device for the CPU 601, and stores various types of information used by the CPU 601, including programs.
The RAM 603 is a main storage device for the CPU 601 .

The display unit 604 displays an image according to the display control of the CPU 601. The display unit 604 may be configured as a touch panel that allows operations to be performed on the screen.
The operation unit 605 collectively indicates various operators provided on the mobile terminal 60 and various input devices connected to the mobile terminal 60 .

The imaging unit 606 captures images under the control of the CPU 401. In addition, the imaging unit 606 in this embodiment can also function as a part that reads two-dimensional codes in a code reader that processes two-dimensional codes.

The network-compatible communication unit 607 is a part that communicates wirelessly via a network. The network-compatible communication unit 607 enables the mobile terminal 60 to communicate with the transaction management device 20.

[Example of transaction management device configuration]
6 shows an example of the configuration of transaction management device 20. Transaction management device 20 in the figure includes a CPU 201, a storage unit 202, a RAM 203, and a network-compatible communication unit 204.
The CPU 201 is a central processing unit, and controls the operation of the transaction management device 20 by reading and executing programs stored in the memory unit 202 .
The storage unit 202 is an auxiliary storage device for the CPU 201, and stores various types of information used by the CPU 201, including programs.
The RAM 203 is a main storage device for the CPU 201 .
The network-compatible communication unit 204 is a part that performs communication via a network.

The memory unit 202 of the transaction management device 20 stores customer information, store information, product information, and cart information.

[Information stored in the transaction management device]
The customer information stored in the memory unit 202 is information for managing individual customers. The transaction management device 20 generates customer information when a customer is registered (a customer may be considered to have been registered when the customer information of that customer is stored). The transaction management device 20 also updates the customer information as appropriate based on cart information, etc. The transaction management device 20 may refer to the cart information at a specified time every day, for example, and update the customer information.

7A shows an example of customer information stored in memory unit 602 of transaction management device 20. The customer information in this figure includes customer identification information, customer name, customer registration date, cancellation information, customer rank, number of points, etc.
The customer identification information is identification information that uniquely identifies a customer.
The customer identification information may be application identification information that is uniquely assigned to each installed shopping application when the shopping application of this embodiment is installed. The application identification information may be, for example, a number that is uniquely (for example, sequentially) assigned to each installed shopping application. Alternatively, the application identification information may be terminal identification information that is unique to the mobile terminal 60 in which the shopping application is installed. The terminal identification information may be, for example, a telephone number assigned to a customer who is the user of the mobile terminal 60, or hardware identification information that is uniquely assigned to the hardware of the mobile terminal 60, such as a MAC (Media Access Control) address.
The customer name is the customer's name or nickname, etc. The customer registration date is the date and time when the customer was registered. Cancellation information is information regarding the cancellation of registered products after registration. The customer rank is a rank according to the customer's purchasing history. When generating customer information for a new customer, the customer identification information, customer name, and customer registration date are generated, but since this is before the start of actual trading (product registration), other information (cancellation information, etc.) is not generated.

The transaction management device 20 generates and stores customer identification information, for example, when a customer is registered (for example, when the mobile terminal 60 downloads or installs a shopping application from an external device (for example, a specified server that provides general applications, the transaction management device 20) to use a shopping service provided by the transaction management device 20). In addition, the transaction management device 20 acquires information entered into the name field of a registration form (input form) using the mobile terminal 60, for example, when a customer is registered, and stores it as the customer name. In addition, the transaction management device 20 acquires the current date and time when the customer is registered, and stores it as the customer registration date.

In addition, instead of or in addition to storing the customer information in its own memory, the transaction management device 20 may store some or all of the customer information in another device (such as a file server accessible to the transaction management device 20).

7B shows an example of store information stored in the memory unit 602 of the transaction management device 20. The store information in the figure includes store identification information, store name (branch name), store specific information 1, and store specific information 2.
The store identification information is identification information that uniquely identifies a store. The store identification information shown in Fig. 7B is composed of a store (trade name) or company code and a branch code.
The store name is the name of the store. The store name shown in Fig. 7B is composed of the store (trade name) or company and the branch name.
The store identification information 1 is information on a two-dimensional code (QR code (registered trademark) or the like) for identifying a store in which a transaction takes place (a store where goods are bought and sold).
The store identification information 2 is store location information (GPS information) for identifying the store where the transaction is to be carried out.
In the example shown in FIG. 7B, the store identification information and the store specifying information 1 are different, but the store identification information and the store specifying information 1 may be the same.

The transaction management device 20 may obtain and store store information from an external source (such as a server (not shown) at a headquarters that manages each store). Also, instead of or in addition to storing the store information in its own memory, the transaction management device 20 may store some or all of the store information in another device (such as a file server accessible by the transaction management device 20).

Product information is information obtained, for example, from the store management device 10, and is information about products sold in the store. Product information for one product includes information such as the product code (product identifier), product name, and price.

The cart information is information for managing individual transactions. The transaction management device 20 generates the cart information at the start of a transaction. In addition, the transaction management device 20 updates the cart information as the transaction progresses (each time a product is registered) (it may be considered that a product has been registered when the product is stored in the cart information).

FIG. 7(C) is an example of cart information stored in the memory unit 602 of the transaction management device 20.
The cart information in the figure includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, registered product information, reserved product information, cancellation information, and the like.

Note that the accounting status information corresponds to the second embodiment described below, so a description of it will be omitted here.

The cart identification information is identification information that uniquely identifies the cart information, and includes, for example, store identification information, a date, and a serial number (for example, a serial number for each store and date).
The transaction start date and time is the date and time when the transaction starts. The transaction start date and time may be, for example, the date and time when the cart information is generated. The transaction start date and time may be the date and time when the first item of product is registered (the date and time when the registered product information (registered product 1) in FIG. 7(C) is stored). The date and time when the cart information is generated and the date and time when the first item of product is registered may both be stored separately.

The transaction end date and time is the end date and time of the transaction. For example, the transaction start date and time may be the settlement date and time. The customer identification information is customer identification information that identifies the customer of the transaction. When the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are generated, but since this is before the start of the actual transaction (product registration), other information (transaction end date and time, etc.) is not generated. The settlement date and time may be the settlement start date and time or the settlement end date and time. Both the settlement start date and time and the settlement end date and time may be stored separately as the transaction end date and time.

Registered product information (total) is information that is updated each time a product is registered. Registered product information (total) includes the number of products (number of products), estimated subtotal amount (e.g., subtotal amount before discounts, etc. are reflected), subtotal amount, etc. Registered product information (registered product 1) is the registration information for one product item. Registered product information (registered product 2) is the registration information for two products. In the example shown in Figure 7 (C), registered product information (registered product 3) to registered product information (registered product 5) are omitted from the illustration. Registered product information (N; N is an integer) includes the product code, product name (product name), price, etc.

The registered product information (N) may include the registration date and time of the Nth item of product. In other words, the transaction management device 20 may store the registration date and time of the registered product as the registered product information. The registration date and time of each product may also be used as a basis for determining whether or not to apply a service such as a time service and the effect of applying the service.

Pending product information (total) is information that is updated each time a pending product (described below) is registered. Pending product information (total) includes the number of pending products (number of products), the number of pending products that are NO-FILE, the number of pending products that cannot be read (check for tampering), etc.

NO-FILE refers to a product whose barcode or product code was successfully scanned at the store (the product code was read), but the product code is not stored in the product information (a product whose product information is not registered).

A "read NG" refers to a failure to scan the product code at the store (the product code could not be read), or a product whose product code has not been scanned at the store. In other words, a "read NG" refers to a timeout process when it is possible to determine that the product has been imaged for a certain period of time using image recognition technology but has not been able to be recognized as a barcode, or a product that has been timed out.
For example, if the barcode cannot be correctly acquired (recognized) even though the customer has performed an operation to read the barcode due to wrinkles in the package, smudges or dirt on the barcode print, or the barcode being printed on a transparent film, it will be determined that the barcode cannot be read.

Pending product information (pending product 1) is information on pending product 1. Pending product information (pending product 2) is information on pending product 2. Pending product information (pending product 3) is information on pending product 3.

The reserved product information (reserved product N; N is an integer) includes the reserved product type (information indicating whether the reserved product is NO-FILE or cannot be read) and image data (image data captured when reading was NG). For example, if the Nth item is a reserved product due to NO-FILE, the reserved product information (reserved product N) includes the reserved product type "1 (NO-FILE)" and image data. Also, if the Nth item is a reserved product due to reading being NG, the reserved product information (reserved product N) includes the reserved product type "2 (cannot be read)" and image data.

8 shows an example of the definition of reserved products corresponding to the cart information of this embodiment. In this figure, an example is shown in which reserved products are classified into eight categories. Classification numbers "1" to "8" are assigned to each of the eight reserved product classifications.
The classification of the reserved product with classification number 1 is "NO-FILE" as explained in FIG.
The classification of the reserved product with classification number 2 is “reading NG” as described with reference to FIG.

The classification of reserved items with classification number 3 is "pseudo-scan." A "pseudo-scan" reserved item is an item where the customer attempted to have the mobile terminal 60 read the code of the item, but the mobile terminal 60 was unable to read the code properly.

As a specific example, a "pseudo scan" corresponds to a situation in which the product is brought within the imaging range of the imaging unit 606, but the product code is not brought within the imaging range of the imaging unit 606.
Moreover, a "pseudo scan" corresponds to a situation in which, although the product is brought within the imaging range of the imaging unit 606, an image of a portion that does not have a code is captured.
Additionally, a "pseudo scan" corresponds to a situation in which the product is brought into the imaging range of the imaging unit 606 while the code portion is covered by a finger or the like.
Additionally, a "pseudo scan" refers to a situation in which the image capture unit 606 captures an image of the code on a product, but the product is passed through the imaging range of the imaging unit 606 at a speed that the mobile terminal 60 cannot read the code.

The classification of the reserved item with classification number 4 is "non-scanned entry." A reserved item with "non-scanned entry" is an item that was added to the shopping cart 80 without any operation being performed to position the item within the imaging range of the imaging unit 606.

The classification of reserved items with classification number 5 is "age verification items." "Age verification items" are items such as alcohol and tobacco that require a store clerk to verify the customer's age before selling them.

The classification of reserved product number 6 is "pharmaceuticals." "Pharmaceuticals" are products that require a licensed person, such as a pharmacist, to confirm with the purchaser.

The classification of reserved items with classification number 7 is "anti-theft tag items." "Anti-theft tag items" are items that have anti-theft tags attached and require a store clerk to remove the anti-theft tag from the item when it is sold.

The classification of reserved items with classification number 8 is "cancelled items." "Cancelled items" are items that were once registered but were cancelled by the customer using the cancellation operation on the mobile terminal 60. "Cancelled items" are taken from the customer by a store clerk at the time of payment, for example.

Of the classification numbers 1 to 7, the three categories "Cannot read", "Pseudo scan", and "Input without scanning" correspond to products for which the code was not read when the customer was registering the product on the mobile terminal 60. Hereinafter, this situation in which the code is not read during registration and the product is registered as a reserved product in the cart information is referred to as an inadequate registration situation. Products that fall into any of the three categories of "Cannot read", "Pseudo scan", and "Input without scanning" may also be referred to as inadequate registration products.

In addition, instead of or in addition to storing the cart information in its own memory, the transaction management device 20 may store some or all of the cart information in another device (such as a file server accessible to the transaction management device 20).

[Example of basic processing procedure for self-operation]
The sequence diagram in Fig. 9 shows an example of a processing procedure executed by the mobile terminal 60, the settlement device 40, and the transaction management device 20 in response to a transaction in a self-service operation. Specifically, the processing in Fig. 9 shows an example of the processing of the mobile terminal 60, the settlement device 40, and the transaction management device 20 from when a customer registers a product to be purchased from among products displayed in the store, until the settlement for the registered product is completed.

Step S1: The mobile terminal 60 acquires information that identifies the store (store identification information). For example, a two-dimensional code for identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output to a display screen, a medium with the two-dimensional code printed on it may be attached, etc.), and a customer visiting the store may scan (read) the two-dimensional code with the mobile terminal 60, thereby causing the mobile terminal 60 to acquire the store identification information. When a customer visiting the store starts a shopping application, a screen instructing the customer to scan the two-dimensional code may be displayed as a start-up screen, or when a customer visiting the store scans the two-dimensional code with the mobile terminal 60, the shopping application may start and a screen informing the customer that they are currently connected to the transaction management device 20 may be displayed as a start-up screen.

For example, since a store is identified by its location, a customer who visits the store may obtain location information (GPS information) on the mobile terminal 60 at the store (i.e., the store's location information may be obtained as store identification information). When a customer who visits the store starts a shopping application, the location information may be obtained and a screen may be displayed as a startup screen informing the customer that the application is currently connected to the transaction management device 20. If multiple stores are detected from the location information and it is not possible to identify one, a selection screen may be displayed to allow the customer to select. Alternatively, the application may switch to a mode that forcibly obtains a two-dimensional code.

10A shows an example of a start-up screen of a shopping application displayed on the display unit 604 of the mobile terminal 60. The start-up screen in the figure displays a store name. This store name is displayed based on the store-specific information acquired in step S1.
When a customer wishes to start registering a product for shopping, the customer operates a transaction start button BT10 arranged on the start-up screen.
In response to the operation of the transaction start instruction button BT10, the mobile terminal 60 transmits a transaction start request including the acquired store specific information and customer identification information to the transaction management device 20. The transaction start request may be transmitted in response to the customer's operation on the mobile terminal 60 as described above, or may be transmitted in response to the acquisition of store specific information without receiving any particular operation.
Regarding the customer identification information, the information entered in the name field of the registration form using the mobile terminal 60 during customer registration (when downloading or installing a shopping application on the mobile terminal 60) may be stored in the customer information of the transaction management device 20, but may also be stored in the memory of the mobile terminal 60 in addition to the transaction management device 20. When a store is identified (when the product registration initial screen described below is acquired), the store name, the services being offered (flyer information distributed on that day), and available coupon information of the store may be displayed on a screen (the product registration initial screen or a screen other than the product registration initial screen). The service and coupon information may be acquired from the transaction management device 20 as screen information, for example.

The destination information (address of the transaction management device 20) may also be obtained when registering the customer (when downloading or installing a shopping application to the mobile terminal 60) and stored in the memory of the mobile terminal 60. When scanning a two-dimensional code, the destination information may be converted into a two-dimensional code in addition to the store-specific information, and the mobile terminal 60 may obtain the destination information along with the store-specific information by scanning the two-dimensional code.

Step S2: The transaction management device 20, which has received the customer identification information and store identification information as a transaction start request from the mobile terminal 60, generates cart information for the transaction. Note that in the initial state in which the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are stored, but since this is before the start of the actual transaction (product registration), other information (transaction end date and time, product information (registered product information, pending product information), etc.) is not stored.

As described above, the transaction management device 20 stores store information such as that shown in FIG. 7(B), so when it receives store identification information (it also receives customer identification information) as a transaction start request from the mobile terminal 60, if the received store identification information is a two-dimensional code, it acquires the store identification information by referring to store identification information 1, and if the received store identification information is location information (GPS information), it acquires the store identification information by referring to store identification information 2. Note that if the store identification information received from the mobile terminal 60 is a two-dimensional code of the store identification information, the transaction management device 20 can simply acquire it as is.

In other words, the transaction management device 20, which receives customer identification information and store identification information from the mobile terminal 60 as a transaction start request, acquires the store identification information from the store identification information received from the mobile terminal 60, and further acquires the current date, issues (assigns) a serial number, combines the store identification information, the current date, and the serial number, and stores them as cart identification information in the cart information. Also, the transaction management device 20, which receives store identification information and customer identification information from the mobile terminal 60 as a transaction start request, acquires the current date and time, and stores them as the transaction start date and time (generation date and time) in the cart information. Also, the transaction management device 20, which receives store identification information and customer identification information from the mobile terminal 60 as a transaction start request, stores the customer identification information received from the mobile terminal 60 as customer identification information in the cart information.

Step S3: The transaction management device 20, which has generated the cart information for the transaction, generates product registration initial screen information (screen information of the product registration screen, which is the initial screen), and transmits it to the mobile terminal 60. Specifically, the transaction management device 20, for example, generates product registration initial screen information, and transmits the generated product registration initial screen information together with the cart identification information to the mobile terminal 60.

Step S4: The mobile terminal 60 receives the cart identification information and the product registration initial screen information from the transaction management device 20, stores the cart identification information, and displays the registration screen on the display unit. Specifically, the mobile terminal 60 displays the product registration initial screen as shown in FIG. 10(B), for example.

Step S5: The customer operates the mobile terminal 60 to scan the barcode attached to the product and read the product code. In the figure, it is assumed that the barcode scan was successful. Steps S5 to S9 are repeated each time a barcode attached to a product is scanned.

The mobile terminal 60 that has acquired the barcode transmits the cart identification information and the product registration information, including the product code obtained by scanning, to the transaction management device 20.

Step S6: The transaction management device 20 receives the product registration information (cart identification information and product code) from the mobile terminal 60 and identifies the cart information of the transaction from the cart identification information included in the received product registration information.

Step S7: The transaction management device 20 updates the product data in the identified cart information. Specifically, if the received product registration information contains a product code for item N, the transaction management device 20 stores the product code as the product code of the registered product information (registered product N) in the identified cart information, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). The transaction management device 20 also updates the registered product information (total) in the identified cart information.

Step S8: After updating the product data in the cart information, the transaction management device 20 generates product registration update screen information (screen information of the product registration screen, which is an update screen to which the registered product has been added) and transmits it to the mobile terminal 60. Specifically, the transaction management device 20 generates product registration update screen information such that the mobile terminal 60 displays a product registration update screen to which the list item of the currently registered product has been added to the registered product list, as shown in FIG. 10 (C), for example, and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information.

The product registration screen (product registration update screen) shown in FIG. 10(C) is displayed on the mobile terminal 60 after "XX bread" is registered as the third product. In other words, when "XX yogurt" is stored in the cart information as the first product, the transaction management device 20 generates product registration update screen information such that "XX yogurt" is displayed on the mobile terminal 60 and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information. When "XX chocolate" is stored in the cart as the second product, the transaction management device 20 generates product registration update screen information such that "XX yogurt" and "XX chocolate" are displayed on the mobile terminal 60 and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information. When "XX bread" is stored in the cart information as the third product, the transaction management device 20 generates product registration update screen information such that "XX yogurt", "XX chocolate", and "XX bread" are displayed on the mobile terminal 60 as shown in FIG. 10(C) and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information.

In response to the registration of one product, a registered product presentation screen WD1 may be displayed, presenting the currently registered product, as shown in FIG. 11(A), for example. In addition, in the registered product presentation screen WD1 in the same figure, if the product in question is available for eat-in, a "Yes" button BT1 and a "No" button BT2 are provided to confirm with the customer whether or not they would like to eat in the product. In this case, the price of the product will be calculated so that the consumption tax rate for the product in question differs depending on whether the "Yes" button BT1 is operated or the "No" button BT2 is operated.

Step S9: The mobile terminal 60, which has received the cart identification information and product registration update screen information from the transaction management device 20, adds the product to the registration screen. Specifically, the mobile terminal 60 displays a product registration update screen such as that shown in FIG. 10(C). As described above, the product registration screen (product registration update screen) shown in FIG. 10(C) is displayed on the mobile terminal 60 after "XX bread" has been registered as the third product item.

Step S10: The mobile terminal 60 accepts a payment instruction operation as a customer operation. For example, it accepts the touch of the "Proceed to checkout" button shown in FIG. 10(C).

Step S11: The mobile terminal 60 that has accepted the settlement transition instruction operation generates a two-dimensional code. That is, the mobile terminal 60 converts information required to execute the settlement process for the items purchased by the mobile terminal 60 (e.g., cart identification information) into a two-dimensional code. The mobile terminal 60 that has generated the two-dimensional code displays the generated two-dimensional code on the display unit. For example, a code display screen with a two-dimensional code arranged as shown in FIG. 10 (D) is displayed on the display unit.

Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the mobile terminal 60. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the mobile terminal 60 that is pointed by the customer within the recognition range of the scanner unit 406.

Step S13: The settlement device 40 reads the two-dimensional code displayed on the display of the mobile terminal 60 and requests the transaction management device 20 to calculate the subtotal amount. For example, the settlement device 40 sends a calculation request (subtotal calculation request information) requesting calculation of the subtotal amount to the transaction management device 20 together with the cart identification information obtained from the two-dimensional code.

Step S14: The transaction management device 20 receives the cart identification information and the sub-calculation output request information from the mobile terminal 60 and identifies the cart information of the transaction from the cart identification information.

Step S15: After identifying the cart, the transaction management device 20 calculates the subtotal amount using information such as the price in the registered product information included in the cart information corresponding to the identified cart information.

Step S16: After calculating the subtotal, the transaction management device 20 updates the cart information (stores the subtotal (calculated subtotal)) and transmits the subtotal information indicating the calculated subtotal together with the cart identification information to the settlement device 40.

Step S17: The settlement device 40, which has received the cart identification information and subtotal information from the transaction management device 20, displays the subtotal amount (a settlement screen including the subtotal amount) on the display unit 405.

Step S18: The settlement device 40, which has displayed the subtotal amount on the display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the change amount, and, if there is change, dispenses the change and issues a receipt. In addition, when the settlement is completed, the settlement device 40 sends settlement completion information together with the cart information to the transaction management device 20, and the transaction management device 20 stores the transaction end date and time (settlement date and time) for that cart.

[Configuration for handling registration failure situations]
9 shows a process corresponding to the case where the reading of the product code is successful in step S5. However, in the stage of the product code reading process corresponding to step S5, the mobile terminal 60 may determine that a registration failure situation corresponding to any of the classification numbers 2 to 4 in FIG. 8 (a registration failure situation) has occurred.

As a specific example, FIG. 11(B) shows, by way of a captured image of a product registration screen, an example corresponding to a case where the mobile terminal 60 has determined that a registration failure situation has occurred as a pseudo scan of classification number 3. The figure shows the product registration screen displayed on the mobile terminal 60. The product registration screen in the figure shows a state in which a captured image is displayed of a customer attempting to have the mobile terminal 60 read the barcode of a product with the barcode portion obscured by their finger.

In such a case, the mobile terminal 60 obtains a processing result indicating that it recognized the presence of a barcode from the captured image, but was unable to extract the code from the barcode. Since this type of processing result falls under one type of processing result in a pseudo scan, the mobile terminal 60 detects that a registration failure situation has occurred as a pseudo scan. In this case, the mobile terminal 60 transmits product registration information to the transaction management device 20 indicating that the product has been pseudo-scanned with classification number 3.

When the mobile terminal 60 transmits product registration information in response to the occurrence of a registration failure situation in this manner, the mobile terminal 60 transmits captured images corresponding to the occurrence of the registration failure situation. The captured images to be transmitted may be extracted from the buffered captured images, for example, by buffering the captured images captured by the imaging unit 606 for a certain period of time in order to read the product code. The captured images to be transmitted may be, for example, a predetermined number of still images, or may be moving images for a predetermined period of time in response to the occurrence of the registration failure situation.

In addition, when a registration failure situation including a pseudo scan occurs as described above, the mobile terminal 60 treats the product that has become registration failure corresponding to the registration failure situation (registered failure product) as a reserved product. In this case, the mobile terminal 60 may display a guide image WD2 that, when it is determined that a registration failure situation has occurred, marks the registered failure product as a reserved product and encourages the user to place the product in a bag for reserved products, as shown in FIG. 11(C). Specifically, the guide image WD2 in the figure displays a message saying, "If you cannot scan, please place the product in a special bag." Note that the form of the guide image WD2 is not particularly limited. The message displayed in the guide image WD2 may be, for example, "Scanning has failed" or "The product has been determined to be registered failure."
The guide image WD2 in the figure is displayed so as to be superimposed on the product registration screen. In the guide image WD2 in the figure, a confirmation button BT3 is arranged. In this case, the guide image WD2 is erased in response to an operation on the confirmation button BT3. Alternatively, the guide image WD2 may be erased in response to a certain period of time having elapsed since the start of display. After the guide image WD2 is erased, the screen may return to the product registration screen of FIG. 10(B) or FIG. 10(C), for example.

Registered defective products are treated as reserved products as described above. On the product registration screen, a registered product list is displayed as shown in FIG. 10 (C), but for reserved products that are classified as registered defective products, list items do not need to be displayed in the registered product list. Reserved products that are not classified as registered defective products are displayed as list items in the registered product list. Registered defective products may also have list items displayed in the registered product list to indicate that they are registered defective products.

With reference to the flowchart in FIG. 12, an example of a processing procedure executed by the mobile terminal 60 and the transaction management device 20 in this embodiment in relation to product registration, including processing for responding to a registration failure situation, will be described. In correspondence with FIG. 9, the processing in this figure shows the process of steps S5 to S11 more specifically, including processing for responding to a registration failure situation.

First, an example of a process procedure executed by the mobile terminal 60 will be described.
Step S101: With the product registration screen displayed, the mobile terminal 60 determines whether or not the reading of the product code has been completed normally.

Step S102: When the reading of the product code is completed normally, the mobile terminal 60 transmits the product registration information including the read product code and the cart identification information to the transaction management device 20.
Step S103: Upon receiving a registration completion notification sent from the transaction management device 20 in response to the transmission of the product registration information in step S102, the mobile terminal 60 updates the product registration screen, for example as illustrated in FIG. 10 (C).

Step S104: The mobile terminal 60 is configured to detect a registration failure situation based on the captured image obtained by having the imaging unit 606 capture an image to read the product code. If it is determined in step S101 that the reading of the product code has not been completed successfully, or after the processing of step S103, the mobile terminal 60 determines whether or not a registration failure situation has occurred based on the detection result regarding the registration failure situation.

The mobile terminal 60 may be adapted to detect a registration failure situation as follows.
First, when the mobile terminal 60 recognizes that the entire barcode of the product has been captured in the captured image, but a certain amount of time has passed without the reading of the barcode being completed successfully (successful), it determines that a registration failure situation classified as reading NG has occurred.

In addition, if the portable terminal 60 recognizes a product in a captured image but the barcode cannot be recognized at all and therefore reading of the barcode is not completed successfully, it determines that a registration failure situation classified as a pseudo scan has occurred.
Alternatively, as illustrated in FIG. 11 (B), if the reading of the barcode is not completed successfully because only a portion of the product's barcode can be recognized in the captured image, the mobile terminal 60 determines that a registration failure situation classified as a pseudo scan has occurred.
Alternatively, if the portable terminal 60 is able to recognize the barcode of a product in the captured image but the reading of the barcode is not completed successfully because the product is moving too quickly, it will determine that a registration failure situation classified as a pseudo scan has occurred.
Such a registration failure situation classified as a pseudo scan may indicate fraudulent activity in which a customer places an item in the shopping cart 80 simply by pretending to perform an operation to have the barcode of the item read.

3 can capture an image of the contents of a shopping basket 80 placed in a shopping cart 70. When the mobile terminal 60 detects that a new product has been added to the shopping basket 80 even if no attempt to read the barcode of the product is recognized in the captured image, the mobile terminal 60 determines that a registration failure situation classified as a non-scan input has occurred.
For example, a customer may move an already registered product within the shopping basket 80 in order to organize the contents of the shopping basket 80. The mobile terminal 60 may be configured to distinguish between such a movement of a product and the additional insertion of a new product corresponding to a registration failure situation as a non-scan insertion, based on the captured image.
The occurrence of such an incomplete registration situation, classified as a non-scan insertion, may indicate fraud, such as a customer placing an item in shopping cart 80 without having the barcode of the item read by mobile terminal 60.

Step S105: If it is determined in step S104 that a registration failure situation has occurred, the mobile terminal 60 assigns a classification number to the pending item corresponding to the current registration failure situation that has occurred, according to the classification of the registration failure situation detected in step S104. As an example, if it is determined in step S104 that a registration failure situation classified as a reading NG has occurred, the mobile terminal 60 assigns classification number 2 (see FIG. 8) to the corresponding pending item.

Step S106: The mobile terminal 60 acquires (extracts) an image corresponding to the currently occurring registration failure situation from among the images captured by the imaging unit 606. The image acquired by the mobile terminal 60 here is intended to allow the store clerk to visually recognize the currently occurring registration failure situation. For this reason, the format of the image acquired by the mobile terminal 60 (video, still image), the number of images, the time and period for which the images are acquired, etc. may be varied depending on the classification of the detected registration failure situation.

The captured image of the imaging unit 606 may be transmitted to the transaction management device 20, and the transaction management device 20 may acquire an image corresponding to the registration failure situation depending on the time when the registration failure situation occurred, the classification result, the image analysis result, etc.

Step S107: The mobile terminal 60 transmits the registration failure information to the transaction management device 20. The registration failure information includes the classification number of the reserved product assigned in step S105 and the captured image acquired in step S106.

Step S108: The mobile terminal 60 updates the product registration screen in response to receiving the registration completion notification transmitted from the transaction management device 20 in response to the transmission of the product registration information in step S102. The mobile terminal 60 may display a guide image WD2 as illustrated in FIG. 11(C), for example, at a timing in response to the update of the product registration screen in step S103.

Step S109: If it is determined in step S104 that a registration failure state has not occurred, or after the process of step S108, the mobile terminal 60 determines whether or not a settlement transition instruction operation has been accepted.
If the settlement transition instruction operation has not been received, the process returns to step S101.

Step S110: When the payment transfer instruction operation is received, the mobile terminal 60 converts the cart identification information into a two-dimensional code, for example, and displays the generated two-dimensional code on the display unit 604, for example, using a code display screen as shown in FIG. 10 (D).

Next, an example of a processing procedure executed by the transaction management device 20 will be described.
Step S201: The transaction management device 20 determines whether or not the product registration information transmitted from the mobile terminal 60 in step S102 has been received.

Step S202: When the transaction management device 20 receives the product registration information, it identifies the cart information indicated by the cart identification information included in the received product registration information.
Step S203: The transaction management device 20 updates the cart information identified in step S202. That is, the transaction management device 20 adds the registered product information based on the product code included in the product registration information received in response to step S201 to the identified cart information. As a result, one product is registered in the cart information.

Note that the update of the cart information in step S202 includes the registration of products among the reserved products that do not correspond to the registration failure status. In other words, the update of the cart information in step S202 includes the registration of products among the reserved products whose product codes have been read successfully. Among the reserved products, products whose product codes have been read successfully are NO-FILE, age verification products, medicines, and security tag products. In addition, although canceled products are not reflected in the process in the same figure, in response to a cancellation operation being performed on the mobile terminal 60, the transaction management device 20 updates the registered product information of the corresponding product in the cart information so that the corresponding product is managed as a canceled product.

Step S204: After processing step S203, the transaction management device 20 sends a registration completion notification to the mobile terminal 60.

Step S205: After the processing of step S204, or if it is determined in step S201 that the product registration information has not been received, the transaction management device 20 determines whether or not the registration failure information transmitted from the mobile terminal 60 in step S107 has been received. If the registration failure information has not been received, the process returns to step S201.

Step S206: When the transaction management device 20 receives the registration failure information, it identifies the cart information indicated by the cart identification information included in the received product registration information.
Step S207: The transaction management device 20 updates the cart information identified in step S206. In this case, the transaction management device 20 adds the registered product information including the classification number included in the registration failure information received in response to step S205 to the identified cart information. As a result, one reserved product (registered failure product) corresponding to the occurrence of the registration failure situation is registered in the cart information.

Step S208: After processing step S207, the transaction management device 20 sends a registration completion notification to the mobile terminal 60.

When the customer has finished registering all the products to be purchased, he or she performs a settlement transition instruction operation and has the settlement device 40 read the two-dimensional code displayed on the mobile terminal 60. After reading the two-dimensional code, the settlement device 40 requests the transaction management device 20 to calculate the subtotal amount, as shown as step S13 in FIG. 9. Here, in this embodiment, when requesting the calculation of the subtotal amount, the settlement device 40 requests settlement information to be used in the settlement process using the settlement screen.

In response to a request for settlement information, the transaction management device 20 identifies the corresponding cart information and generates settlement information based on the identified cart information. The settlement information includes registered product information in the cart information and information such as a subtotal amount according to the registered products. In addition, among the registered product information included in the settlement information, the registered product information corresponding to the insufficient registered product includes a captured image corresponding to the insufficient registered status. The transaction management device 20 transmits the generated settlement information to the settlement device 40.
When the settlement device 40 receives the settlement information, it displays a settlement screen on the display unit 405 that reflects the contents of the received settlement information.

FIG. 13 shows an example of a settlement screen displayed on the display unit 405 of the settlement device 40.
The checkout screen in the figure includes a registered product list area AR3, which displays a list of products registered by the mobile terminal 60.

If the registered products include pending products, the list items for the pending products are given priority and are arranged from the top of the left column. In the example shown in the figure, since the registered products include six pending products, a pending product list area AR31 is formed in the registered product list area AR3, containing list items for the six pending products from the top left. The list items in the pending product list area AR31 are designed to differ from the list items for other products, for example in terms of background color, etc.

In the list items in the registered product list area AR3, the product name of the corresponding product, the number of registered items, the price according to the number of registered items, etc. are shown. In addition, in the list items corresponding to reserved products, for example, the classification number, classification name, etc. of the reserved product may be shown.

A call button BT31 is provided for each list item of a reserved item in the reserved item list area AR31. Each call button BT31 shows the name of the reserved item category to which the corresponding item falls.

Furthermore, a customer monitor area AR32 is provided on the settlement screen. The customer monitor area AR32 is an area in which the captured image obtained by the imaging unit 416 is displayed as a monitor image in real time. The customer monitor area AR32 displays in real time the state of the customer's hands as he or she operates the settlement device 40 to settle the payment. By the settlement device 40 displaying such an image in the customer monitor area AR32, the customer becomes aware that his or her state is being imaged, which has the effect of deterring fraudulent payments by the customer.
In addition, the settlement device 40 may be configured to store (save) images used for display in the customer monitor area AR32.

Here, as shown in the figure, when the reserved item list area AR31 is formed in the registered item list area AR3 on the checkout screen, the store clerk needs to perform an operation to process the reserved items. When the checkout screen with the reserved item list area AR31 formed in this way is displayed, the checkout device 40 executes a store clerk call to have the store clerk perform an operation to process the reserved items. To call a store clerk, the checkout device 40 may, for example, cause a display such as a sign pole to be displayed in a predetermined manner, or send a notice of a store clerk call to a terminal device carried or monitored by the store clerk.
When a store clerk responds to the call and goes to the settlement device 40, the store clerk performs operations to process the reserved items in accordance with each list item shown in the reserved item list area AR31 on the displayed settlement screen.

To process the first reserved item, the store clerk operates the call button BT31 in the list item of the reserved item to be processed. In this case, the store clerk operates the call button BT31 of the list item in the first row corresponding to canned beer in the reserved item list area AR31. In response to the operation of the call button BT31, a reserved item processing screen for processing the reserved item corresponding to the list item including the operated call button BT31 is displayed on the checkout screen.
In addition, the entire list item of the reserved product to be processed may be made touch-operable.

For example, if a store clerk operates the call button BT31 of the first list item corresponding to a can of beer, which is an age verification product, a pending product processing screen for the can of beer will be displayed on the checkout screen.

FIG. 14 is an example of a reserved product processing screen AR4 for a can of beer, which is an age verification product.
The reserved product processing screen AR4 includes a product information area AR41, an input operation area AR42, a registration button BT41, and a cancel button BT42.

The product information area AR41 is an area that shows information about the corresponding reserved product. The product information area AR41 in the figure shows the product name, product code, price, registered quantity, etc. In the case of an age verification product, the content displayed in the product information area AR41 reflects the content of the registered product information of the corresponding product included in the payment information.

The input operation area AR42 is an area where a software keyboard for inputting product information in the product information area AR41 is arranged. The input operation area AR42 can be switched between a keyboard with a key arrangement corresponding to numeric input and a keyboard with a key arrangement corresponding to character input, depending on the operation. In addition, a preset call key labeled, for example, "Preset" is arranged in the input operation area AR42. By operating the preset call key, a preset key screen is displayed on which preset keys corresponding to each product are arranged. By operating a preset key arranged on the preset key screen, the store clerk can input information about the product corresponding to the operated preset key into the product information area AR41.

The registration button BT41 is a button for canceling the pending registration of the corresponding product and registering it as a product to be purchased by the customer.
The cancel button BT42 is a button operated to instruct the cancellation of registration of a product whose registration is in a pending state.

For example, the store clerk will check the customer's age, and if there are no problems, will operate the registration button BT41 to finalize the registration of the canned beer. This will cause the settlement device 40 to cancel the pending registration status of the canned beer and process it to be included in the registered products. Alternatively, if the store clerk determines that the canned beer should not be registered, he or she will operate the cancel button BT42 to cancel the registration of the canned beer, and collect the canned beer product.

In addition, for registered defective products, the call button BT31 in the list item of the reserved product list area AR31 displays, for example, "Unscanned." When the store clerk processes a registered defective product, he or she operates the call button BT31 that displays "Unscanned" in the list item of the registered defective product.

For example, in the case of a reserved item classified as "NO-FILE," in the initial display state of the reserved item processing screen AR4, no item information is shown in the item information area AR41. In this case, the store clerk can register the item by inputting a numeric string as the item code or a character string as the item name in the item information area AR41 by performing a numeric input operation or a character input operation in the input operation area AR42.
The product information registered in this manner is transmitted to transaction management device 20. Transaction management device 20 synchronizes the cart information by updating the registered product information of the corresponding product in the cart information from the "NO-FILE" state so that the information entered in product information area AR41 is reflected.

The pending item processing screen may also display the classification number, classification name, etc. of the corresponding pending item.

FIG. 15 shows an example of a reserved product processing screen AR4 for processing a product that has not been properly registered, which is displayed in response to operating the call button BT31 that displays "unscanned."
In the initial display state of the pending product processing screen AR4 corresponding to a registered unsuccessful product, information about the product has not yet been acquired, so no information about the product is presented in the product information area AR41 as shown in the figure.

The pending product processing screen AR4 corresponding to the unregistered product also includes a product information area AR41, an input operation area AR42, a registration button BT41, a cancel button BT42, and a captured image area AR43.

In the captured image area AR43, a captured image corresponding to the occurrence of the corresponding registration failure situation is displayed. This captured image is acquired by the mobile terminal 60 in step S106 of FIG. 12.

Specifically, the image in the captured image area AR43 in the figure shows a registration failure situation that is classified as a non-scan input. Two captured images G1 and G2 are arranged in the captured image area AR43 in the figure. Captured image G1 is an image that shows the state before an additional product is added to the shopping cart 80. Captured image G2 is an image that shows the state after captured image G1 when an additional product has been added to the shopping cart 80.

By looking at the captured image in captured image area AR43 in the figure, the store clerk can see that the unregistered product to be processed was added to the shopping cart 80 without having its barcode read. Also, in captured image G2, a frame FL is placed in the area of the added product, allowing the store clerk to easily identify what product was added without being registered.
The additionally inserted product may be identified and the frame FL may be arranged by either the mobile terminal 60 or the settlement device 40.

Furthermore, if the store clerk wishes to check the captured image in the captured image area AR43 in more detail, the store clerk can perform a captured image enlargement operation, for example, by tapping on an arbitrary position in the captured image area AR43.
In response to the captured image enlargement operation, the settlement device 40 places an enlarged captured image area AR431 on the reserved product processing screen AR4, as shown in Fig. 16. The enlarged captured image area AR431 can be erased (closed) in response to a predetermined operation.

Returning to FIG. 15 for explanation, the sales clerk confirms with the customer whether or not they wish to purchase the registered defective product. If the customer does not wish to purchase the product, the sales clerk operates the cancel button BT42 and collects the registered defective product. If the customer wishes to purchase the product, the sales clerk operates the input operation area AR42 to input product information into the product information area AR41, and then operates the registration button BT41 to register the product.

Moreover, the reserved product processing screen AR4 in FIG. 17 shows an example of a case where the registered defective product is classified as a "pseudo scan".
In the captured image area AR43 of the figure, one captured image is displayed. The captured image displayed in the captured image area AR43 shows that when the registration failure occurred, the image was captured with part of the barcode hidden by a finger. By looking at the captured image in the captured image area AR43, the store clerk can understand that the registration failure product to be processed is classified as a pseudo scan.

In this case, too, by performing an operation on the captured image area AR43, it is possible to display the enlarged captured image area AR431 on the reserved item processing screen AR4, as shown in FIG. 18.

In this case, the store clerk will also ask the customer whether or not they wish to purchase the defective registered product, and depending on whether or not they wish to purchase it, will register or cancel the product.

The pseudo scan also includes a situation where a product is held over the imaging range of the imaging unit 606 (the range in which the barcode can be read) but the barcode is not included in the imaging range. In this case, the captured image area AR43 may display an image in which part of the product is included in the imaging range, but the barcode is not included in the imaging range.

The pseudo scan also includes a situation in which a product passes through the imaging range of the imaging unit 606 at a speed that makes it impossible to read the product code. In this case, the captured image area AR43 may display a still image captured when the product passes through the imaging range, or a moving image captured when the product passes through the imaging range.

With reference to the flowchart in Figure 19, an example of the processing procedure executed by the settlement device 40 in relation to the display of the reserved item processing screen will be described.

Step S301: While the settlement screen is displayed, the settlement device 40 waits for an operation to be performed to specify a reserved item to be processed (reserved item specification operation).

Step S302: When the reserved product designation operation is performed, the settlement device 40 determines whether or not the product designated by the reserved product designation operation is a registered insufficient product.
In this embodiment, the reserved goods whose registered goods information, which is included in the settlement information transmitted from the transaction management device 20, includes a captured image are limited to the unregistered goods. Therefore, in the process of step S302, the settlement device 40 may determine whether or not the registered goods information of the goods specified by the reserved goods specification operation includes a captured image.

Step S303: If step S302 determines that the product is a registered defective product, the settlement device 40 generates a pending product processing screen including an image capture area AR43 for the registered defective product designated as the product to be processed. The settlement device 40 may obtain the captured image of the captured image area AR43 from the registered product information of the product in the settlement information. The settlement device 40 displays the generated pending product processing screen.

Step S304: On the other hand, if step S302 determines that the reserved item is not a registered defective item, the settlement device 40 generates a reserved item processing screen that does not include the captured image area AR43 for the reserved item specified as the item to be processed. The settlement device 40 displays the generated reserved item processing screen.

<Modification>
A modification of this embodiment will now be described.

[First Modification]
In the above embodiment, one or two captured images (FIGS. 15 and 17) are presented in the captured image area AR43 that is displayed when the reserved product is a non-registered product. However, the number of captured images presented in the captured image area AR43 may be three or more.

Figure 20 shows an example in which three captured images G11, G12, and G13 are arranged in the captured image area AR43 of the pending product processing screen AR4. The pending product processing screen AR4 in the figure shows an example in which the registered defective product to be processed is classified as a non-scan input.

The captured images G11, G12, and G13 in the figure show the following. In the captured image area AR43, the captured images G11, G12, and G13 are arranged in this order from left to right. The arrangement order of the captured images G11, G12, and G13 corresponds to the time lapse in which the images were captured.
The first captured image G11 is an image of the inside of the shopping cart 80 before the target non-registered product is added. The second captured image G12 is an image of the process in which the captured image G11 changes to the next, third captured image G13. The third captured image G13 is an image of the state in which the target non-registered product has been added to the shopping cart 80.
The second image G12 may be extracted, for example, from among the images that include the product to be inserted, or may be an image taken at a specified time, such as the exact midpoint between the times at which the images G11 and G13 were taken.
Also in the captured image G13, a frame FL is placed at the portion of the product that has been added to the shopping cart 80.

[Second Modification]
In the above embodiment, the display surface of the display unit 405 of the settlement device 40 is horizontally long. However, the display surface of the display unit 405 of the settlement device 40 may be vertically long.
Fig. 21(A) shows an example of a settlement screen displayed on display unit 405 with a vertically long display surface corresponding to this modified example. In this figure, the same parts as in Fig. 13 are given the same reference numerals. As can be seen by comparing this figure with Fig. 13, the information presented as the settlement screen is similar, but the layout of the information presented on the settlement screen is different in accordance with the fact that the size of the display surface is vertically long.
However, in the checkout screen of Figure 21 (A), a pending product list area AR31 containing list items of pending products is formed in the upper left corner of the registered product list area AR3, and the background color, etc. are different from list items other than pending products, which is the same as the checkout screen of Figure 13.

In the case of this modified example, for example, the reserved item list area AR31 in FIG. 21(A) can be enlarged and displayed as an enlarged reserved item list area AR311 as shown in FIG. 21(B). The operation to display the enlarged reserved item list area AR311 may be an operation of touching any position within the reserved item list area AR31. Alternatively, it may be an operation of touching the reserved item count area AR33 located on the checkout screen in FIG. 21(A).

By displaying the enlarged reserved item list area AR311 in this manner, the store clerk can easily see the list items of the reserved items and easily understand what types of reserved items are available.
Also, for example, depending on the arrangement position of the display unit 405 in the settlement device 40, the position of the display unit 405 may be high, and particularly when the display surface of the display unit 405 is vertically long, it may be difficult to operate the upper side of the display surface of the display unit 405. Therefore, in the example of Fig. 21 (B), the enlarged reserved item list area AR311 is arranged on the lower side of the display surface. This makes it easier for the store clerk to operate the enlarged reserved item list area AR311.

When a store clerk performs a process corresponding to a pending item, the clerk operates the call button BT31 located on the list item of the pending item to be processed among the list items in the expanded pending item list area AR311 displayed as shown in Figure 21 (B).
In response to the operation of the call button BT31, as shown in Fig. 21(C), a reserved item processing screen AR4 is displayed in place of the enlarged reserved item list area AR311. The reserved item processing screen AR4 displayed in this manner targets the reserved items indicated by the list items of the operated call button BT31.

In addition, if the pending product to be processed is a product that has not been registered, the settlement device 40 displays the captured image area AR43 on the same screen as the pending product processing screen AR4, as shown in the same Figure 21 (C).
In addition, the figure shows an example in which one image is displayed in the captured image area AR43. However, even in the case of such a vertically long screen, a predetermined number of captured images may be displayed in the captured image area AR43 according to the classification of the registration-deficient product (registration-deficient situation) as shown in Figs. 15 and 20.

For example, in the example of Fig. 15, the captured image area AR43 is arranged in the reserved product processing screen AR4. In contrast, in Fig. 21(C), since the display surface of the display unit 405 is vertically long, the captured image area AR43 is not included in the reserved product processing screen AR4.
As a result, the captured image area AR43 can be enlarged and arranged to a larger size than when it is arranged in the reserved item processing screen AR4, making it easier to see the captured image. Also, as shown in Fig. 21 (C) , on a vertically long display surface, the captured image area AR43 is arranged on the upper side, and the reserved item processing screen AR4 is arranged on the lower side. This makes it easier for the store clerk to operate the reserved item processing screen AR4.

[Third Modification]
In the above embodiment, an example was given in which the captured image area AR43 is placed on the pending product processing screen which processes pending products that are not registered defective products, and the captured image area AR43 is not placed on the pending product processing screen for pending products that are not registered defective products.
However, in this modified example, the captured image area AR43 may be arranged on the pending product processing screen for pending products that are not defectively registered products. In this case, the mobile terminal 60 may transmit product registration information including captured images of registered products whose barcodes have been successfully read.

[Fourth Modification]
The captured image area AR43 may be displayed as an independent screen different from the reserved item processing screen, for example, depending on the store clerk's operation on the settlement device 40.

[Fifth Modification]
In addition, when the transaction start instruction button BT10 arranged on the start screen of FIG. 10(A) is operated to start product registration, the mobile terminal 60 may display a warning screen informing the customer in advance that "images at the time of product registration may be displayed on the settlement device". By displaying such a warning screen, for example, fraudulent product registration can be deterred. Alternatively, instead of the warning screen, the mobile terminal may display a consent screen to confirm the customer's consent to "images at the time of product registration may be displayed on the settlement device". In the consent screen, buttons for accepting an operation to confirm intention, such as a "Yes" button and a "No" button, are arranged. If the "Yes" button is operated, the customer may be treated as a customer who does not commit fraud, and the cancellation may be confirmed as it is without registering the canceled product as a reserved product.

[Sixth Modification]
In the above embodiment, a registration failure situation where the barcode is not read can be treated as a product registration error. In this modified example, in addition to a registration failure situation, a situation where, for example, the barcode is successfully read but the target product cannot be added to the shopping cart 80 may also be determined as a product registration error. Note that, for example, rice over a certain weight or canned drinks in packs of six are often placed at the bottom of the cart. Therefore, for such products, even if it is detected that they cannot be added to the shopping cart 80, it may be determined that there is no product registration error.

[Seventh Modification]
For example, due to a communication error or a communication speed limit set in the mobile terminal 60, it may be impossible to transmit the captured image to the transaction management device 20 when the registration failure information is transmitted in step S107 in Fig. 12. In such a case, the transaction management device 20 can receive the registration failure information indicating the classification of the registration failure situation, even though it does not include the captured image, and therefore can register the registered product information indicating the classification of the registration failure situation in the cart information.
As a result, the settlement device 40 can use settlement information based on the cart information to display, for example, a pending product processing screen for the non-registered product as shown in Fig. 15, although no captured image is displayed in the captured image area AR43. In this case, a message that can inform the store clerk that the product is a non-registered product, such as "Are there any products that you forgot to scan?", may be displayed in the blank area of the pending product processing screen where the captured image area AR43 is not displayed.

[Eighth Modification]
The occurrence of a registration failure situation may be displayed on a terminal (store operating terminal) operating in the store other than the settlement device 40, such as the monitoring device 50, for example.
For example, the monitoring device 50, which is one of the store operating terminals, can monitor the status of each transaction in the store by communicating with, for example, the transaction management device 20. Thus, when a pending product is registered, the monitoring device 50 may obtain information identifying the customer to whom the pending product corresponds, and product information and price information of the target product at the time of the registration of the pending product, and display the obtained information. Then, for example, when a store clerk performing monitoring duties using the monitoring device 50 confirms that there is no problem with the displayed product, the monitoring device 50 may be able to perform an operation such as canceling the registration of the product as a pending product and re-registering it as a normal product.

[Ninth Modification]
The imaging device used in response to product registration is not particularly limited as long as it is capable of reading product barcodes, is capable of capturing images of the contents of the shopping basket 80, etc., and is capable of outputting captured images to the mobile terminal 60, etc. Such an imaging device may be capable of capturing images of the inside of a customer's own bag, instead of the shopping basket 80 provided by the store, for example.

[Tenth Modification]
The configuration of this embodiment may be applied to the extent possible even when a customer registers a product by holding the mobile terminal 60 in his/her hand without setting it in the shopping cart 70. In such a case, for example, the mobile terminal 60 can detect a read NG or a pseudo scan as a registration failure situation, and output a captured image corresponding to the detected registration failure situation.

[Eleventh Modification]
In the above embodiment, a part of the processing executed by the mobile terminal 60 may be executed by a higher-level device such as the transaction management device 20 connected via communication to the mobile terminal 60. In the above embodiment, a part of the processing executed by the settlement device 40 may be executed by a higher-level device such as the transaction management device 20 connected via communication to the settlement device 40.

<Overview of the embodiment>
[Technical field]
The present invention relates to a product sales data processing system.
[Background Art]
There is known a system in which a customer himself/herself registers an item using a mobile terminal equipped with a camera (see, for example, Japanese Patent Application Laid-Open No. 2003-233663).
[Prior Art Literature]
[Patent Documents]
[Patent Document 1] JP 2018-147252 A [Summary of the Invention]
[Problem to be solved by the invention]
However, because customers themselves register products using a mobile terminal, there is the possibility of problems occurring, such as customers thinking they have scanned the barcode of a product but actually not being able to do so, or customers intentionally putting products into their shopping cart without having the barcode scanned.
The present invention has been made in view of the above circumstances, and has an object to make it possible to appropriately deal with problems that arise when customers themselves register products.

[Means for solving the problems]
(1) One aspect of this embodiment that solves the above-mentioned problems is a product sales data processing system (1) in which a product is registered using a mobile terminal device (e.g., mobile terminal 60) and a settlement device (40) performs settlement processing corresponding to the product registered by the mobile terminal device, the product sales data processing system including: an imaging means (e.g., imaging unit 606) that is configured to be able to capture an image of a situation related to the registration of a product by a customer; a detection means that detects a predetermined registration failure situation in which the registration of a product is deemed to be incomplete while the product is being registered in accordance with a transaction; and a display control means that, when settlement is performed for the product registered in accordance with the transaction at the settlement device, displays an image corresponding to the registration failure situation detected by the detection means (e.g., an image in an image area AR43) based on the image obtained by the imaging means.

According to the above configuration, if a registration failure occurs while a customer is registering a product using the mobile terminal 60, an image of the registration failure can be displayed. This makes it possible to, for example, prevent registration failures (e.g., prevent fraudulent acts regarding product registration) and appropriately deal with products that correspond to registration failures, thereby making it possible to appropriately deal with problems that arise when customers register products themselves.

(2) One aspect of this embodiment is the product sales data processing system described in (1), in which the display control means displays, as a classification of a registration failure situation, a plurality of captured images corresponding to a predetermined period before and after the product is inserted into the product insertion section (e.g., shopping cart 80) without the product information being read.

According to the above configuration, when the classification of the registration failure status is non-scan input, it is possible to display an image capturing the contents of the shopping basket 80 from before the target product is input until it is input. This allows the state transition of the contents of the shopping basket 80 when the product is input without being scanned to be accurately understood.

(3) One aspect of this embodiment is the product sales data processing system described in (2), which displays a portion of a specific captured image among the multiple captured images that differs from the other captured images.

According to the above configuration, a frame FL showing the part of the additionally added product can be arranged in an image of the additionally added product among the multiple images displayed when the classification of the registration failure status is non-scan input. This allows the additionally added product to be immediately identified.

(4) One aspect of this embodiment is a product sales data processing system according to any one of (1) to (3), in which the display control means displays captured images having different captured objects according to the classification of the registration failure situation detected by the detection means.

According to the above configuration, by displaying a captured image of an imaging target according to the classification of the registration failure situation, it is possible to display an appropriate captured image as evidence according to the registration failure situation.

(5) One aspect of this embodiment is a product sales data processing system according to any one of (1) to (4), in which the display control means displays an image captured when an operation to read the product information was being performed in the case of a registration failure status classified as an unsuccessful reading of the product information.

According to the above configuration, for example, when the classification of the registration failure state is a reading NG or a pseudo scan, it is possible to display an image of a product that was within the imaging range of the imaging unit 606 when the registration failure state was determined. This makes it possible to identify the product whose barcode should have been read in a registration failure state such as a reading NG or a pseudo scan.

(6) One aspect of this embodiment is a product sales data processing system according to any one of (1) to (5), in which the display control means displays a captured image corresponding to the registered defective product together with a screen (e.g., a reserved product processing screen AR4) for processing the reserved product as a registered defective product at the settlement device.

With the above configuration, if the reserved item is a registered defective item, the captured image area AR43 can be displayed together with the reserved item processing screen AR4 that is displayed so that the store clerk can process the reserved item. This allows the store clerk to properly process the reserved item as a registered defective item.

(6) One aspect of this embodiment is a mobile terminal device (e.g., mobile terminal 60) capable of registering products, the mobile terminal device including: imaging means capable of capturing images of the status of product registration by a customer; detection means for detecting a predetermined registration failure status in which product registration is deemed to be unsuccessful while a product is being registered according to a transaction; and display control means for displaying product information of the registered product and an image captured by the imaging means (e.g., a product registration screen) and displaying information about the detected registration failure status (e.g., a guide image WD2) in response to the detection of the registration failure status by the detection means.

According to the above configuration, when a registration failure occurs, a display related to the registration failure is displayed on the mobile terminal 60. This allows the customer to understand that a registration failure has occurred. Furthermore, this has a deterrent effect on customers who attempt to commit fraud.

In addition, a program for implementing the functions of the above-mentioned transaction management device 20, mobile terminal 60, settlement device 40, management device 10, registration settlement device 30, etc. may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing of each of the above-mentioned devices. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer system" may include multiple computer devices connected via a network including the Internet, WAN, LAN, dedicated lines, and other communication lines. In addition, the "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. In addition, the recording medium also includes a recording medium provided inside or outside and accessible from a distribution server to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format executable by the terminal device. In other words, the format in which the program is stored in the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a format executable by the terminal device. The program may be divided into multiple parts, downloaded at different times, and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a memory that holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted over a network. The program may also be a program for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

1 Product sales data processing system, 10 Management device, 20 Transaction management device, 40 Settlement device, 60 Mobile terminal, 70 Shopping cart, 80 Shopping basket

Claims (7)

A payment device in a product sales data processing system, which registers products using a registration terminal and performs a payment process corresponding to the products registered by the registration terminal,
a display control means for displaying an image corresponding to a registration failure situation detected by a detection means for detecting a predetermined registration failure situation in which the registration of the product is deemed to be unsuccessful during the registration of the product in accordance with one transaction, the image being obtained by an imaging means provided in the registration terminal so as to be able to image a situation related to the registration of the product by a customer when settling a payment for the product registered by the registration terminal in accordance with one transaction ;
The display control means displays captured images with different imaging targets according to the classification of the registration failure state detected by the detection means.
Clearing device. A payment device in a product sales data processing system, which registers products using a registration terminal and performs a payment process corresponding to the products registered by the registration terminal,
a display control means for displaying an image corresponding to a registration failure situation detected by a detection means for detecting a predetermined registration failure situation in which the registration of the product is deemed to be unsuccessful during the registration of the product in accordance with one transaction, the image being obtained by an imaging means provided in the registration terminal so as to be able to image a situation related to the registration of the product by a customer when settling a payment for the product registered by the registration terminal in accordance with one transaction;
The display control means displays a captured image when an operation for reading the product information is being performed, when the registration failure state is classified as an unsuccessful reading of the product information.
Clearing device. The settlement device described in claim 1 or 2, wherein the display control means displays a plurality of captured images corresponding to a predetermined period before and after the product is inserted into the product insertion section when the product is classified as having been inserted into the product insertion section without its information being read, indicating that the product has not been registered properly. The settlement device according to claim 3 , wherein the display control means performs a display showing a portion of a predetermined captured image of the plurality of captured images that differs from other captured images. The settlement device according to claim 1 , wherein the display control means displays a screen for processing reserved products as registered defective products at the settlement device, together with an image captured corresponding to the registered defective product. A computer as a settlement device in a product sales data processing system in which a product is registered by a registration terminal and a settlement device performs a settlement process corresponding to the product registered by the registration terminal,
A display control means for displaying captured images obtained by an imaging means provided in the registration terminal so as to be able to image a situation related to the registration of a commodity by a customer when settling a payment for a commodity registered by the registration terminal in accordance with one transaction, the display control means for displaying captured images corresponding to a registration failure situation detected by a detection means for detecting a predetermined registration failure situation in which the registration of a commodity is deemed to be unsuccessful while the commodity is being registered in accordance with one transaction, and the display control means for displaying captured images with different imaging targets according to the classification of the registration failure situation detected by the detection means
A program to function as a A computer as a settlement device in a product sales data processing system in which a product is registered by a registration terminal and a settlement device performs a settlement process corresponding to the product registered by the registration terminal,
A display control means for displaying an image corresponding to a registration failure situation detected by a detection means for detecting a predetermined registration failure situation in which the registration of the product is deemed unsuccessful while the product is being registered in accordance with a transaction, the image being obtained by an imaging means provided in the registration terminal so as to be able to image a situation related to the registration of the product by a customer when settling the payment for the product registered by the registration terminal in accordance with a transaction, the display control means for displaying an image corresponding to a registration failure situation detected by a detection means for detecting a predetermined registration failure situation in which the registration of the product is deemed unsuccessful, the display control means displaying an image captured when an operation for reading the product information was being performed in the case where the registration failure situation is classified as a failure to read the product information
A program to function as a

Images