JP7575038B2 - Product sales data processing system, mobile terminal, and program - Google Patents

Description

The present invention relates to a product sales data processing system, a mobile terminal, and a program.

There is a known system in which customers themselves register products using a mobile terminal equipped with a camera (see, for example, Patent Document 1).

JP 2018-147252 A

However, various problems can arise when customers scan barcodes to register their products.

The present invention was made in light of these circumstances, and aims to make it possible to appropriately deal with problems that arise when customers register products themselves.

One aspect of the present invention that solves the above-mentioned problems is a product sales data processing system that registers products using a mobile terminal operated by a customer of a store, and includes a reading means for reading a product code, an acquisition means for acquiring product information corresponding to the product code read by the reading means, a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer, and a display means for displaying candidate products in a selectable manner when the detection means detects the read failure state, and the acquisition means acquires product information of the candidate product selected by the customer's selection operation.

1 is a diagram illustrating an example of the overall configuration of a product sales data processing system according to an embodiment of the present invention. 1 is a diagram showing an example of the state near the entrance of a store in which a product sales data processing system according to an embodiment of the present invention has been introduced. 1A to 1C are diagrams illustrating examples of how a mobile terminal is set in a shopping cart according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a settlement device according to the present embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a mobile terminal according to the present embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a transaction management device according to the present embodiment. 4A to 4C are diagrams illustrating an example of customer information, store information, and cart information according to the embodiment. FIG. 2 is an explanatory diagram illustrating classification of reserved products according to the embodiment. A sequence diagram showing an example of processing procedures executed by a mobile terminal, a settlement device, and a transaction management device according to this embodiment in response to the accounting of one transaction in self-service operation. 11A to 11C are diagrams showing examples of a start-up screen, a product registration initial screen, a product registration update screen, and a code display screen according to the present embodiment. 11A to 11C are diagrams illustrating an example of a registered product presentation screen, a product registration screen, and a guide image according to the embodiment. 6 is a flowchart illustrating an example of an operation of the mobile terminal according to the embodiment. 6 is a flowchart illustrating an example of an operation of the mobile terminal according to the embodiment. 6 is a flowchart illustrating an example of an operation of the mobile terminal according to the embodiment. 5 is a flowchart showing an example of the operation of the transaction management device according to the present embodiment. 11A and 11B are diagrams showing an example of a re-read instruction screen and a preset key screen according to the embodiment; 11A and 11B are diagrams showing an example of a re-read instruction screen and a preset key screen according to the embodiment; FIG. 13 is a diagram showing an example of a settlement screen according to the embodiment. FIG. 13 is a diagram showing an example of a reserved product processing screen according to the embodiment; FIG. 13 is a diagram showing an example of a reserved product processing screen according to the embodiment; 1 is a diagram showing an example of an enlarged captured image area according to the embodiment; FIG. 13 is a diagram showing an example of a reserved product processing screen according to the embodiment; 1 is a diagram showing an example of an enlarged captured image area according to the embodiment; 13 is a flowchart showing an example of a processing procedure executed by the settlement device according to the embodiment in relation to displaying a reserved product processing screen. FIG. 11 is a diagram showing an example of a captured image area according to a first modified example of the embodiment. FIG. 11 is a diagram showing an example of a settlement screen according to a second modified example of the present embodiment.

<Embodiment>
[Example of overall configuration of product sales data processing system]
FIG. 1 shows an example of the overall configuration of a product sales data processing system 1 according to the present embodiment.
The product sales data processing system 1 in the figure includes a store controller 10, a transaction management device 20 (cloud server), a registration and settlement device 30, a settlement device 40, a transaction status management device 50 (e.g., a personal computer, etc.), and a mobile terminal 60. The product sales data processing system 1 in the figure can be introduced in various stores, etc. (supermarkets, convenience stores, drug stores, home centers, home appliance retailers, etc.).

In addition, although one registered settlement device 30 is illustrated in the figure, the number of registered settlement devices 30 installed in one store may be two or more. Also, although two settlement devices 40 are illustrated in the figure, the number of settlement devices 40 in one store may be one or three or more. Also, although one transaction status management device 50 is illustrated in the figure, the number of transaction status management devices 50 in one store may be two or more. Also, there may be two or more store controllers 10 in one store.

The product sales data processing system 1 in the figure is capable of handling both a store clerk-assisted operation in which a store clerk uses a registration and settlement device 30 to register products, and a self-service operation in which a customer registers products using a mobile terminal 60 and then uses a settlement device 40 to settle the payment.
In the following explanation, self-operation will be mainly described.

The store controller 10, the registration settlement device 30, the settlement device 40, and the transaction status management device 50 are installed in the store. The store controller 10, the registration settlement device 30, the settlement device 40, and the transaction status management device 50 are connected to each other so as to be able to communicate with each other via a LAN 19 (which may be wired or wireless). The store controller 10 is capable of communicating with the transaction management device 20.

The store controller 10 is a device that mainly manages various devices installed in a store. For example, the store controller 10 has a communication function with external devices, and receives product files (product masters) from external devices (e.g., a headquarters server not shown, which may be the transaction management device 20) and supplies them to the registered settlement device 30, etc., and receives transaction information from the settlement device 40, etc. and transmits it to the external device (e.g., the transaction management device 20, etc.).

The transaction management device 20 is a server that provides various services. For example, the transaction management device 20 manages various information corresponding to self-operation (e.g., transaction information (cart information), etc.).

The registration and settlement device 30 is a device that executes the registration process and the settlement process. The registration and settlement device 30 is used by a store clerk (the registration process and the settlement process are executed by the operation of the store clerk).

The settlement device 40 is a device that executes settlement processing. The settlement device 40 is used by a customer. That is, the settlement device 40 executes settlement processing in response to operations by the customer.
In addition, when the payment includes items whose processing is temporarily suspended for confirmation by a store clerk (hereinafter referred to as "pending items"; details will be described later), a store clerk will be called and the store clerk (or the customer) will operate the payment device 40 to confirm the pending items.

The transaction status management device 50 is a device that mainly manages (or monitors) various devices within the LAN 19. The transaction status management device 50 manages the operating status of other devices (e.g., the registered settlement device 30, the settlement device 40, etc.), by displaying various information on the other devices and outputting various commands to the other devices.

The mobile terminal 60 is a terminal used by a customer under self-operation. In this embodiment, the mobile terminal 60 is used while attached to a shopping cart 70 under self-operation.

The mobile terminal 60 has a communication function, an imaging function (camera), and also a recognition function that recognizes various codes (barcodes, two-dimensional codes, etc.). For example, the mobile terminal 60 recognizes the barcode attached to a product. In other words, the mobile terminal 60 reads product identification information (product code). The mobile terminal 60 may also extract feature points in an image captured by the imaging function (an image acquired as a through image) and recognize objects (various codes).

The mobile terminal 60 is a smartphone or the like owned by the customer. The mobile terminal 60 may be owned by the store and loaned to the customer when shopping. Although two mobile terminals 60 are shown in the figure, the number of mobile terminals 60 used for product registration in the store varies depending on the number of customers.

A shopping application having a function for registering products in accordance with the self-service operation of this embodiment is installed on the mobile terminal 60.

Figure 2 shows an example of the state near the entrance of a store that has introduced the product sales data processing system 1 of this embodiment. As shown in Figure 2, near the entrance (or the entrance if the entrance and exit are separate), a rental shopping cart 70, rental shopping baskets 80, and sales shopping bags 90 are prepared. When a customer visits the store, they place the shopping basket 80 in the shopping cart 70, place the shopping bag 90 in the shopping basket 80, etc., and proceed inside the store. At this time, if the customer wishes to do their own shopping, they place the mobile terminal 60 in the shopping cart 70.

The customer may bring the shopping bag 90 that they purchased this time as their own bag the next time they visit the store, or may bring a different bag to use as their own bag. They may also purchase a plastic bag.

Although not shown in the figure, the shopping cart 70 is also provided with a bag for reserved items (described below) in addition to the shopping basket 80.

[Example of setting a mobile terminal in a shopping cart]
An example of how the mobile terminal 60 is set in the shopping cart 70 will be described with reference to Fig. 3. Fig. 3(A) is a side view of the portion of the shopping cart 70 where the mobile terminal 60 is set. Fig. 3(B) is a top view of the shopping cart 70.

As shown in FIG. 3(A), a holder 71 for setting the mobile terminal 60 is attached to the shopping cart 70. The customer sets the mobile terminal 60 in the holder 71. When the mobile terminal 60 is set in the holder 71, as shown in FIG. 3(B), it is in a state where an image of the entire shopping bag 90 or my bag can be captured by the imaging unit 606 (see FIG. 4(B)).

[About the settlement device]
Fig. 4 is a diagram for explaining the settlement device 40. Fig. 4(A) is a diagram showing an example of the appearance of the settlement device 40 (a front view seen from the customer side). Fig. 4(B) is a diagram showing an example of the configuration of the settlement device 40. In Fig. 4(A) and Fig. 4(B), the same parts are given the same reference numerals.

Below, an example of the configuration of the settlement device 40 shown in FIG. 4(B) will be described with reference to FIG. 4(A). The settlement device 40 comprises a CPU 401, a ROM 402, a RAM 403, a hard disk 404, a display unit 405, a scanner unit 406, a card payment unit 408, a change dispenser 409, a printing unit 413, an audio output unit 414, a communication unit 415, and an imaging unit 416 (camera). These can communicate with each other via a bus. The settlement device 40 may further include a key operation unit and an alarm unit (for example, a lamp on a sign pole, etc.).

The CPU 401 is a central processing unit, and controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 .
The ROM 402 is a read-only memory, and stores various types of information used by the CPU 401, including programs.

RAM 403 is a read/write memory, and stores various information as a main storage device. For example, RAM 403 stores information read from ROM 402 or hard disk 404, information acquired from the outside, information generated during processing, etc.

The hard disk 404 stores various information as an auxiliary storage device. For example, the hard disk 404 may store programs executed by the CPU 401 instead of the ROM 402. Also, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information obtained from the outside, information generated during processing, etc. Note that a storage device such as an SSD (Solid State Drive) may be provided instead of the hard disk 404.

The display unit 405 is a touch display that displays various information and accepts various inputs.

The scanner unit 406 optically reads various information. For example, the scanner unit 406 scans a code (barcode, two-dimensional code, etc.) displayed on the display unit of the mobile terminal 60 and reads information necessary for the settlement process. The scanner unit 406 may also scan a code printed on an account ticket (registered trademark) issued by another device (for example, a registered settlement device 30) and reads information necessary for the settlement process. The scanner unit 406 may also scan a code attached to a product and read the product code. The scanner unit 406 may also scan a code attached to a store clerk's name tag and read the store clerk code.

The card payment unit 408 is a payment mechanism that uses various cards (credit cards, prepaid cards such as transportation cards, cash cards for debit payments, point cards for point payments, etc.). The card payment unit 408 is equipped with a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient that it is equipped with at least the card recognition unit. The card recognition unit may recognize various cards (e.g., point cards, membership cards, shareholder cards, etc.) for the purpose of granting benefits, discounts, etc.

The change dispenser 409 (cash settlement unit) is a cash settlement mechanism that has an input port for banknotes and coins and an output port for banknotes and coins, calculates the amount of money inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. When banknotes or coins are inserted into the input port, they are detected by a sensor (detecting that they have been inserted, detecting the number of coins by denomination, etc.).

The printing unit 413 prints and issues various media (receipts, bills, etc.).
The audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like.
The communication unit 415 transmits and receives information between other devices (the store controller 10, the registered settlement device 30, the other settlement devices 40, the transaction status management device 50, etc.).

The imaging unit 416 is configured to capture, for example, an image of the hands of a customer operating the change machine.

[Registered payment devices]
The registration and settlement device 30 may be configured to have a display unit and a scanner unit on each side (clerk side, customer side) and a settlement unit (card settlement unit, cash settlement unit) on one side (customer side). The registration and settlement device 30 configured in this manner may be a device that can be switched to any of a plurality of operation modes (a mode in which the registration process is executed by the operation of the clerk and the settlement process is executed by the operation of the customer, a mode in which the registration process is executed by the operation of the clerk and the customer and the settlement process is executed by the operation of the customer, etc.) depending on, for example, the operating status. Also, if the mode in which the settlement process is executed by the operation of the customer, the registration and settlement device 30 can be used for self-operation. That is, the customer can make a transaction by having the mobile terminal 60 execute the registration process and have the registration and settlement device 30 execute the settlement process.
However, in the following explanation, an example will be given in which the settlement device 40 is used for self-service operation.

[Example of configuration of a mobile terminal]
5 shows an example of the configuration of the portable terminal 60. The portable terminal 60 in the figure includes a CPU 601, a storage unit 602, a RAM 603, a display unit 604, an operation unit 605, an imaging unit 606, and a network-compatible communication unit 607.

The CPU 601 is a central processing unit that controls the operation of the mobile terminal 60 by reading and executing programs stored in the storage unit 602 .
The storage unit 602 is an auxiliary storage device for the CPU 601, and stores various types of information used by the CPU 601, including programs.
The RAM 603 is a main storage device for the CPU 601 .

The display unit 604 displays an image according to the display control of the CPU 601. The display unit 604 may be configured as a touch panel that allows operations to be performed on the screen.
The operation unit 605 collectively indicates various operators provided on the mobile terminal 60 and various input devices connected to the mobile terminal 60 .

The imaging unit 606 captures images under the control of the CPU 401. In addition, the imaging unit 606 in this embodiment can also function as a part that reads two-dimensional codes in a code reader that processes two-dimensional codes.

The network-compatible communication unit 607 is a part that communicates wirelessly via a network. The network-compatible communication unit 607 enables the mobile terminal 60 to communicate with the transaction management device 20.

[Example of transaction management device configuration]
6 shows an example of the configuration of transaction management device 20. Transaction management device 20 in the figure includes a CPU 201, a storage unit 202, a RAM 203, and a network-compatible communication unit 204.
The CPU 201 is a central processing unit, and controls the operation of the transaction management device 20 by reading and executing programs stored in the memory unit 202 .
The storage unit 202 is an auxiliary storage device for the CPU 201, and stores various types of information used by the CPU 201, including programs.
The RAM 203 is a main storage device for the CPU 201 .
The network-compatible communication unit 204 is a part that performs communication via a network.

The memory unit 202 of the transaction management device 20 stores customer information, store information, and cart information.

[Information stored in the transaction management device]
The customer information stored in the memory unit 202 is information for managing individual customers. The transaction management device 20 generates customer information when a customer is registered (a customer may be considered to have been registered when the customer information of that customer is stored). The transaction management device 20 also updates the customer information as appropriate based on cart information, etc. The transaction management device 20 may refer to the cart information at a specified time every day, for example, and update the customer information.

7A shows an example of customer information stored in the memory unit 202 of the transaction management device 20. The customer information in the figure includes customer identification information, customer name, customer registration date, product cancellation information, customer rank, number of points, etc.
The customer identification information is identification information that uniquely identifies a customer.
As the customer identification information (or instead of the customer identification information), for example, application identification information that is uniquely (e.g., sequentially) assigned to each installed shopping application may be used. Also, as the customer identification information (or instead of the customer identification information), terminal identification information unique to the mobile terminal 60 in which the shopping application is installed may be used. The terminal identification information may be hardware identification information uniquely assigned to the hardware of the mobile terminal 60, such as a telephone number assigned to a customer who is a user of the mobile terminal 60, or a MAC (Media Access Control) address.
The customer name is the customer's name or nickname, etc. The customer registration date is the date and time when the customer was registered. Product cancellation information is information regarding cancellation of a product after it has been registered. The customer rank is a rank according to the customer's purchasing history. When generating customer information for a new customer, the customer identification information, customer name, and customer registration date are generated, but since this is before the start of actual trading (product registration), other information (product cancellation information, etc.) is not generated.

The transaction management device 20 generates and stores customer identification information, for example, when a customer is registered (for example, when the mobile terminal 60 downloads or installs a shopping application from an external device (for example, a specified server that provides general applications, the transaction management device 20) to use a shopping service provided by the transaction management device 20). In addition, the transaction management device 20 acquires information entered into the name field of a registration form (input form) using the mobile terminal 60, for example, when a customer is registered, and stores it as the customer name. In addition, the transaction management device 20 acquires the current date and time when the customer is registered, and stores it as the customer registration date.

In addition, instead of or in addition to storing the customer information in its own memory, the transaction management device 20 may store some or all of the customer information in another device (such as a file server accessible to the transaction management device 20).

7B shows an example of store information stored in the memory unit 202 of the transaction management device 20. The store information in the figure includes store identification information, store name (branch name), store specific information 1, and store specific information 2.
The store identification information is identification information that uniquely identifies a store. The store identification information shown in Fig. 7B is composed of a store (trade name) or company code and a branch code.
The store name is the name of the store. The store name shown in Fig. 7B is composed of the store (trade name) or company and the branch name.
The store identification information 1 is information on a two-dimensional code (QR code (registered trademark) or the like) for identifying a store in which a transaction takes place (a store where goods are bought and sold).
The store identification information 2 is store location information (GPS information) for identifying the store where the transaction is to be carried out.
In the example shown in FIG. 7B, the store identification information and the store specifying information 1 are different, but the store identification information and the store specifying information 1 may be the same.

The transaction management device 20 may obtain and store store information from an external source (such as a server (not shown) at a headquarters that manages each store). Also, instead of or in addition to storing the store information in its own memory, the transaction management device 20 may store some or all of the store information in another device (such as a file server accessible by the transaction management device 20).

The cart information is information for managing individual transactions. The transaction management device 20 generates the cart information at the start of a transaction. In addition, the transaction management device 20 updates the cart information as the transaction progresses (each time a product is registered) (it may be considered that a product has been registered when the product is stored in the cart information).

FIG. 7(C) is an example of cart information stored in the memory unit 202 of the transaction management device 20.
The cart information in FIG. 1 includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, registered product information, and reserved product information.

The cart identification information is identification information that uniquely identifies the cart information, and includes, for example, store identification information, a date, and a serial number (for example, a serial number for each store and date).
The transaction start date and time is the date and time when the transaction starts. The transaction start date and time may be, for example, the date and time when the cart information is generated. The transaction start date and time may be the date and time when the first item of product is registered (the date and time when the registered product information (registered product 1) in FIG. 7(C) is stored). The date and time when the cart information is generated and the date and time when the first item of product is registered may both be stored separately.

The transaction end date and time is the end date and time of the transaction. For example, the transaction start date and time may be the settlement date and time. The customer identification information is customer identification information that identifies the customer of the transaction. When the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are generated, but since this is before the start of the actual transaction (product registration), other information (transaction end date and time, etc.) is not generated. The settlement date and time may be the settlement start date and time or the settlement end date and time. Both the settlement start date and time and the settlement end date and time may be stored separately as the transaction end date and time.

Registered product information (total) is information that is updated each time a product is registered. Registered product information (total) includes the number of products (number of products), estimated subtotal amount (e.g., subtotal amount before discounts, etc. are reflected), subtotal amount, etc. Registered product information (registered product 1) is the registration information for one product item. Registered product information (registered product 2) is the registration information for two products. In the example shown in Figure 7 (C), registered product information (registered product 3) to registered product information (registered product 5) are omitted from the illustration. Registered product information (registered product N; N is an integer) includes the product code, product name (product name), price, etc.

The registered product information (N) may include the registration date and time of the Nth item of product. In other words, the transaction management device 20 may store the registration date and time of the registered product as the registered product information. The registration date and time of each product may also be used as a basis for determining whether or not to apply a service such as a time service and the effect of applying the service.

The reserved item information (total) is information that is updated each time a reserved item is registered. The reserved item information (total) includes the number of reserved items (items) of all types and the number of reserved items (items) of each type. In this embodiment, the reserved items are classified into multiple types, and each classification is managed by classification number (reserved major classification number, reserved minor classification number) (details are explained in Figure 8). In the example of Figure 7 (C), "Total number of items: 3" in the reserved item information (total) indicates that the number of items in all types (total of reserved items with reserved major classification numbers "1" to "4") is 3, "NON-FILE: 2" indicates that the number of reserved items classified as NON-FILE (total of reserved items with reserved major classification number "1") is 2, and "Read NG: 1" indicates that the number of reserved items classified as read NG (total of reserved items with reserved major classification number "2") is 1.

A NON-FILE means that the barcode or product code was successfully scanned at the store (the product code was read), but the product code has not been stored (registered) in the product file. In other words, reserved products classified as NON-FILE are reserved products whose product code was successfully scanned, but whose product code has not been stored in the product file.

A "no read" means that the product code was not successfully read (scanned) at the store. In other words, a reserved product classified as a "no read" is a reserved product whose product code was not successfully read.

Pending product information (pending product 1) is information on pending product 1. Pending product information (pending product 2) is information on pending product 2. Pending product information (pending product 3) is information on pending product 3.

The reserved product information (reserved product N; N is an integer) includes a reserved major classification number (described below). For example, if the Nth item is a reserved product classified as a NON-FILE with a reserved major classification number of "1", the reserved product information (reserved product N) includes the reserved major classification number "1 (NON-FILE)" and a product code. Also, if the Nth item is a reserved product classified as a read NG with a reserved major classification number of "2", the reserved product information (reserved product N) includes the reserved major classification number "2 (read NG)" and image data (image captured by the mobile terminal 60). Note that the reserved product information for reserved products classified as NON-FILE may include image data in addition to the reserved major classification number "1 (NON-FILE)" and the product code.

Note that the information shown in FIG. 7 is an example of information stored by the transaction management device 20, and the transaction management device 20 may store other information (e.g., a product file). Information about one product in a product file may include, for example, a product code (product identifier), product name, price, etc.

Figure 8 is an explanatory diagram explaining the classification of reserved items according to this embodiment. As shown in Figure 8, in this embodiment, reserved items are classified into major categories (classification by reserved major category number) and minor categories (classification by reserved minor category number). Note that the devices in the product sales data processing system 1 may be able to refer to information such as that shown in Figure 8 (such as a file showing the relationship between reserved major category numbers and reserved minor category numbers).

As shown in FIG. 8, in this embodiment, reserved items are classified into four major categories, with reserved major category numbers "1" to "4."

Reserved items classified under reserve category number "1" are NON-FILE reserved items. As described above, NON-FILE reserved items are reserved items whose product code has been successfully scanned, but whose product code is not stored in the product file. In other words, if a product code has been successfully scanned, but the product code has not been stored in the product file, the reserved item is classified under reserve category number "1." Reserved items classified under reserve category number "1" have an unknown price, so confirmation by a store clerk is required at the time of payment.

In FIG. 8, "NON-FILE" in "1 (NON-FILE)" is the classification name of the reserved item (major reserved classification name). The same applies to "Read NG" in "2 (Read NG)", "Specific item" in "3 (Specific item)", and "Cancelled item" in "4 (Cancelled item)" described below.

In this embodiment, reserved minor category number "11" exists as a subordinate (lower) minor category of reserved major category number "1". Since there is only one reserved minor category under reserved major category number "1", reserved minor category number "11", all reserved products classified under reserved major category number "1" are classified under reserved minor category number "11".

In FIG. 8, "NON-FILE" in "11 (NON-FILE)" is the category name of the reserved product (reserved subcategory name). The same applies to "Pseudo scan (coated, etc.)" in "21 (Pseudo scan (coated, etc.)" and "Pseudo scan (short time)" in "22 (Pseudo scan (short time))" which will be described later.

In Figure 8, a read operation "Yes" indicates that a read operation was performed, a read operation "No" indicates that a read operation was not performed, a read success/failure indicator "Success" indicates that the read operation was successful, a read success/failure indicator "Failure" indicates that the read operation was unsuccessful, and a read success/failure indicator "-" indicates that the read operation was not performed, so the problem is prior to the success or failure of the read.

Reserved items classified under reserve category number "2" are reserved items that could not be read. As described above, reserved items that could not be read are reserved items whose product codes could not be read successfully. In other words, when an item is placed on hold because the product code could not be read successfully, the item is classified under reserve category number "2." Reserved items classified under reserve category number "2" have an unknown price and are also susceptible to fraudulent manipulation, so they require confirmation by a store clerk at the time of payment.

In this embodiment, reserved minor category numbers "21", "22", "23", and "24" exist as minor categories under the reserved major category number "2". Reserve minor category number "21" is a reserved product with a "pseudo scan (covered, etc.)". Reserve minor category number "22" is a reserved product with a "pseudo scan (short time)". Reserve minor category number "23" is a reserved product with a "bad barcode". Reserve minor category number "23" is a reserved product with a "non-scanned input".

A reserved item with reserved subcategory number "21" (pseudo scan (covered, etc.)) is, for example, a reserved item for which a reading operation was performed (the item was held over), that is, the item was placed within the imaging range of the imaging unit 606 (more specifically, the readable range of the barcode), but the barcode of the item was not captured (captured within the readable range) (because only parts other than the barcode of the item are captured within the readable range), so the item could not be read; a reserved item for which the barcode of the item was captured but part of the barcode was covered, for example by a finger, so the item could not be read; or a reserved item for which the barcode of the item was captured but the imaging angle of the barcode was inappropriate, so the item could not be read.

A reserved product with reserved subcategory number "22" (pseudo scan (short time)) is, for example, a reserved product whose barcode was imaged but could not be read because the barcode moved (including passing by) too quickly to be read.

A reserved product with the reserved major category number "23" (defective barcode) is, for example, a reserved product whose barcode has been imaged but cannot be read due to wrinkles, fading, dirt, reflections, etc. in the printed portion of the barcode.

A reserved item with a reserved major classification number of "24" (non-scanned entry) is, for example, a reserved item that could not be read because it was placed in the shopping cart 80 without the product being scanned (without being held over the cart).

Pending products with reserved subcategory numbers "21" to "24" (i.e. pending products with reserved major category number "2") may be referred to as unreadable products. Also, a situation resulting in pending products with reserved subcategory numbers "21" to "24" may be referred to as a unreadable situation (or simply "unreadable").

The reserved items with reserved subcategory number "21" (pseudo scan (covered, etc.)), reserved items with reserved subcategory number "22" (pseudo scan (short time)), and reserved items with reserved major category number "23" (bad barcode) have in common that they are all reserved items for which a reading operation was performed but the reading was unsuccessful. However, the reserved items with reserved subcategory number "21" (pseudo scan (covered, etc.)) and reserved items with reserved subcategory number "22" (pseudo scan (short time)) are considered to have less of an intention to successfully read the item than the reserved items with reserved major category number "23" (bad barcode). In other words, the reserved items with reserved subcategory number "21" (pseudo scan (covered, etc.)) and reserved items with reserved subcategory number "22" (pseudo scan (short time)) can also be said to be pseudo reading operations (fake reading operations), and are therefore referred to as pseudo-scanned (fake scan) reserved items.

Pending items classified under the reserve category number "3" are specific items (described below). Specific items are pending items that have been successfully registered, but because they are specific items, they require the assistance of a store clerk when checking out. In other words, when a pending item becomes a pending item because it is a specific item, it is classified under the reserve category number "3". Pending items classified under the reserve category number "3" are specific items, and therefore require confirmation by a store clerk when checking out.

In this embodiment, reserved minor category numbers "31," "32," and "33" exist as minor categories under the reserved major category number "3." Reserve minor category number "31" is a reserved product of "age verification product." Reserve minor category number "32" is a reserved product of "pharmaceuticals." Reserve minor category number "33" is a reserved product of "anti-theft tags."

Reserved items with reservation subcategory number "31" (age verification items) are reserved items that require a store clerk to verify the customer's age before selling them, such as alcohol or tobacco.

"Reserved products with reservation subcategory number 32 (pharmaceuticals)" are reserved products that require, for example, a licensed person such as a pharmacist to explain the product to the customer.

A reserved item with reservation subcategory number "33" (item with security tag) is, for example, a reserved item that has a security tag attached and therefore requires a store clerk to remove the security tag from the item at the time of sale.

Reserved items classified under reserve category number "4" are canceled reserved items. Cancelled reserved items are reserved items that were successfully registered, but whose registration was cancelled after registration. In other words, if a product becomes reserved because its registration was cancelled after it was registered, the reserved item is classified under reserve category number "4". Reserved items classified under reserve category number "4" are cancelled items (i.e. items that will not be purchased) and will not be taken out of the store, so confirmation by a store clerk is required when making the payment.

In this embodiment, reserved minor category number "41" exists as a minor category under reserved major category number "4". Since reserved major category number "4" has only one reserved minor category under reserved minor category number "41", all reserved products classified under reserved major category number "4" are classified under reserved minor category number "41".

The above explanation is based on the assumption that a reading operation has been performed for reserved items with reserved subcategory number "31" (age verification items), but if a reading operation has not been performed, the item will be classified as reserved item with reserved subcategory number "24" (non-scanned input). The same applies to reserved items with reserved subcategory number "32" (pharmaceuticals), reserved items with reserved subcategory number "33" (anti-theft tag items), and reserved items with reserved subcategory number "41" (cancelled items).

In the above explanation, it is assumed that a reading operation is performed on reserved items with reserved subcategory number "31" (age verification items) and that the reading is successful. However, if a reading operation is performed and the reading fails, the reserved item will be classified as either a reserved item with reserved subcategory number "21" (pseudo scan (covered, etc.)), a reserved item with reserved subcategory number "22" (pseudo scan (short time)), or a reserved item with reserved subcategory number "23" (bad barcode). The same applies to reserved items with reserved subcategory number "32" (pharmaceuticals), reserved items with reserved subcategory number "33" (anti-theft tag items), and reserved items with reserved subcategory number "41" (cancelled items).

As shown in FIG. 8, the reserved major category number and the reserved minor category number are in a higher-ranking relationship, but one or both may be used depending on the situation. For example, in the cart information shown in FIG. 7(C), an example is described in which the reserved major category number is stored in the reserved product information, but the reserved minor category number may be stored instead of or in addition to the reserved major category number.

In addition, instead of or in addition to storing the cart information in its own memory, the transaction management device 20 may store some or all of the cart information in another device (such as a file server accessible to the transaction management device 20).

[Example of basic processing procedure for self-operation]
The sequence diagram in Fig. 9 shows an example of a processing procedure executed by the mobile terminal 60, the settlement device 40, and the transaction management device 20 in response to a transaction in a self-service operation. Specifically, the processing in Fig. 9 shows an example of the processing of the mobile terminal 60, the settlement device 40, and the transaction management device 20 from when a customer registers a product to be purchased from among products displayed in the store, until the settlement for the registered product is completed.

Step S1: The mobile terminal 60 acquires information that identifies the store (store identification information). For example, a two-dimensional code for identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output to a display screen, a medium with the two-dimensional code printed on it may be attached, etc.), and a customer visiting the store may scan (read) the two-dimensional code with the mobile terminal 60, thereby causing the mobile terminal 60 to acquire the store identification information. When a customer visiting the store starts a shopping application, a screen instructing the customer to scan the two-dimensional code may be displayed as a start-up screen, or when a customer visiting the store scans the two-dimensional code with the mobile terminal 60, the shopping application may start and a screen informing the customer that they are currently connected to the transaction management device 20 may be displayed as a start-up screen.

Figure 10 (A) shows an example of a start-up screen of a shopping application displayed on the display unit 604 of the mobile terminal 60. The start-up screen in this figure displays the store name. This store name is displayed based on the store-specific information acquired in step S1. When a customer wants to start registering products for shopping, he or she operates the transaction start button BT10 located on the start-up screen.

In response to the operation of the transaction start instruction button BT10, the mobile terminal 60 transmits a transaction start request including the acquired store specific information and customer identification information to the transaction management device 20. The destination information (the address of the transaction management device 20) may be acquired when the customer is registered (when a shopping application is downloaded or installed on the mobile terminal 60) and stored in the memory of the mobile terminal 60. Note that, when scanning a two-dimensional code, in addition to the store specific information, the destination information may also be converted into a two-dimensional code, and by scanning the two-dimensional code with the mobile terminal 60, the mobile terminal 60 may acquire the destination information along with the store specific information.

Step S2: The transaction management device 20, which has received the customer identification information and store identification information as a transaction start request from the mobile terminal 60, generates cart information for the transaction. For example, the transaction management device 20 acquires store identification information from the store identification information received from the mobile terminal 60, and further acquires the current date, issues (assigns) a serial number, combines the store identification information, the current date, and the serial number, and stores the combined information as cart identification information in the cart information. The transaction management device 20 also stores the current date and time as the transaction start date and time (generation date and time) in the cart information. The transaction management device 20 also stores the customer identification information received from the mobile terminal 60 as customer identification information in the cart information.

Step S3: The transaction management device 20, which has generated the cart information for the transaction, generates product registration initial screen information (screen information of the product registration screen, which is the initial screen), and transmits it to the mobile terminal 60. Specifically, the transaction management device 20, for example, generates product registration initial screen information, and transmits the generated product registration initial screen information together with the cart identification information to the mobile terminal 60.

Step S4: The mobile terminal 60 receives the cart identification information and the product registration initial screen information from the transaction management device 20, stores the cart identification information, and displays the registration screen on the display unit. Specifically, the mobile terminal 60 displays the product registration initial screen as shown in FIG. 10(B), for example.

Step S5: The customer operates the mobile terminal 60 to scan the barcode attached to the product and read the product code. In the figure, it is assumed that the barcode has been successfully scanned. Steps S5 to S9 are repeated each time a barcode attached to a product is scanned.

The mobile terminal 60 that has acquired the barcode transmits the cart identification information and the product registration information, including the product code obtained by scanning, to the transaction management device 20.

Step S6: The transaction management device 20 receives the product registration information (cart identification information and product code) from the mobile terminal 60 and identifies the cart information of the transaction from the cart identification information included in the received product registration information.

Step S7: The transaction management device 20 updates the product data in the identified cart information. Specifically, if the received product registration information contains a product code for item N, the transaction management device 20 stores the product code as the product code of the registered product information (registered product N) in the identified cart information, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). The transaction management device 20 also updates the registered product information (total) in the identified cart information.

Step S8: After updating the product data in the cart information, the transaction management device 20 generates product registration update screen information (screen information of the product registration screen, which is an update screen to which the registered product has been added) and transmits it to the mobile terminal 60. Specifically, the transaction management device 20 generates product registration update screen information such that the mobile terminal 60 displays a product registration update screen to which the list item of the currently registered product has been added to the registered product list, as shown in FIG. 10 (C), for example, and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information.

The product registration screen (product registration update screen) shown in FIG. 10(C) is displayed on the mobile terminal 60 after "XX bread" is registered as the third product. In other words, when "XX yogurt" is stored in the cart information as the first product, the transaction management device 20 generates product registration update screen information such that "XX yogurt" is displayed on the mobile terminal 60 and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information. When "XX chocolate" is stored in the cart as the second product, the transaction management device 20 generates product registration update screen information such that "XX yogurt" and "XX chocolate" are displayed on the mobile terminal 60 and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information. When "XX bread" is stored in the cart information as the third product, the transaction management device 20 generates product registration update screen information such that "XX yogurt", "XX chocolate", and "XX bread" are displayed on the mobile terminal 60 as shown in FIG. 10(C) and transmits the generated product registration update screen information to the mobile terminal 60 together with the cart identification information.

In response to the registration of one product, a registered product presentation screen WD1 may be displayed, presenting the currently registered product, as shown in FIG. 11(A), for example. In addition, in the registered product presentation screen WD1 in the same figure, if the product in question is available for eat-in, a "Yes" button BT1 and a "No" button BT2 are provided to confirm with the customer whether or not they would like to eat in the product. In this case, the price of the product will be calculated so that the consumption tax rate for the product in question differs depending on whether the "Yes" button BT1 is operated or the "No" button BT2 is operated.

Step S9: The mobile terminal 60, which has received the cart identification information and product registration update screen information from the transaction management device 20, adds the product to the registration screen. Specifically, the mobile terminal 60 displays a product registration update screen such as that shown in FIG. 10(C). As described above, the product registration screen (product registration update screen) shown in FIG. 10(C) is displayed on the mobile terminal 60 after "XX bread" has been registered as the third product item.

Step S10: The mobile terminal 60 accepts a payment instruction operation as a customer operation. For example, it accepts the touch of the "Proceed to checkout" button shown in FIG. 10(C).

Step S11: The mobile terminal 60 that has accepted the settlement transition instruction operation generates a two-dimensional code. That is, the mobile terminal 60 converts information required to execute the settlement process for the items purchased by the mobile terminal 60 (e.g., cart identification information) into a two-dimensional code. The mobile terminal 60 that has generated the two-dimensional code displays the generated two-dimensional code on the display unit. For example, a code display screen with a two-dimensional code arranged as shown in FIG. 10 (D) is displayed on the display unit.

Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the mobile terminal 60. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the mobile terminal 60 that is pointed by the customer within the recognition range of the scanner unit 406.

Step S13: The settlement device 40 reads the two-dimensional code displayed on the display of the mobile terminal 60 and requests the transaction management device 20 to calculate the subtotal amount. For example, the settlement device 40 sends a calculation request (subtotal calculation request information) requesting calculation of the subtotal amount to the transaction management device 20 together with the cart identification information obtained from the two-dimensional code.

Step S14: The transaction management device 20 receives the cart identification information and the sub-calculation output request information from the mobile terminal 60 and identifies the cart information of the transaction from the cart identification information.

Step S15: After identifying the cart, the transaction management device 20 calculates the subtotal amount using information such as the price in the registered product information included in the cart information corresponding to the identified cart information.

Step S16: After calculating the subtotal, the transaction management device 20 updates the cart information (stores the subtotal (calculated subtotal)) and transmits the subtotal information indicating the calculated subtotal together with the cart identification information to the settlement device 40.

Step S17: The settlement device 40, which has received the cart identification information and subtotal information from the transaction management device 20, displays the subtotal amount (a settlement screen including the subtotal amount) on the display unit 405.

Step S18: The settlement device 40, which has displayed the subtotal amount on the display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the change amount, and, if there is change, dispenses the change and issues a receipt. In addition, when the settlement is completed, the settlement device 40 sends settlement completion information together with the cart information to the transaction management device 20, and the transaction management device 20 stores the transaction end date and time (settlement date and time) for that cart.

[About reading failure situations]
9 shows a process corresponding to the case where the reading of the product code is successful in step S5. However, during the stage of the product code reading process corresponding to step S5, a situation may occur in which the portable terminal 60 fails to read any of the reserved classification numbers "21" to "24" in FIG. 8 (reading failure situation).

As a specific example, FIG. 11(B) shows a scene in which a reserved product with reserved subcategory number "21" (pseudo scan (covered, etc.)) may occur, using a captured image of the product registration screen. The figure shows the product registration screen displayed on the mobile terminal 60. The product registration screen in the figure shows a captured image of a customer attempting to have the mobile terminal 60 read the barcode of a product, with the barcode portion obscured by their finger.

In the example shown in FIG. 11B, the mobile terminal 60 recognizes the presence of a barcode from the captured image, but fails to read the barcode (it cannot extract the code). In this case, the mobile terminal 60 executes a read failure process (described below) to deal with the read failure situation.

Details will be described later, but in the reading failure process, a retry (reading) is performed on the product that has become a reading failure, and if the retry does not resolve the reading failure, a preset key is displayed. Products that have become a reading failure can be registered via a preset key. In other words, the reading failure can be resolved by retrying or using a preset key. In many cases, the reading failure can be resolved by retrying or using a preset key, but if the reading failure is not resolved, the product becomes a pending item.

If the reading failure situation is not resolved (i.e., if the product becomes a pending product), for example, as shown in FIG. 11(C), a guide image WD2 may be displayed to encourage the user to place the pending product in a bag for pending products. Specifically, the guide image WD2 in the figure displays a message saying, "If scanning is not possible, please place the product in a special bag." Note that the form of the guide image WD2 is not particularly limited. The message displayed in the guide image WD2 may be, for example, "Scanning has failed" or "The product has been determined to be a reading failure."
The guide image WD2 in the figure is displayed so as to be superimposed on the product registration screen. In the guide image WD2 in the figure, a confirmation button BT3 is arranged. In this case, the guide image WD2 is erased in response to an operation on the confirmation button BT3. Alternatively, the guide image WD2 may be erased in response to a certain time having elapsed since the start of display. After the guide image WD2 is erased, the display may return to the product registration screen of FIG. 10(B) or FIG. 10(C), for example.

When the mobile terminal 60 places a misreadable product on hold, the mobile terminal 60 may transmit captured images corresponding to the misreadable state to the transaction management device 20. The captured images transmitted may be extracted from the buffered captured images, for example, by buffering the captured images captured by the imaging unit 606 for a certain period of time in order to read the product code. The captured images transmitted may be, for example, a predetermined number of still images, or may be moving images for a predetermined period of time in response to the occurrence of the misreadable state.

In the product registration screen, a registered product list is displayed as shown in FIG. 10(C), but the mobile terminal 60 may display the products as list items regardless of the type of reserved product (i.e., all reserved products in the subcategory). When displaying the reserved products, information indicating that the products are reserved products (icon or button indicating that the products are reserved products, classification number (one or both of the major reserved classification number and the minor reserved classification number), classification name (one or both of the major reserved classification name and the minor reserved classification name)) may be displayed. When displaying the reserved products, information indicating the type of the reserved product (classification number (one or both of the major reserved classification number and the minor reserved classification number), classification name (one or both of the major reserved classification name and the minor reserved classification name), icon or button corresponding to the classification number or classification name) may be displayed. In addition, the mobile terminal 60 may not display some reserved products (reserved products with a specific minor reserved classification number) as list items, and may display other reserved products as list items.

Figures 12 to 14 are flowcharts showing an example of the operation of the mobile terminal 60 according to this embodiment. Figure 15 is a flowchart showing an example of the operation of the transaction management device 20 according to this embodiment. The flowcharts in Figures 12 to 15 correspond to the processing of steps S5 to S9 in Figure 9. Note that the flowcharts in Figures 12 to 15 omit processing related to reserved items (specific items) classified under reserved major classification number "3" and reserved items (cancelled items) classified under reserved major classification number "4".

Although not shown in the flowcharts of Figures 12 to 14, the mobile terminal 60 is constantly capturing images using the imaging unit 606 in order to read barcodes and recognize when products are placed in the shopping cart 80.

[Explanation of the flow chart in FIG. 12]
Step S100: The mobile terminal 60 judges whether or not the customer has started to read a barcode. For example, when the mobile terminal 60 recognizes an object resembling a barcode in the captured image (through image), it judges that the customer has started to read a barcode. Note that several images of objects resembling a barcode may be stored as comparative images in the storage unit 602 of the mobile terminal 60 (or in another device that the mobile terminal 60 can refer to, for example, the storage unit 202 of the transaction management device 20), and when the mobile terminal 60 recognizes an image similar to any of the comparative images in the captured image, it may judge that it has recognized an object resembling a barcode (i.e., it may judge that the customer has started to read a barcode). If the reading operation has started (step S100: YES), the process proceeds to step S101. If the reading operation has not started (step S100: NO), the process proceeds to step S104.

Step S101: The mobile terminal 60 determines whether or not the reading of the barcode has been completed (successful). In other words, the mobile terminal 60 determines whether or not the product code has been obtained from the barcode. If the reading has been completed (Step S101: YES), proceed to Step S110. If the reading has not been completed (Step S101: NO), proceed to Step S102.

Step S102: The mobile terminal 60 determines whether or not it has detected the addition of an item to the shopping basket 80. That is, the mobile terminal 60 set in the manner illustrated in FIG. 3 captures an image of the contents of the shopping basket 80 placed in the shopping cart 70, and is able to detect the addition of an item to the shopping basket 80 based on changes in the captured image. If it detects the addition of an item (step S102: YES), it proceeds to step S120. If it does not detect the addition of an item (step S102: NO), it proceeds to step S103.

Step S103: The mobile terminal 60 determines whether a predetermined time has elapsed since the start of the reading operation. If the predetermined time has elapsed (Step S103: YES), the process proceeds to Step S120. If the predetermined time has not elapsed (Step S103: NO), the process returns to Step S101.

Step S104: As in step S102, the mobile terminal 60 determines whether or not it has detected the addition of an item to the shopping cart 80. If it detects the addition of an item (step S104: YES), it proceeds to step S120. If it does not detect the addition of an item (step S104: NO), this flow chart ends.

Step S110: Following step S101 (YES), the mobile terminal 60 executes the product registration process (described below). This flow chart then ends.

Step S120: Following step S102 (YES), step S103 (YES), and step S104 (YES), the mobile terminal 60 executes a reading failure process (described below). This flow chart then ends.

[Explanation of the flow chart in FIG. 13]
The flowchart in FIG. 13 is an example of the product registration process (S110) in FIG.
Step S111: The mobile terminal 60 transmits the product registration information to the transaction management device 20. The product registration information includes the product code obtained from the barcode read in step S101 and the cart identification information. Then, the process proceeds to step S112.

Step S112: The mobile terminal 60 receives the registration completion notification sent from the transaction management device 20 in response to the sending of the product registration information in step S111. Then, proceed to step S113.

Step S113: The mobile terminal 60 updates the product registration screen, for example as shown in FIG. 10(C).

[Explanation of the flow chart in FIG. 14]
The flowchart in FIG. 14 is an example of the reading failure process (S120) in FIG.
Step S121: The portable terminal 60 judges whether the value of the reread operation instruction flag is "1". The reread operation instruction flag is information indicating whether a reread operation (a re-read operation) has been instructed. The value "0" of the reread operation instruction flag indicates that a reread operation has not yet been instructed in the transaction, and the value "1" of the reread operation instruction flag indicates that a reread operation has already been instructed in the transaction. If the value of the reread operation instruction flag is "1", that is, if a reread operation has already been instructed in the transaction (step S121 (YES)), proceed to step S124. If the value of the reread operation instruction flag is "0", that is, if a reread operation has not yet been instructed in the transaction (step S121 (NO)), proceed to step S122. The re-read operation instructed flag may be stored in the memory unit 602 of the portable terminal 60 (or in another device that can be accessed by the portable terminal 60, for example, the memory unit 202 of the transaction management device 20). When the re-read operation instructed flag is stored in another device, the re-read operation instructed flag for each portable terminal 60 (or customer) may be stored in the other device so that each can be identified.

Step S122: The mobile terminal 60 sets the value of the re-read operation instruction completion flag to "1". Then, the process proceeds to step S123.

Step S123: The mobile terminal 60 displays a message prompting the user to perform a reading operation. Then, the process returns to step S100 in FIG. 12.

Step S124: The mobile terminal 60 sets the value of the re-read operation instruction completion flag to "0". Then, proceed to step S125.

Step S125: The mobile terminal 60 displays the preset keys on the display unit 604. The customer can operate the preset keys to register the product. Details of the preset keys will be described later. Then, proceed to step S126.

Step S126: The mobile terminal 60 determines whether or not a preset key has been operated. In other words, the mobile terminal 60 determines whether or not an operation to register a product using a preset key has been performed. If a preset key has been operated (step S126: YES), the process returns to step S110 (product registration process) in FIG. 12 and FIG. 13. If a preset key has not been operated (step S126: NO), the process proceeds to step S127.

Step S127: The mobile terminal 60 determines whether a predetermined time has elapsed since the display of the preset keys in step S125 (start of display). If the predetermined time has elapsed (step S127: YES), the process proceeds to step S130. If the predetermined time has not elapsed (step S127: NO), the process returns to step S126.

Step S130: The mobile terminal 60 determines a classification number (e.g., a reserved sub-classification number) for the reserved product. For example, the mobile terminal 60 may determine the reserved sub-classification number as follows:

[Determination of reserved product classification number (reserved sub-classification number)]
(1) When the process transitions from step S102 (YES) in FIG. 12 to the reading failure process (step S120), the reserved subcategory number is determined to be "22." In other words, since the insertion of a product was detected before the predetermined time had elapsed, the reserved product is determined to be a reserved product with reserved subcategory number "22" (pseudo scan (short time)).
(2) When the process transitions from step S103 (YES) in Fig. 12 to the reading failure process (step S120) and it is determined that the operation is a pseudo scan, the reserved subcategory number is determined to be "21." In other words, the reading operation is permitted until a predetermined time has elapsed, and if it is determined that the operation is a pseudo scan, the reserved product is determined to be a reserved product with reserved subcategory number "21" (pseudo scan (coated, etc.)).
(3) When the process transitions from step S103 (YES) in Fig. 12 to the reading failure process (step S120) and it is not determined to be a pseudo scan, the reserved subcategory number is determined to be "23." In other words, the reading operation is permitted until a predetermined time has elapsed, and if it is determined that the operation is not a pseudo scan, the reserved product is determined to be a reserved product with reserved subcategory number "23" (bad barcode).
(4) When the process transitions from step S104 (YES) in Fig. 12 to the reading failure process (step S120), the reserved subcategory number is determined to be "24." In other words, since the insertion of a product was detected without recognizing the start of the reading operation, the reserved product is determined to be a reserved product (non-scan-inserted) with the reserved subcategory number "24."

Further explanation will be given regarding the reserved item with reserved subcategory number "21" (pseudo scan (covered, etc.)) and the reserved item with reserved subcategory number "23" (bad barcode). Depending on whether or not the customer's reading operation up until a specified time has elapsed is a pseudo scan (specifically, a pseudo scan of covering, etc.), the mobile terminal 60 decides whether or not the item is a pseudo scan with reserved subcategory number "21" (pseudo scan (covered, etc.)) or the reserved item with reserved subcategory number "23" (bad barcode). Whether or not the item is a pseudo scan may be determined, for example, as follows.

The portable terminal 60 may determine whether or not the scan is a pseudo scan based on a plurality of captured images taken until a predetermined time has elapsed (until step S103 (YES) is reached).
(Determination Method 1)
The mobile terminal 60 determines that a pseudo-scan has occurred if at least a portion of the barcode is not captured in multiple images taken over a specified period of time (for example, if a portion is covered by a finger or the like), or if the barcode is within the imaging range of the imaging unit 606 but at an inappropriate angle, and determines that a pseudo-scan has not occurred if the entire barcode is captured and the angle of the barcode is appropriate.
(Determination method 2)
Instead of determination method 1, the portable terminal 60 may determine that it is a pseudo scan if the object has not changed (e.g., the angle or position of the object has changed by more than a predetermined amount) in the multiple captured images taken over a predetermined time period, even if the entire barcode is captured and the angle of the barcode is appropriate (i.e., there is no evidence of any attempt to read it). In other words, the portable terminal 60 may determine that it is not a pseudo scan if the entire barcode is captured in the multiple captured images taken over a predetermined time period, the angle of the barcode is appropriate, and the object has changed in the multiple captured images (i.e., there is evidence of any attempt to read it), and may determine that it is a pseudo scan if this is not the case.

The portable terminal 60 may determine (distinguish) how the process transitioned to the reading failure processing in the processing of step S130 (i.e., whether the process transitioned to the reading failure processing from step S102 (YES) in FIG. 12, the process transitioned to the reading failure processing from step S103 (YES) in FIG. 12, or the process transitioned to the reading failure processing from step S104 (YES) in FIG. 12) as follows. As an example, the portable terminal 60 may set a different flag (with a value of 1) for each of the cases where the process transitioned to the reading failure processing in step S130 was made, the case where the process transitioned to the reading failure processing in step S102 (YES) in FIG. 12 was made, the case where the process transitioned to the reading failure processing in step S103 (YES) in FIG. 12 was made, and the case where the process transitioned to the reading failure processing in step S130 was made based on which flag was set.

In addition, the process for determining the reserved sub-category number when transitioning to reading failure processing from step S102 (YES) in Figure 12, the process for determining the reserved sub-category number when transitioning to reading failure processing from step S103 (YES) in Figure 12, and the process for determining the reserved sub-category number when transitioning to reading failure processing from step S104 (YES) in Figure 12 may each be a different process (i.e., by branching the processing flow), and the corresponding reserved sub-category number may be determined (assigned) in each process. In other words, the reserved sub-category number determination process (classification number determination process A) when transitioning to reading failure processing from step S102 (YES) in FIG. 12 may determine the reserved sub-category number as "22," the reserved sub-category number determination process (classification number determination process B) when transitioning to reading failure processing from step S103 (YES) in FIG. 12 may determine the reserved sub-category number as "23" or "21," and the reserved sub-category number determination process (classification number determination process C) when transitioning to reading failure processing from step S104 (YES) in FIG. 12 may determine the reserved sub-category number as "24."

It is also possible that the transition to the reading failure processing before (first time) step S123 in FIG. 14 is passed through is different from the transition to the reading failure processing after (second time) step S123 in FIG. 14 is passed through. If the first transition to the reading failure processing is different from the second transition to the reading failure processing, the second transition (i.e. the newer one) may be adopted to determine the reserved sub-classification number. As an example, in a case where the first transition is from step S104 (YES) in FIG. 12 to the reading failure processing, and the second transition is from step S102 (YES) in FIG. 12 to the reading failure processing (the first transition is non-scanning, the second transition is short), the second transition (the transition from step S102 (YES) in FIG. 12 to the reading failure processing) may be adopted and the reserved sub-classification number may be determined to be "22".

Step S131: Following step S130, the mobile terminal 60 acquires (extracts) an image corresponding to the currently occurring reading failure situation from among the images captured by the imaging unit 606. Then, the process proceeds to step S132.

The captured images acquired by the mobile terminal 60 in step S131 will be described. The captured images acquired by the mobile terminal 60 are intended to allow the store clerk to visually recognize the reading failure situation that has occurred. For this reason, the format of the captured images acquired by the mobile terminal 60 (video, still image), the number of captured images, the time and period for which the captured images are acquired, etc. may differ depending on the reading failure situation.

[Images acquired for reserved products with reserved sub-classification numbers "21" to "23"]
In the case of reserved products with reservation subcategory numbers "21" to "23" (i.e., when a barcode reading operation has been performed), the mobile terminal 60 acquires captured images that allow recognition of the barcode reading status. In other words, the mobile terminal 60 always captures images using the imaging unit 606 in order to recognize the barcode reading operation and the insertion of products into the shopping cart 80 as described above, and temporarily stores these images, and extracts captured images that allow recognition of the barcode reading status in step S131. In the case of reserved products with reservation subcategory numbers "21" to "23" (i.e., when a barcode reading operation has been performed), the mobile terminal 60 may acquire (extract) captured images that allow recognition of the insertion of products into the shopping cart 80 (images before and after insertion, etc.) in addition to captured images that allow recognition of the barcode reading status in the case of reserved products with reservation subcategory numbers "21" to "23" (i.e., when a barcode reading operation has been performed).

[Image captured for reserved item with reserved sub-classification number "24"]
In the case of a reserved item with reserved subcategory number "24" (i.e., when the barcode has not been read), the mobile terminal 60 obtains captured images (images before and after the item has been added, etc.) that enable recognition of the item being added to the shopping basket 80. In other words, the mobile terminal 60 is always capturing images using the imaging unit 606 in order to recognize the barcode reading operation and the addition of the item to the shopping basket 80 as described above, and temporarily stores these images, and in step S131 extracts captured images (images before and after the item has been added, etc.) that enable recognition of the item being added to the shopping basket 80.

Step S132: The mobile terminal 60 transmits the unsuccessful reading information to the transaction management device 20. The unsuccessful reading information includes the captured image acquired in step S131, the reserved subcategory number determined in step S130, and the cart identification information. Then, proceed to step S133.

Step S133: The mobile terminal 60 receives the registration completion notification sent from the transaction management device 20 in response to the transmission of the reading failure information in step S132. Then, proceed to step S134.

Step S134: The mobile terminal 60 may, for example, update the product registration screen and display the guide image WD2 superimposed thereon, as shown in FIG. 11(C). This flow chart then ends.

As shown in Figures 12 to 14, after displaying a message prompting the user to perform another read operation, if the mobile terminal 60 recognizes the barcode read operation but does not complete the barcode reading normally (successfully) (step S102 (YES), step S103 (YES)), it displays a preset key (step S125) in the read failure process (step S120), and if the product is not registered using the preset key within a predetermined time (step S127 (YES)), it processes the product as a reserved product with a reserved subcategory number of "21" to "23".

As shown in Figures 12 to 14, if the mobile terminal 60 detects the insertion of a product without recognizing the barcode reading operation after displaying a message prompting another reading operation (step S104 (YES)), it displays a preset key (step S125) in the reading failure process (step S120), and if the product is not registered using the preset key within a predetermined time (step S127 (YES)), it processes the product as a reserved product with the reserved subcategory number "24".

In the flowchart shown in FIG. 14, if the first reading is unsuccessful, a second (again) reading is performed, and if the second reading is also unsuccessful, a preset key is displayed; however, if the first reading is unsuccessful, a preset key may be displayed without a second reading. For example, whether or not a second reading is performed may vary depending on the reserved subcategory number. As an example, in the case of a non-scan input with a reserved subcategory number of "24", a preset key may be displayed immediately without a second (again) reading, based on or independent of operation.

[Explanation of the flow chart in FIG. 15]
Step S201: The transaction management device 20 judges whether or not it has received the product registration information transmitted by the mobile terminal 60 in step S111 of Fig. 13. If it has received the product registration information (step S201: YES), it proceeds to step S202. If it has not received the product registration information (step S201: NO), it proceeds to step S211.

Step S202: The transaction management device 20 determines whether or not the product has been identified. In other words, the transaction management device 20 determines whether or not the product code included in the product registration information is stored in the product file (product master). If the product has been identified (step S202: YES), proceed to step S203. If the product has not been identified (step S202: NO), proceed to step S213.

Step S203: The transaction management device 20 identifies the cart information. In other words, the transaction management device 20 identifies the cart information that corresponds to the cart identification information included in the product registration information. Then, the process proceeds to step S204.

Step S204: The transaction management device 20 updates the cart information identified in step S203. That is, the transaction management device 20 updates the registered product information (total, N registered products) in the cart information. As a result, one product is registered in the cart information. Then, proceed to step S205.

Step S205: The transaction management device 20 transmits a registration completion notification to the mobile terminal 60. The mobile terminal 60 receives the registration completion notification in step S112 of FIG. 13. This flowchart then ends.

Step S211: The transaction management device 20 determines whether or not it has received the read failure information sent by the mobile terminal 60 in step S132 of FIG. 14. If it has received the read failure information (step S211: YES), it proceeds to step S213. If it has not received the read failure information (step S211: NO), this flow chart ends.

Step S213: The transaction management device 20 identifies the cart information. In other words, the transaction management device 20 identifies the cart information that corresponds to the cart identification information included in the unread information. Then, the process proceeds to step S214.

Step S214: The transaction management device 20 updates the cart information identified in step S213. That is, the transaction management device 20 updates the reserved item information (total, N reserved items) in the cart information. As a result, one item (reserved item) is registered in the cart information. Then, the process proceeds to step S215. The transaction management device 20 can refer to information showing the relationship between the reserved major classification number and the reserved minor classification number (for example, information such as that shown in FIG. 8), and recognizes the reserved major classification number that is higher than the reserved minor classification number included in the unreadable information.

Step S215: The transaction management device 20 sends a registration completion notification to the mobile terminal 60. The mobile terminal 60 receives the registration completion notification in step S133 of FIG. 14. This flowchart then ends.

If the transaction management device 20 is unable to identify the product (step S202: NO), it will register a reserved product (NON-FILE) with reserved major classification number "1" when updating the cart information (step S214).

When updating the cart information, the transaction management device 20 also registers reserved items (specific items) classified under the reserved major category number "3", i.e., reserved items with reserved minor category number "31" (age verification items), reserved items with reserved minor category number "32" (medicines), and reserved items with reserved minor category number "33" (anti-theft tag items). For example, the mobile terminal 60 may transmit product registration information for age verification items (medicines and anti-theft tag items as well) in the same way as for other items that do not fall under the category of specific items (step S111 in FIG. 13), and the transaction management device 20 may register the reserved items (specific items) in the processing of step S204 (or in the processing of step S214).

When updating the cart information, the transaction management device 20 also similarly registers reserved items (cancelled items) classified as reserved major classification number "4." For example, the mobile terminal 60 may transmit product registration information (including information indicating cancellation) in a cancellation process (cancellation process; not shown), and the transaction management device 20 may register the reserved item (cancelled item) in the process of step S204 (or in the process of step S214).

Figures 16 and 17 are diagrams showing examples of the re-read instruction screen and the preset key screen (small preset key screen) according to this embodiment. The display of the preset keys will be explained using Figures 16 and 17.

Figure 16 shows a situation in which a customer K has purchased one "XX potato" and two "XX tea" and then purchases one "XX chocolate."

16(A) shows a scene where one "XX potato" and two "XX tea" are registered (all successfully read). In FIG. 16(A), the mobile terminal 60 captures the inside of the shopping basket 80. In the captured image of FIG. 16(A), there are an "XX potato" object (symbol a) and an "XX tea" object (symbol b). From the scene of FIG. 16(A), if customer K reads the barcode of "XX chocolate" and puts the "XX chocolate" into the shopping basket 80 without successfully reading it (if step S102 (YES) in FIG. 12 or if step S103 (YES) is obtained), the captured image immediately after the insertion will be as shown in FIG. 16(B). In the captured image of FIG. 16(B), there are an "XX potato" object (symbol a), an "XX tea" object (symbol b), and an "XX chocolate" object (symbol c).

Then, the mobile terminal 60 displays a re-read instruction screen WD31 as shown in FIG. 16(C) (step S123 in FIG. 14). The re-read instruction screen WD31 in FIG. 16(C) displays a message (symbol e) stating that registration has failed (symbol d) and prompting the user to perform a read operation. The re-read instruction screen WD31 in FIG. 16(C) also displays an OK button BT51 that is operated to perform a read operation again in accordance with the message, and a Do not purchase button BT53 that is operated to cancel the purchase of the product (XX Chocolate).

When the OK button BT51 is operated on the reread instruction screen WD31, the reread instruction screen WD31 is cleared, and the process returns to step S100 in FIG. 12, as per the flowchart shown in FIG. 14.

If the "Do not purchase" button BT53 is operated on the re-read instruction screen WD31, the re-read instruction screen WD31 is cleared, and although this is omitted from the flowchart shown in FIG. 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

If the second reading operation is also unsuccessful, as in the first reading operation, the mobile terminal 60 displays the preset key screen WD41 as shown in FIG. 16 (D) (step S125 in FIG. 14).

The preset key screen WD41 in FIG. 16(D) displays a message (symbol d) indicating that registration (reading) has failed and a message (symbol f) prompting the user to select a product using a button (preset key). In addition, the preset key screen WD41 in FIG. 16(D) has multiple preset keys BT60 and a preset key display switch button BT61 (a button to display the next multiple preset keys BT60 and a button to display the previous multiple preset keys BT60. Either one is used in the case of the first or last). In addition, the preset key screen WD41 in FIG. 16(D) has a button BT62 that cannot be selected when the user intends to purchase the product (XX chocolate) but is unable to select the product (XX chocolate) using a preset key for reasons such as not being able to find a preset key corresponding to the product (XX chocolate), and a button BT63 that cannot be purchased when the user wishes to cancel the purchase of the product (XX chocolate).

When any of the preset keys BT60 are operated on the preset key screen WD41, the preset key screen WD41 is cleared, and the process returns to step S110 (product registration process) in FIG. 12 and FIG. 13, as shown in the flowchart in FIG. 14. This causes the product corresponding to the operated preset key BT60 to be registered. Specifically, in this example, the preset key BT60 corresponding to XX chocolate is operated on the preset key screen WD41. The mobile terminal 60 acquires the product code of XX chocolate based on the operation of the preset key BT60 corresponding to XX chocolate. When the mobile terminal 60 acquires the product code of XX chocolate based on the operation of the preset key BT60 corresponding to XX chocolate, the mobile terminal 60 executes the product registration process (step S110) and registers XX chocolate, in the same manner as when the reading of the barcode attached to XX chocolate is successful (i.e., when the product code of XX chocolate is acquired by reading the barcode).

When the display switch button BT61 is operated on the preset key screen WD41, the preset keys BT60 displayed on the preset key screen WD41 are switched. In other words, the mobile terminal 60 switches the preset keys BT60 displayed on the preset key screen WD41 based on the operation of the display switch button BT61.

If the button BT62 that cannot be selected on the preset key screen WD41 is operated, the preset key screen WD41 is cleared, and although omitted from the flowchart shown in FIG. 14, the process proceeds to step S130. In other words, the product (XX Chocolate) is processed as a reserved product, just as in the case where a predetermined time has elapsed (step S127 (YES)).

When the "Do not purchase" button BT63 on the preset key screen WD41 is operated, the preset key screen WD41 is cleared, and although this is omitted from the flowchart shown in FIG. 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

Figure 17 shows a scene where a customer K has purchased one "XX potato" and two "XX tea" and then purchases one "apple." Note that the "apple" does not have a barcode.

Like FIG. 16(A), FIG. 17(A) shows a scene in which one "XX potato" and two bottles of "XX tea" have been registered (all successfully read). If customer K adds an "apple" to the shopping basket 80 from the scene in FIG. 17(A) (step S104 (YES) in FIG. 12), the image captured immediately after the addition will be as shown in FIG. 17(B). In the image captured in FIG. 17(B), there are a "XX potato" object (symbol a), a "XX tea" object (symbol b), and an "apple" object (symbol g).

Then, the mobile terminal 60 displays a re-read instruction screen WD32 as shown in FIG. 17(C) (step S123 in FIG. 14). The re-read instruction screen WD32 in FIG. 17(C) displays a message indicating that registration has failed (symbol d) and a message prompting a reading operation (symbol e). In the case of loose products such as fruits and vegetables, a label with a barcode may be placed on the display stand, and the number of items may be input by scanning the label and entering the number of items in the app. If the label is placed, a message prompting a reading operation of the barcode on the label may be displayed. In addition, the re-read instruction screen WD32 in FIG. 17(C) includes an OK button BT51 to be operated when performing a reading operation according to the message, a no barcode button BT52 to be operated when the product (apple) does not have a barcode, and a no purchase button BT53 to be operated when the purchase of the product (apple) is canceled.

When comparing the reread instruction screen WD31 in FIG. 16(C) with the reread instruction screen WD32 in FIG. 17(C), FIG. 16 shows a scene where the barcode reading has failed (although the presence of the barcode was confirmed), so the no barcode button BT52 that is present on the reread instruction screen WD32 in FIG. 17(C) is not located on the reread instruction screen WD31 in FIG. 16(C).

When the OK button BT51 is operated on the re-read instruction screen WD32, the re-read instruction screen WD32 is cleared, and the process returns to step S100 in FIG. 12, as per the flowchart shown in FIG. 14. Note that in the example of FIG. 17, since the apple does not have a barcode attached, customer K does not operate the OK button BT51. Note that if the apple had a barcode attached, customer K might operate the OK button BT51.

When the no barcode button BT52 is operated on the reread instruction screen WD32, the reread instruction screen WD32 is cleared, and although this is omitted from the flowchart shown in FIG. 14, the process proceeds to step S124 in FIG. 14. In other words, if the product does not have a barcode, even if the process returns to step S100 in FIG. 12 and a reading operation is performed, the reading will not be successful, so the process simply proceeds to step S124 in FIG. 14.

If the "Do not purchase" button BT53 is operated on the re-read instruction screen WD32, the re-read instruction screen WD32 is cleared, and although this is omitted from the flowchart shown in FIG. 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

Based on the operation of the no barcode button BT52, the mobile terminal 60 displays the preset key screen WD42 as shown in FIG. 17(D) (step S125 in FIG. 14). Note that if the apple has a barcode attached, the OK button BT51 is operated, and the re-reading operation produces the same result as the first time, the mobile terminal 60 also displays the preset key screen WD42 as shown in FIG. 17(D) (step S125 in FIG. 14).

The preset key screen WD42 in FIG. 17(D) displays a message (symbol d) indicating that registration (reading) has failed, and a message (symbol f) prompting the user to select a product using a button (preset key). In addition, the preset key screen WD42 in FIG. 17(D) has a plurality of preset keys BT60 (preset key BT60a corresponding to apples, preset key BT60b corresponding to pears), and a preset key display switching button BT61. In addition, the preset key screen WD42 in FIG. 17(D) has a button BT62 for not selecting a product (apples) using a preset key, which is operated when the user intends to purchase the product (apples) but is unable to select the product (apples) using a preset key for reasons such as not being able to find a preset key corresponding to the product (apples), and a button BT63 for not purchasing the product (apples) that is operated to cancel the purchase of the product (apples).

For example, when the preset key BT60a corresponding to apples is operated on the preset key screen WD42, the preset key screen WD42 is cleared, and the process returns to step S110 (product registration process) in Figs. 12 and 13 as shown in the flowchart in Fig. 14. This causes apples corresponding to the operated preset key BT60a to be registered.

When the display switch button BT61 is operated on the preset key screen WD42, the preset keys BT60 displayed on the preset key screen WD42 are switched. In other words, the mobile terminal 60 switches the preset keys BT60 displayed on the preset key screen WD42 based on the operation of the display switch button BT61.

If the button BT62 that cannot be selected on the preset key screen WD42 is operated, the preset key screen WD42 is cleared, and although omitted from the flowchart shown in FIG. 14, the process proceeds to step S130. In other words, the product (apple) is processed as a reserved product, just as in the case where a predetermined time has elapsed (step S127 (YES)).

When the "Do not purchase" button BT63 on the preset key screen WD42 is operated, the preset key screen WD42 is cleared, and although this is omitted from the flowchart shown in FIG. 14, a cancellation process (cancellation process, not shown) is executed for the product, and the process ends.

[Preset keys to display]
The preset keys displayed on the preset key screen WD41 (WD42) will be described below. The portable terminal 60 may display the preset keys in any one of the following display modes 1 to 4, for example.

(Display mode 1)
For example, the mobile terminal 60 displays preset keys for all the products. Specifically, as shown in the preset key screen WD41 of Fig. 16(D) , the mobile terminal 60 displays preset keys for some of the products, and displays other preset keys in response to the operation of the display switching button BT61.
In the display mode 1, the number of preset keys to be displayed is very large. Therefore, there is a possibility that the customer cannot select a preset corresponding to the product to be purchased. In such a case, the customer may operate, for example, the button BT62 that cannot be selected in order to proceed with the process.

(Display Mode 2)
The mobile terminal 60 displays a preset key for a specific product. Specifically, as shown in the preset key screen WD42 of FIG. 17(D), the mobile terminal 60 displays a preset key for a specific product and does not display a preset key for other products. The specific product may be a product sold individually, such as fruits and vegetables (whether or not a barcode is attached). In other words, the specific product may be a product sold in a predetermined manner (e.g., sold individually). The specific product may also be a product not having a barcode (whether or not a product sold individually).
In display mode 2, the preset key displayed is the preset key corresponding to the specific product whether the specific product is registered or not. That is, when the specific product is registered, the preset key corresponding to the product to be purchased (specific product) is displayed, but when the product other than the specific product is registered, the preset key corresponding to the product to be purchased (product other than the specific product) is not displayed. Therefore, in the latter case, the customer operates, for example, the unselectable button BT62. Note that a button for displaying preset keys for products other than the specific product may be provided, and in the latter case, the preset key for the product other than the specific product may be displayed by operating the button for displaying preset keys for products other than the specific product.

(Display Mode 3)
The mobile terminal 60 displays a preset key according to the location in the store (the location of the mobile terminal 60, or the location of the customer using the mobile terminal 60). Specifically, the mobile terminal 60 captures (specifies, estimates) the location of each customer (mobile terminal 60) in the store using a GPS, a wireless LAN, a wireless tag, a beacon, or the like, and displays a preset key according to the captured location. For example, if the captured location is the alcohol section, a preset key related to alcohol may be displayed, and if the captured location is the daily necessities section, a preset key related to daily necessities may be displayed. Note that the preset key according to the captured location may be preferentially displayed (the preset key according to the captured location may be displayed on the first page, and other preset keys may be displayed depending on the operation).

(Display mode 4)
The mobile terminal 60 may determine the preset keys to be displayed based on image recognition. In other words, the mobile terminal 60 may display preset keys narrowed down from a large number of preset keys based on image recognition. For example, in the case of a round and red product, an apple or the like may be displayed as shown in the preset key screen WD42 of FIG. 17(D).

Note that display modes 1 to 4 may be combined as appropriate. Also, which of display modes 1 to 3 is adopted may be different depending on the customer.

[After payment transfer instruction]
When the customer has finished registering all the products to be purchased, he or she performs a settlement instruction operation and has the settlement device 40 read the two-dimensional code displayed on the mobile terminal 60. The settlement device 40 that has read the two-dimensional code requests the transaction management device 20 to calculate the subtotal amount, as shown as step S13 in Fig. 9. Here, in this embodiment, when requesting the calculation of the subtotal amount, the settlement device 40 requests settlement information to be used in the settlement process using the settlement screen.

In response to a request for settlement information, the transaction management device 20 identifies the relevant cart information and generates settlement information based on the identified cart information. The settlement information includes various information in the cart information (e.g., some or all of the registered product information, pending product information, cancellation information, etc.). For example, the settlement information includes image data (captured image) related to the pending product information. The transaction management device 20 transmits the generated settlement information to the settlement device 40. Upon receiving the settlement information, the settlement device 40 displays a settlement screen reflecting the contents of the received settlement information on the display unit 405.

FIG. 18 shows an example of a settlement screen displayed on the display unit 405 of the settlement device 40.
The checkout screen in the figure includes a registered product list area AR3, which displays a list of products registered by the mobile terminal 60.

If the registered products include pending products, the list items for the pending products are given priority and are arranged from the top of the left column. In the example shown in the figure, since the registered products include six pending products, a pending product list area AR31 is formed in the registered product list area AR3, containing list items for the six pending products from the top left. The list items in the pending product list area AR31 are designed to differ from the list items for other products, for example in terms of background color, etc.

In the list items in the registered product list area AR3, the product name of the corresponding product, the number registered, the price according to the number registered, etc. are shown. In addition, in the list items corresponding to reserved products, for example, the classification number (reserved major classification number, reserved minor classification number, or both) and classification name (reserved major classification name, reserved minor classification name, or both) of the reserved product may be shown.

A call button BT31 is provided for each list item of a reserved item in the reserved item list area AR31. Each call button BT31 shows the name of the reserved subcategory to which the corresponding item belongs.

The settlement screen also includes a customer monitor area AR32. The customer monitor area AR32 is an area in which the captured image obtained by the imaging unit 416 is displayed as a monitor image in real time. The customer monitor area AR32 displays in real time the state of the customer's hands as he or she operates the settlement device 40 to settle the payment. By the settlement device 40 displaying such an image in the customer monitor area AR32, the customer becomes aware that his or her state is being imaged, which has the effect of deterring fraudulent acts by customers when making payments. The settlement device 40 may be configured to store (save) the images used for display in the customer monitor area AR32.

Here, as shown in the figure, when the pending item list area AR31 is formed in the registered item list area AR3 on the checkout screen, the clerk needs to perform an operation to process the pending items. When the checkout screen with the pending item list area AR31 formed in this way is displayed, the checkout device 40 executes a call to the clerk to have the clerk perform an operation to process the pending items. As a call to the clerk, for example, the checkout device 40 may display a display such as a sign pole in a predetermined manner, or send a notification of the call to a terminal device carried or monitored by the clerk. The clerk who goes to the checkout device 40 in response to the call to the clerk will perform an operation to process the pending items corresponding to each list item shown in the pending item list area AR31 on the displayed checkout screen.

When processing pending item 1, the store clerk operates the call button BT31 in the list item of the pending item to be processed. In this case, the store clerk operates the call button BT31 of the list item in the first row corresponding to canned beer in the pending item list area AR31. In response to the operation of the call button BT31, a pending item processing screen for processing the pending item corresponding to the list item including the operated call button BT31 is displayed on the checkout screen. Note that the entire list item of the pending item to be processed may be touch operable.

For example, if a store clerk operates the call button BT31 of the first list item corresponding to a can of beer, which is an age verification product, a pending product processing screen for the can of beer will be displayed on the checkout screen.

Figure 19 shows an example of a reserved product processing screen AR4 for a can of beer, which is an age verification product. On the reserved product processing screen AR4, a product information area AR41, an input operation area AR42, a registration button BT41, and a cancel button BT42 are arranged.

The product information area AR41 is an area that shows information about the corresponding reserved product. The product information area AR41 in the figure shows the product name, product code, price, registered quantity, etc. In the case of an age verification product, the content displayed in the product information area AR41 reflects the information about the corresponding product (reserved product information) included in the payment information.

The input operation area AR42 is an area where a software keyboard for inputting product information in the product information area AR41 is arranged. The input operation area AR42 can be switched between a keyboard with a key arrangement corresponding to numeric input and a keyboard with a key arrangement corresponding to character input, depending on the operation. In addition, a preset call key labeled, for example, "Preset" is arranged in the input operation area AR42. By operating the preset call key, a preset key screen is displayed on which preset keys corresponding to each product are arranged. By operating the preset keys arranged in the preset key screen (preset key area), the store clerk can input information about the product corresponding to the operated preset key into the product information area AR41.

The registration button BT41 is a button that is used to release the pending registration of the corresponding product and register it as a purchase target for the customer. The cancel button BT42 is a button that is used to instruct the cancellation of the registration of a product whose registration is in a pending state.

For example, the store clerk will check the customer's age, and if there are no problems, will operate the registration button BT41 to finalize the registration of the canned beer. This will cause the settlement device 40 to cancel the pending registration status of the canned beer and process it to be included in the registered products. Alternatively, if the store clerk determines that the canned beer should not be registered, he or she will operate the cancel button BT42 to cancel the registration of the canned beer, and collect the canned beer product.

In addition, for an unreadable product, the call button BT31 in the list item of the pending product list area AR31 displays, for example, "Non-scan." When the store clerk processes the unreadable product, he or she operates the call button BT31 that displays "Non-scan" in the list item of the unreadable product.

For example, in the case of a reserved item classified as "NON-FILE," no product information is shown in the product information area AR41 in the initial display state of the reserved item processing screen AR4. In this case, the store clerk can register the item by inputting a numeric string as the product code or a character string as the product name in the product information area AR41 by performing a numeric input operation or a character input operation in the input operation area AR42. The product information registered in this manner is sent to the transaction management device 20. The transaction management device 20 synchronizes the cart information by updating the information of the corresponding product in the cart information (reserved item information) from the "NON-FILE" state so that the information entered in the product information area AR41 is reflected.

The reserved item processing screen may also display the classification number (reserved major classification number, reserved minor classification number, or both) and classification name (reserved major classification name, reserved minor classification name, or both).

Figure 20 shows an example of a pending product processing screen AR4 for processing an unreadable product, which is displayed in response to operating the call button BT31 marked "Non-scan." In the initial display state of the pending product processing screen AR4 corresponding to an unreadable product, no information about the product has yet been acquired, and therefore no information about the product is presented in the product information area AR41, as shown in the figure.

The pending product processing screen AR4 corresponding to the unread product also includes a product information area AR41, an input operation area AR42, a registration button BT41, a cancel button BT42, and a captured image area AR43.

In the captured image area AR43, a captured image corresponding to the occurrence of the corresponding reading failure situation is displayed. This captured image is acquired by the mobile terminal 60 in step S131 of FIG. 14.

Specifically, the image in the captured image area AR43 in the figure shows a reading failure situation classified as a non-scan input. Two captured images G11 and G12 are arranged in the captured image area AR43 in the figure. Captured image G11 is an image showing the state before an additional product is added to the shopping cart 80. Captured image G12 is an image showing the state after captured image G11 when an additional product has been added to the shopping cart 80.

By looking at the captured image in captured image area AR43 in the figure, the store clerk can see that the unreadable product to be processed was added to the shopping cart 80 without having its barcode read. Furthermore, in captured image G12, a frame FL is placed in the area of the added product. This allows the store clerk to easily identify what the added product is without being registered. The identification of the added product and the placement of the frame FL may be performed by either the mobile terminal 60 or the settlement device 40.

If the store clerk wants to check the captured image in the captured image area AR43 in more detail, the store clerk can perform a captured image enlargement operation, for example by tapping any position in the captured image area AR43. In response to the captured image enlargement operation being performed, the settlement device 40 places an enlarged captured image area AR431 on the pending item processing screen AR4, as shown in FIG. 21. The enlarged captured image area AR431 can be erased (closed) in response to a specified operation.

Returning to FIG. 20 for explanation, the store clerk confirms with the customer whether or not they wish to purchase the misread product. If the customer does not wish to purchase the product, the store clerk operates the cancel button BT42 and collects the misread product. If the customer wishes to purchase the product, the store clerk operates the input operation area AR42 to input product information into the product information area AR41, and then operates the register button BT41 to register the product.

The pending product processing screen AR4 in FIG. 22 shows an example of a case where the unsuccessfully read product is classified as a "pseudo scan." In the captured image area AR43 in the same figure, one captured image is displayed. The captured image in the captured image area AR43 displayed in this way shows that when the unsuccessful reading situation occurred, the image was captured with part of the barcode obscured by a finger. By looking at the captured image in this type of captured image area AR43, the store clerk can understand that the unsuccessfully read product to be processed is classified as a pseudo scan.

In this case, too, by performing an operation on the captured image area AR43, it is possible to display the enlarged captured image area AR431 on the reserved item processing screen AR4, as shown in FIG. 23.

In this case, the store clerk will also ask the customer whether they would like to purchase the misread product, and depending on whether they would like to purchase it, will register or cancel the product.

The pseudo scan also includes a situation where a product is held over the imaging range of the imaging unit 606, but the barcode is not included in the imaging range. In this case, the captured image area AR43 may display an image in which part of the product is included in the imaging range, but the barcode is not included in the imaging range.

The pseudo scan also includes a situation in which a product passes through the imaging range of the imaging unit 606 at a speed that makes it impossible to read the product code. In this case, the captured image area AR43 may display a still image captured when the product passes through the imaging range, or a moving image captured when the product passes through the imaging range.

An example of a processing procedure executed by the settlement device 40 in relation to displaying the reserved item processing screen will be described with reference to the flowchart in FIG. 24. Note that, at the start of the flowchart in FIG. 24, it is assumed that the settlement device 40 is displaying the settlement screen (FIG. 18).

Step S301: While the settlement screen is displayed, the settlement device 40 determines whether an operation to specify a reserved item to be processed (reserved item specification operation) has been performed. If a reserved item specification operation has been performed (step S301: YES), the process proceeds to step S302. If a reserved item specification operation has not been performed (step S301: NO), the process returns to step S301.

Step S302: The settlement device 40 determines whether the reserved product designated by the reserved product designation operation is an imperfectly read product (a reserved product with a reserved subcategory number of "21" to "24"). If it is an imperfectly read product (Step S302: YES), the process proceeds to Step S303. If it is not an imperfectly read product (Step S302: NO), the process proceeds to Step S304.

Step S303: The settlement device 40 generates a reserved item processing screen including an image capture area AR43 for the reserved item designated for processing (reserved items with reserved subcategory numbers "21" to "24" that correspond to the misreadable item). The settlement device 40 may obtain the captured image of the captured image area AR43 from the reserved item information of the corresponding item in the settlement information. The settlement device 40 displays the generated reserved item processing screen. This flowchart then ends.

Step S304: The settlement device 40 generates a reserved item processing screen that does not include the captured image area AR43 for the reserved items designated for processing (reserved items with reserved subcategory numbers "11", "31" to "33", and "41" that do not correspond to misread items). The settlement device 40 displays the generated reserved item processing screen. This flowchart then ends.

<Modification>
A modification of this embodiment will now be described.

[First Modification]
In the above embodiment, one or two captured images (FIGS. 20 and 22) are presented in the captured image area AR43 when the reserved product is an insufficiently read product. However, the number of captured images presented in the captured image area AR43 may be three or more.

Figure 25 shows an example in which three captured images G11, G12, and G13 are arranged in the captured image area AR43 of the pending product processing screen AR4. The pending product processing screen AR4 in the figure shows an example in which the unreadable product to be processed is classified as a non-scanned input.

The captured images G11, G12, and G13 in the figure show the following. In the captured image area AR43, the captured images G11, G12, and G13 are arranged from left to right in this order. The arrangement order of the captured images G11, G12, and G13 corresponds to the time lapse in which the images were captured. The first captured image G11 is an image of the inside of the shopping cart 80 before the target unreadable product is added. The second captured image G12 is an image of the process in which the captured image G11 changes to the next third captured image G13. The third captured image G13 is an image of the state in which the target unreadable product is added to the shopping cart 80. The second captured image G12 may be extracted from the captured images that include the product to be added, for example, or may be an image captured at a time that is determined to be exactly halfway between the times when the captured images G11 and G13 are captured. Also, in the captured image G13, a frame FL is placed on the part of the product that has been added to the shopping cart 80.

[Second Modification]
In the above embodiment, the display surface of the display unit 405 of the settlement device 40 is horizontally long. However, the display surface of the display unit 405 of the settlement device 40 may be vertically long.

Figure 26 (A) shows an example of a checkout screen displayed on display unit 405 with a vertically long display surface corresponding to this modified example. In this figure, parts that are the same as those in Figure 18 are given the same reference numerals. As can be seen by comparing this figure with Figure 18, the information presented on the checkout screen is the same, but the layout of the information presented on the checkout screen is different in accordance with the fact that the size of the display surface has been made vertically long. However, in the checkout screen of Figure 26 (A), a pending product list area AR31 including list items of pending products is formed in the upper left corner of the registered product list area AR3, and the background color, etc. is different from the list items other than the pending products, which is the same as the checkout screen of Figure 18.

In the case of this modified example, for example, the reserved item list area AR31 in FIG. 26(A) can be enlarged and displayed as an enlarged reserved item list area AR311 as shown in FIG. 26(B). The operation to display the enlarged reserved item list area AR311 may be an operation of touching any position within the reserved item list area AR31. Alternatively, it may be an operation of touching the reserved item count area AR33 located on the checkout screen in FIG. 26(A).

By displaying the expanded pending item list area AR311 in this way, the store clerk can easily see the list items of pending items and can easily understand what pending items are available. Also, for example, depending on the position of the display unit 405 in the settlement device 40, the position of the display unit 405 may be high, and particularly when the display surface of the display unit 405 is vertically long, it may be difficult to operate the upper side of the display surface of the display unit 405. Therefore, in the example of Figure 26 (B), the expanded pending item list area AR311 is arranged on the lower side of the display surface. This makes it easier for the store clerk to operate the expanded pending item list area AR311.

When a store clerk processes a pending item, the clerk operates the call button BT31 arranged on the list item of the pending item to be processed among the list items in the expanded pending item list area AR311 displayed as in FIG. 26 (B). In response to the operation of the call button BT31, the pending item processing screen AR4 is displayed in place of the expanded pending item list area AR311 as shown in FIG. 26 (C). The pending item processing screen AR4 displayed in this way processes the pending item indicated by the list item of the operated call button BT31.

Furthermore, if the pending product to be processed is an unreadable product, the settlement device 40 displays the captured image area AR43 on the same screen as the pending product processing screen AR4, as shown in the same Figure 26 (C). Note that the figure shows an example in which one image is displayed in the captured image area AR43. However, even in the case of a vertically long screen like this, a predetermined number of captured images may be displayed in the captured image area AR43 according to the reading failure status (according to the pending subcategory number), as shown in Figures 20 and 25.

In the example of FIG. 20 etc., the captured image area AR43 was arranged within the pending item processing screen AR4. In contrast to this, in FIG. 26(C), since the display surface of the display unit 405 is vertically long, the captured image area AR43 is not included in the pending item processing screen AR4. This allows the captured image area AR43 to be enlarged and arranged to a larger size than when it is arranged within the pending item processing screen AR4, making the captured image easier to see. Also, as shown in FIG. 26(C), on the vertically long display surface, the captured image area AR43 is arranged on the upper side, and the pending item processing screen AR4 is arranged on the lower side. This makes it easier for the store clerk to operate the pending item processing screen AR4.

[Third Modification]
In the above embodiment, an example was given in which the captured image area AR43 is arranged on the pending product processing screen for processing pending products that have not been read correctly, and the captured image area AR43 is not arranged on the pending product processing screen for pending products that have not been read correctly. However, in this modified example, the captured image area AR43 may also be arranged on the pending product processing screen for pending products that have not been read correctly. In this case, the mobile terminal 60 may be configured to transmit product registration information including the captured image for registered products whose barcodes have been successfully read.

[Fourth Modification]
The captured image area AR43 may be displayed as an independent screen different from the reserved item processing screen, for example, depending on the operation of the store clerk on the settlement device 40.

[Fifth Modification]
In addition, when the transaction start instruction button BT10 arranged on the start screen of FIG. 10(A) is operated to start product registration, the mobile terminal 60 may display a warning screen informing the customer in advance that "images at the time of product registration may be displayed on the settlement device". By displaying such a warning screen, for example, fraudulent product registration can be deterred. Alternatively, instead of the warning screen, the mobile terminal may display a consent screen to confirm the customer's consent to "images at the time of product registration may be displayed on the settlement device". In the consent screen, buttons for accepting an operation to confirm intention, such as a "Yes" button and a "No" button, are arranged. If the "Yes" button is operated, the customer may be treated as a customer who does not commit fraud, and the cancellation may be confirmed as it is without registering the canceled product as a reserved product.

[Sixth Modification]
In the above embodiment, a situation where the barcode cannot be read successfully is treated as a product registration error. In this modified example, in addition to a situation where the barcode cannot be read successfully, a situation where the barcode is successfully read but the target product cannot be added to the shopping cart 80 may also be treated as a product registration error. Note that, for example, rice over a certain weight or canned drinks in packs of six are often placed at the bottom of the cart. Therefore, for such products, even if it is detected that they cannot be added to the shopping cart 80, they may not be treated as a product registration error.

[Seventh Modification]
For example, due to a communication error or a communication speed limit set on the mobile terminal 60, when the unreadable information is sent in step S132 of FIG. 14, the captured image may not be sent to the transaction management device 20. In such a case, the transaction management device 20 may receive the unreadable information including the reserved subcategory number but not the captured image. Alternatively, the mobile terminal 60 resends the unreadable information not including the captured image, so that the transaction management device 20 can receive the unreadable information including the reserved subcategory number but not the captured image. In this way, even in the case of a communication error or a communication speed limit set on the mobile terminal 60, the transaction management device 20 may register at least the reserved subcategory number of the unreadable product in the cart information as much as possible.
As a result, the settlement device 40, by using settlement information based on the cart information, can display a pending product processing screen for the unreadable product as shown in Fig. 20 etc., although the captured image is not displayed in the captured image area AR43, for example. In this case, a message that can inform the store clerk that the unreadable product is a product, such as "Are there any products that were not scanned?", may be displayed in the blank area of the pending product processing screen where the captured image area AR43 is not displayed.

[Eighth Modification]
The occurrence of the reading failure may be displayed on a terminal (store operating terminal) operating in the store other than the settlement device 40, such as the transaction status management device 50. For example, the transaction status management device 50, which is one of the store operating terminals, can manage the status of each transaction in the store by communicating with the transaction management device 20, for example. Thus, when a pending product is registered, the transaction status management device 50 may acquire information identifying the customer to whom the pending product corresponds, product information of the target product, amount information, etc. at the time of the registration of the pending product, and display the acquired information. Then, for example, when a store clerk performing management work using the transaction status management device 50 confirms that there is no problem with the displayed product, the store clerk may perform an operation such as canceling the registration as a pending product and re-registering it as a normal product.

[Ninth Modification]
The imaging device used in response to product registration is not particularly limited as long as it is capable of reading product barcodes, is capable of capturing images of the contents of the shopping basket 80, etc., and is capable of outputting captured images to the mobile terminal 60, etc. Such an imaging device may be capable of capturing images of the inside of a customer's own bag, instead of the shopping basket 80 provided by the store, for example.

[Tenth Modification]
The mobile terminal 60 may be held by the customer to register products without being set in the shopping cart 70. In this case, it is sufficient that the insertion of products into the shopping cart 70 is detected. For example, a device (terminal) equipped with an imaging means capable of detecting the insertion of products into the shopping basket 80 (shopping bag 90) may be installed, or a device equipped with a weighing means capable of detecting the insertion of products into the shopping basket 80 (shopping bag 90) may be installed. Even in a mode in which the insertion of products into the shopping cart 70 is not detected (a mode in which no device other than the mobile terminal 60 is used), the mobile terminal 60 cannot execute the transition to the reading failure process from step S102 (YES) in FIG. 12 related to the insertion of products, but can execute the transition to the reading failure process from step S103 (YES) in FIG. 12 not related to the insertion of products. In other words, at least the reserved product with the reserved subcategory number "21" (pseudo scan (covered, etc.)) and the reserved product with the reserved subcategory number "23" (defective barcode) can be recognized.

[Eleventh Modification]
In the above embodiment, the preset key is not displayed in the first reading failure state, and is displayed in the second reading failure state (FIG. 14). However, the preset key may be displayed in the first reading failure state. For example, the processes of steps S121 to S124 in FIG. 14 may not be required. Alternatively, the preset key may not be displayed in the Nth reading failure state (N is an integer of 2 or more), and may be displayed in the (N+1)th reading failure state. In this case, instead of a flag (re-read operation instruction completion flag) indicating whether or not reading has been performed after the reading failure state, a counter may be used to count the number of readings after the reading failure state. The number of times (0 or more) until the preset key is displayed may be different depending on the customer. For example, the past purchase history and the number of occurrences of the reading failure state may be stored for each customer, and the number of times (0 or more) until the preset key is displayed may be different depending on the past purchase history and the number of occurrences of the reading failure state. Also, instead of or in addition to the customer, the number of times (0 or more) until the preset key is displayed may be varied depending on the type of product.

[Twelfth Modification]
In the above embodiment, the products registered by reading the barcode and the products registered using the preset key are not particularly distinguished, but the products registered by reading the barcode and the products registered using the preset key may be stored and output (displayed, etc.) in a manner that allows them to be distinguished from each other. For example, in the cart information (registered products N), the products registered by reading the barcode and the products registered using the preset key may be stored in a manner that allows them to be distinguished from each other. For example, information indicating that the products registered using the preset key are products registered using the preset key may be stored in the cart information (registered product information (total)), the products registered by reading the barcode and the products registered using the preset key may be stored in a manner that allows them to be distinguished from each other. For example, the breakdown (number of items, estimated subtotal amount, etc.) of the products registered using the preset key may be stored. In addition, the mobile terminal 60 may display the products registered by reading the barcode and the products registered using the preset key in a manner that allows them to be distinguished from each other in a registered product list such as that shown in FIG. 10 (C). For example, the mobile terminal 60 may display, in the registered product list, information indicating that a product registered using a preset key is a product registered using a preset key. Furthermore, the settlement device 40 may display, in a settlement screen such as that shown in Fig. 18, products registered by reading a barcode and products registered using a preset key in a distinguishable manner. For example, the settlement device 40 may display, in the settlement screen, information indicating that a product registered using a preset key is a product registered using a preset key.

[Thirteenth Modification]
In the above embodiment, the mobile terminal 60 detects a read failure state indicating that a product whose product code has not been read has been secured by a customer (put into the shopping basket 80 (shopping bag 90)). However, instead of the mobile terminal 60, another device (e.g., the transaction status management device 50, the store controller 10, the transaction management device 20, etc.) may detect the read failure state. The transaction status management device 50 (similar to the store controller 10 and the transaction management device 20) may detect the read failure state, for example, as follows. The mobile terminal 60 notifies the transaction status management device 50 at each timing that the product code has been read and that the product has been put into the shopping basket 80. If the transaction status management device 50 receives a notification from the mobile terminal 60 that the product code has been read but receives a notification that the product has been put into the shopping basket 80, the transaction status management device 50 determines that a read failure state has occurred (i.e., detects a read failure state). When the transaction status management device 50 (same for the store controller 10 and transaction management device 20) detects a reading failure, it notifies the mobile terminal 60 that a reading failure has occurred. When the mobile terminal 60 is notified by the transaction status management device 50 (same for the store controller 10 and transaction management device 20) that a reading failure has occurred, it executes a reading failure process ( FIG. 14 ).

[Fourteenth Modification]
In the above embodiment, the imaging unit 606 of the mobile terminal 60 is used to read the product code and to determine whether a product has been inserted, but it is not necessary to use the imaging unit 606 for both. For example, a device equipped with an imaging means capable of detecting the insertion of a product into the shopping basket 80 (shopping bag 90) may be installed (for example, installed on each shopping cart 70, installed on the ceiling of the store, etc.). In other words, the imaging unit 606 of the mobile terminal 60 may be used to read the product code, and the insertion of the product may be detected using a device installed on the shopping cart 70 or on the ceiling of the store, etc.

[Fifteenth Modification]
In the above embodiment, the insertion of a commodity is determined by the imaging unit 606 (captured image), but it does not have to be determined by the captured image. For example, instead of or in addition to the imaging means, a weighing device (for example, installed in each shopping cart 70) capable of detecting the insertion of a commodity into the shopping basket 80 (shopping bag 90) may be used.

[Sixteenth Modification]
In the above embodiment, in the case of a reading failure, a message prompting the user to read is displayed regardless of the classification of the reserved subcategory numbers "21" to "24", and a preset key is displayed when the reading failure occurs again. However, the control may be different depending on the classification. For example, in the case of a certain reserved subcategory number (e.g., reserved subcategory number "24"), a preset key may be displayed without displaying a message prompting the user to read, and in the case of other reserved subcategory numbers (e.g., reserved subcategory numbers "21" to "23"), a message prompting the user to read may be displayed and then the preset key may be displayed. Also, for example, in the case of a certain reserved subcategory number (e.g., reserved subcategory number "24"), a preset key may be displayed (a message prompting the user to read may be displayed or a message prompting the user to read may not be displayed), and in the case of other reserved subcategory numbers (e.g., reserved subcategory numbers "21" to "23"), the product may be reserved without displaying the preset key (a message prompting the user to read may be displayed or a message prompting the user to read may not be displayed). This allows processing to be performed according to the situation (reading failure situation). In the above embodiment (where different processing is performed for each classification of reserved items), the process of determining the reserved subcategory number of the reserved item (step S130 in FIG. 14) may be performed earlier (for example, at the beginning of the reading failure process (S120)). Note that although the "certain reserved subcategory number" is reserved subcategory number "24" and the "other reserved subcategory numbers" are reserved subcategory numbers "21" to "23," the combination of the "certain reserved subcategory number" and the "other reserved subcategory numbers" is not limited to this. Also, instead of the two combinations of "certain reserved subcategory number" and "other reserved subcategory numbers," there may be three to five combinations.

[Seventeenth Modification]
In the above embodiment, the mobile terminal 60 displays the preset keys (candidate products), but instead of or in addition to the mobile terminal 60, other devices (e.g., the settlement device 40, the registered settlement device 30, etc.) may display the preset keys (candidate products). For example, the mobile terminal 60 may not display the preset keys, and the settlement device 40 may display the preset keys (candidate products) for each of the reserved products (reserved products with reserved subcategory numbers "21" to "24"). Also, for example, the mobile terminal 60 may display the preset keys (candidate products), and when registration using the preset keys is not performed (i.e., when the product becomes a reserved product), the settlement device 40 may display the preset keys (candidate products) for each of the reserved products (reserved products with reserved subcategory numbers "21" to "24").

[Eighteenth Modification]
In addition, all or part of each function (input/output, storage, processing (including judgment)) of each device (store controller 10, transaction management device 20, registration settlement device 30, settlement device 40, transaction status management device 50, mobile terminal 60, etc.) described above may be realized by other devices different from the device described as the execution subject of the function. For example, as the processing of the mobile terminal 60, at least part of the processing described using the left column of the sequence diagram of FIG. 9 and at least part of the processing described using the flowcharts of FIG. 12 to FIG. 15 may be executed by the transaction management device 20, the store controller 10, or the settlement device 40 instead of the mobile terminal 60. In addition, as the processing of the settlement device 40, at least part of the processing described using the middle column of the sequence diagram of FIG. 9 may be executed by the transaction management device 20, the store controller 10, or the mobile terminal 60 instead of the settlement device 40.
In relation to the above, the mobile terminal 60 may function as a so-called thin client specialized in an input/output interface with respect to the present shopping service (such as the product registration described in the sequence diagram of FIG. 9 and the flowcharts of FIG. 12 to FIG. 15). For example, instead of the mobile terminal 60 executing the shopping application (no need to install it on the mobile terminal 60), an external device (which may be the transaction management device 20 or another cloud server) may execute the shopping application. In other words, the mobile terminal 60 may accept various inputs (customer operations, detection by a device such as a camera), transmit input information (operation information, image information, etc.) to an external device, receive the processing results of the external device based on the input information (update screen information, device control information, etc.), and perform various outputs (display on a display unit, control of a device). Other devices (such as the settlement device 40) may also function as thin clients instead of or in addition to the mobile terminal 60.

<Overview of the embodiment>
[Technical field]
The present invention relates to a product sales data processing system, a mobile terminal, and a program.
[Background Art]
There is known a system in which a customer himself/herself registers an item using a mobile terminal equipped with a camera (see, for example, Japanese Patent Application Laid-Open No. 2003-233663).
[Prior Art Literature]
[Patent Documents]
[Patent Document 1] JP 2018-147252 A [Summary of the Invention]
[Problem to be solved by the invention]
However, various problems may arise in relation to reading the barcode when a customer registers a product by himself/herself. The present invention has been made in consideration of such circumstances, and has an object to appropriately deal with problems that arise when a customer registers a product by himself/herself.

[Means for solving the problems]
(A1) One aspect of this embodiment that solves the above-mentioned problems is a product sales data processing system (1) that registers products using a mobile terminal (e.g., mobile terminal 60) operated by a customer of a store, the product sales data processing system comprising: a reading means for reading a product code; an acquisition means for acquiring product information corresponding to the product code read by the reading means; a detection means (e.g., processing of step S102 in FIG. 12 ) for detecting a reading failure state indicating that a product whose product code has not been read by the reading means has been secured by a customer (e.g., a product whose product code has not been read has been placed in a shopping basket 80 (shopping bag 90)), and a display means for displaying candidate products in a selectable manner (e.g., displaying the products in FIG. 16 (D) and FIG. 17 (D)), when the detection means detects the reading failure state, and the acquisition means acquires product information of the candidate product selected by a selection operation by the customer.

The above configuration makes it possible to appropriately deal with problems that arise when a customer registers a product by himself. For example, problems related to the state of the barcode, including the presence or absence of a barcode, and various problems such as customer misreading the barcode and fraud can be appropriately dealt with. Misreading the barcode and fraud by a customer refers to, for example, a customer intending to read the barcode of a product but actually failing to do so, or maliciously putting a product into a shopping cart without having the barcode read.
(a) For example, in a system in which customers themselves register products, it is preferable to affix barcodes to loose products such as fruits and vegetables, just like other products. This is because customers can register loose products by reading the barcodes attached to the products, just like other products. However, depending on the shape of the product, the affixed barcode may become wrinkled, making it difficult to read. In addition, the task of affixing barcodes to loose products is itself a burden for the store.
According to the above configuration, candidate products are displayed if the reading of the barcode fails due to, for example, wrinkles. Also, according to the above configuration, candidate products are displayed even if the barcode cannot be read because, for example, there is no barcode attached. Therefore, there is no problem even if the barcode attached to the loose product is wrinkled, and furthermore, the work of attaching the barcode to the loose product can be omitted, eliminating the burden involved.
(b) In addition, if a customer is not familiar with reading barcodes or if the customer's reading action is not appropriate, the reading may fail and products that have not been read may be placed in the basket. Also, there is a possibility that a customer may accidentally forget to read the barcode and products that have not been read may be placed in the basket. Furthermore, there may be customers who intentionally fail to read the barcode or who place products in the basket without reading it.
According to the above configuration, in any situation, when the barcode reading fails or when a product is put into the basket without reading the barcode, the candidate products are displayed. Therefore, it is possible to prevent or inhibit the removal of unregistered products (so-called basket drop-off), regardless of whether it is done with good or bad intentions.
(c) Furthermore, products that are not registered at the product registration stage in the store are treated as so-called reserved products at the checkout stage. However, as the number of reserved products increases, the number of operations required for the reserved products at the checkout stage increases, which becomes a burden.
With the above configuration, when the barcode reading fails or when a product is added to the basket without reading the barcode, candidate products are displayed and the selected reserved product is registered. Therefore, the number of reserved products at the registration stage is reduced, and the operation at the checkout stage is reduced, reducing the burden on the customer.

(A2) One aspect of this embodiment is the product sales data processing system described in (A1), in which the display means displays information (e.g., the display of Figures 16(C) and 17(C)) encouraging the user to re-read the product code before displaying the candidate products when the detection means detects the unsuccessful reading situation, does not display the candidate products when the reading means successfully re-reads the product code, and displays the candidate products when the reading means fails to re-read the product code.

According to the above configuration, it is possible to display candidate products in an optimal manner. In other words, according to the above configuration, candidate products are not displayed when a reading failure is detected the first time, but are displayed when a reading failure is detected again. Therefore, for example, if a reading failure occurs by chance, the customer is prompted to re-scan and is made to register using the original (regular) registration method as much as possible (this also serves to inform the customer), and if the re-scanning does not work, the customer can be made to register from the candidate products as necessary.

(A3) One aspect of this embodiment is a product sales data processing system as described in (A1) or (A2), which includes a pending product determination means for determining that the product that caused the reading failure (e.g., the unreadable product) is a pending product for which payment is put on hold at checkout (the process is temporarily put on hold for confirmation by a store clerk, etc., at checkout) if a candidate product is not selected after the detection of the reading failure, and the pending product determination means determines that the product that caused the reading failure is not the pending product if a candidate product is selected after the detection of the reading failure.

The above configuration makes it possible to appropriately determine (sort) whether or not to place an incompletely read product on hold, depending on whether or not a candidate product has been selected.

(A4) One aspect of this embodiment is the product sales data processing system described in (A3), in which the acquisition means, if a selection operation of a candidate product is performed after the detection of the reading failure state, acquires product information of the candidate product selected by the selection operation as product information of the product that caused the reading failure state, and, if a selection operation of a candidate product is not performed after the detection of the reading failure state, acquires information used to detect the reading failure state (e.g., a captured image) as information about the pending product.

According to the above configuration, depending on whether or not a selection operation of a candidate product has been performed, information corresponding to the presence or absence of a selection operation can be acquired. In other words, if a selection operation of a candidate product has been performed, the selection operation is deemed to be correct and product information of the candidate product can be acquired, and if a selection operation of a candidate product has not been performed, a captured image or the like can be acquired as information regarding the reserved product.
In addition, if no selection operation of a candidate product is performed after a reading failure situation is detected, information regarding the reading failure situation (e.g., a reserved sub-category number, etc.) may be obtained instead of or in addition to the information used to detect the reading failure situation.

(A5) One aspect of this embodiment is a product sales data processing system according to any one of (A1) to (A4), in which, when the detection means detects the reading failure state, products sold in a predetermined sales method (e.g., sold individually) are displayed as candidate products so that they can be selected.

According to the above configuration, suitable candidate products can be displayed. For example, according to the above configuration, in a case where a reading error situation is likely to occur for a product sold in a certain sales method, suitable products (products that are likely to cause a reading error situation) can be displayed as candidate products.

(A6) One aspect of this embodiment is a product sales data processing system according to any one of (A1) to (A4), in which when the detection means detects the reading failure state, products corresponding to the customer's position in the store are displayed as candidate products so that they can be selected.

According to the above configuration, suitable candidate products can be displayed. For example, according to the above configuration, when the reading failure state is detected near a certain display shelf, suitable products (products displayed on that display shelf) can be displayed as candidate products.

(A7) One aspect of this embodiment is a product sales data processing system as described in any one of (A1) to (A6), which includes an imaging means for imaging a reserved product that has been reserved by a customer, the imaging means being capable of imaging the reserved product as well as the reading operation in which the customer reads the product code, the reading means reading the product code from the captured image of the reading operation, and the detection means detecting the reading failure situation based on a change in the captured image of the reserved product.

The above configuration can be easily achieved using the imaging means.

(A8) One aspect of this embodiment is a mobile terminal operated by a customer in a store, comprising: a reading means for reading a product code; an acquisition means for acquiring product information corresponding to the product code read by the reading means; a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by the customer; and a display means for displaying candidate products in a selectable manner when the detection means detects the read failure state, wherein the acquisition means acquires product information of the candidate product selected by the customer's selection operation.

The above configuration can achieve the same effect as (A1).

(A9) One aspect of this embodiment is a program for causing a mobile terminal operated by a customer in a store to function as a computer, the program causing the computer to function as a reading means for reading a product code, an acquisition means for acquiring product information corresponding to the product code read by the reading means, a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by the customer, and a display means for displaying candidate products in a selectable manner when the detection means detects the read failure state, the acquisition means acquiring product information of the candidate product selected by the customer's selection operation.

The above configuration can achieve the same effect as (A1).

(B1) One aspect of this embodiment that solves the above-mentioned problems is a product sales data processing system (1) in which a product is registered by a mobile terminal (e.g., mobile terminal 60) and a settlement device (40) performs settlement processing corresponding to the product registered by the mobile terminal, the product sales data processing system having an imaging means (e.g., imaging unit 606) that is capable of capturing an image of the situation regarding the product registration by a customer, a detection means that detects a predetermined reading failure situation in which the registration (reading) of the product is unsuccessful while the product is being registered according to a transaction, and a display control means that displays an image (e.g., an image in an image area AR43) corresponding to the reading failure situation detected by the detection means based on the image obtained by the imaging means when the settlement device settles the product registered according to the transaction.

According to the above configuration, if a reading failure occurs while a customer is registering a product using the mobile terminal 60, an image of the reading failure can be displayed. This makes it possible to, for example, prevent reading failures (for example, to prevent fraudulent acts in product registration) and appropriately deal with products that correspond to reading failures, thereby making it possible to appropriately deal with problems that arise when customers register products themselves.

(B2) One aspect of this embodiment is the product sales data processing system described in (B1), in which the display control means classifies a reading failure situation as a situation in which a product is inserted into a product insertion section (e.g., shopping cart 80) without the product information being read (non-scan insertion), and causes multiple captured images corresponding to a predetermined period before and after the product is inserted into the product insertion section to be displayed.

According to the above configuration, when the classification of the reading failure situation is non-scan input (in the case of a reserved product with the reserved sub-category number "24"), it is possible to display an image of the contents of the shopping basket 80 from before the target product is inserted until it is inserted. This allows the state transition of the contents of the shopping basket 80 when the product is inserted without being scanned to be accurately understood.

(B3) One aspect of this embodiment is the product sales data processing system described in (B2), which displays a portion of a specific captured image among the multiple captured images that differs from the other captured images.

According to the above configuration, a frame FL showing the part of the additionally inserted product can be placed in an image of the additionally inserted product among the multiple images displayed when the classification of the reading failure state is non-scan-inserted. This allows the additionally inserted product to be immediately identified.

(B4) One aspect of this embodiment is a product sales data processing system as described in any one of (B1) to (B3), in which the display control means displays captured images with different imaging subjects according to the classification of the reading failure situation detected by the detection means (reserved sub-classification numbers "21" to "24").

According to the above configuration, by displaying a captured image of an object corresponding to the classification of the reading failure situation, it is possible to display an appropriate captured image as evidence according to the reading failure situation.

(B5) One aspect of this embodiment is a product sales data processing system as described in any one of (B1) to (B4), in which the display control means displays an image captured when an operation to read the product information was being performed in the case of a reading failure status due to classification that the product information could not be read successfully (in the case of a reserved product with a reserved subcategory number of "21" to "24").

According to the above configuration, for example, in the case of a false scan or a defective barcode, it is possible to display an image of a product that was within the imaging range of the imaging unit 606 when it was determined that the reading was incomplete. This makes it possible to identify the product whose barcode should have been read in the case of a false scan, a defective barcode, or the like.

(B6) One aspect of this embodiment is a product sales data processing system described in any one of (B1) to (B5), in which the display control means displays an image captured corresponding to the unreadable product together with a screen (e.g., a pending product processing screen AR4) for processing the pending product as an unreadable product at the settlement device.

With the above configuration, if the reserved item is an unreadable item, the captured image area AR43 can be displayed together with the reserved item processing screen AR4 that is displayed so that the store clerk can process the reserved item. This allows the store clerk to properly process the reserved item as an unreadable item.

(B7) One aspect of this embodiment is a mobile terminal (e.g., mobile terminal 60) capable of registering products, which includes an imaging means capable of capturing images of the status of product registration by a customer, a detection means for detecting a predetermined reading failure status in which product registration (reading) is unsuccessful while the product is being registered according to a transaction, and a display control means for displaying product information of the registered product and an image captured by the imaging means (e.g., a product registration screen) and displaying information about the detected reading failure status (e.g., a guidance image WD2) in response to the detection of the reading failure status by the detection means.

According to the above configuration, when a reading failure occurs, a display related to the reading failure is displayed on the mobile terminal 60. This allows the customer to understand that a reading failure has occurred. Furthermore, this has the effect of deterring customers who attempt to commit fraud.

In addition, a program for implementing the functions of the above-mentioned store controller 10, transaction management device 20, registration settlement device 30, settlement device 40, transaction status management device 50, mobile terminal 60, etc. may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing of each of the above-mentioned devices. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer system" may include multiple computer devices connected via a network including the Internet, WAN, LAN, dedicated lines, and other communication lines. In addition, the "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. In addition, the recording medium also includes a recording medium provided inside or outside and accessible from a distribution server to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format executable by the terminal device. In other words, the format in which the program is stored in the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a format executable by the terminal device. The program may be divided into multiple parts, downloaded at different times, and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a memory that holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted over a network. The program may also be a program for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

1 Product sales data processing system, 10 Store controller, 20 Transaction management device, 40 Payment device, 50 Transaction status management device, 60 Mobile terminal, 70 Shopping cart, 80 Shopping basket

Claims (12)

A product sales data processing system for registering products using a mobile terminal operated by a customer of a store,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state,
the display means displays, when the reading failure state is detected, a product having a predetermined sales method as a candidate product in a selectable manner;
The merchandise sales data processing system is characterized in that the acquiring means acquires merchandise information of a candidate merchandise selected by a customer's selection operation. A product sales data processing system for registering products using a mobile terminal operated by a customer of a store,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state,
the display means displays, when the reading failure state is detected, a product corresponding to the position of the customer in the store as a candidate product in a selectable manner;
The merchandise sales data processing system is characterized in that the acquiring means acquires merchandise information of a candidate merchandise selected by a customer's selection operation. A product sales data processing system for registering products using a mobile terminal operated by a customer of a store,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state ;
An imaging means for imaging the reserved product reserved by the customer;
Equipped with
The imaging means is capable of imaging the reserved product as well as a reading operation of the customer reading the product code;
The reading means reads a product code from a captured image obtained by capturing the reading operation,
The detection means detects the reading failure state based on a change in a captured image of the reserved product,
The merchandise sales data processing system is characterized in that the acquiring means acquires merchandise information of a candidate merchandise selected by a customer's selection operation. The display means is
A product sales data processing system as described in any one of claims 1 to 3, characterized in that when the detection means detects the reading failure state, information prompting the user to re-read the product code is displayed before displaying the candidate products, and when the reading means successfully re-reads the product code, the candidate products are not displayed, and when the reading means fails to re-read the product code, the candidate products are displayed. a reserved commodity determination means for determining, when no selection operation of a candidate commodity is performed after the detection of the reading failure state, that the commodity causing the reading failure state is a reserved commodity whose payment is to be suspended at the time of payment;
The reserved product determination means
The product sales data processing system according to any one of claims 1 to 4, characterized in that if a selection operation of a candidate product is performed after the detection of the reading failure state, the product that caused the reading failure state is determined to be not the reserved product. The acquisition means includes:
When a selection operation of a candidate product is performed after the detection of the reading failure state, product information of the candidate product selected by the selection operation is acquired as product information of the product that caused the reading failure state;
The product sales data processing system according to claim 5 , characterized in that if no selection operation of a candidate product is performed after the detection of the reading failure situation, the information used to detect the reading failure situation is obtained as information regarding the reserved product. A mobile terminal operated by a customer of a store,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state,
the display means displays, when the reading failure state is detected, a product having a predetermined sales method as a candidate product in a selectable manner;
The mobile terminal is characterized in that the acquiring means acquires product information of a candidate product selected by a selection operation of a customer. A mobile terminal operated by a customer of a store,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state,
the display means displays, when the reading failure state is detected, a product corresponding to the position of the customer in the store as a candidate product in a selectable manner;
The mobile terminal is characterized in that the acquiring means acquires product information of a candidate product selected by a selection operation of a customer. A mobile terminal operated by a customer of a store,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state ;
An imaging means for imaging the reserved product reserved by the customer;
Equipped with
The imaging means is capable of imaging the reserved product as well as a reading operation of the customer reading the product code;
The reading means reads a product code from a captured image obtained by capturing the reading operation,
The detection means detects the reading failure state based on a change in a captured image of the reserved product,
The mobile terminal is characterized in that the acquiring means acquires product information of a candidate product selected by a selection operation of a customer. A program for causing a mobile terminal operated by a customer of a store to function as a computer,
The computer,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display unit that displays candidate products in a selectable manner when the detection unit detects the reading failure state;
the display means displays, when the reading failure state is detected, a product having a predetermined sales method as a candidate product in a selectable manner;
The program, wherein the acquiring means acquires product information of a candidate product selected by a customer's selection operation. A program for causing a mobile terminal operated by a customer of a store to function as a computer,
The computer,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display unit that displays candidate products in a selectable manner when the detection unit detects the reading failure state;
the display means displays, when the reading failure state is detected, a product corresponding to the position of the customer in the store as a candidate product in a selectable manner;
The program, wherein the acquiring means acquires product information of a candidate product selected by a customer's selection operation. A program for causing a mobile terminal operated by a customer of a store to function as a computer,
The computer,
A reading means for reading a product code;
an acquisition means for acquiring product information corresponding to the product code read by the reading means;
a detection means for detecting a read failure state indicating that a product whose product code has not been read by the reading means has been reserved by a customer;
a display means for displaying candidate products in a selectable manner when the detection means detects the reading failure state ;
An imaging means for imaging the reserved product reserved by the customer;
and make it work.
The imaging means is capable of imaging the reserved product as well as a reading operation of the customer reading the product code;
The reading means reads a product code from a captured image obtained by capturing the reading operation,
The detection means detects the reading failure state based on a change in a captured image of the reserved product,
The program, wherein the acquiring means acquires product information of a candidate product selected by a customer's selection operation.

Images