JP2026002572A - Information processing device, information processing system, information processing method and program - Google Patents

Abstract

A route from a worker's current location to an item's storage location on a storage shelf is provided in real time.
[Solution] The information processing device disclosed herein comprises an acquisition unit that continuously acquires images taken by a work terminal over time and acquires multiple item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; a spatial location information generation unit that estimates the three-dimensional relative positions of the items and the work terminal in real time based on the item identification codes and storage location identification codes and generates three-dimensional spatial location information; a spatial map creation unit that integrates the spatial location information to create a spatial map; and a route information generation unit that generates route information that uses the spatial map to guide the route from the position of the work terminal to the storage location of the item identified based on the item identification code.
[Selected Figure] Figure 1

Description

This disclosure relates to an information processing device, an information processing system, an information processing method, and a program.

Patent Document 1 discloses an item storage assistance device that uses AR/VR. In Patent Document 1, individual product identification information is attached to products stored in a warehouse. In addition, shelf address identification information, which is information about the location within the warehouse, is attached to storage shelves within the warehouse.

When the individual product identification information for a product is acquired using the HMC (head-mounted camera) of the HMD (head-mounted display) worn by the worker, the information processing device searches a database using the individual product identification information as a key to identify storage location information for the product. Upon receiving the storage location information, the HMD displays information related to the location where the product should be stored so that it can be seen by the worker. The worker can confirm the storage location instructions and begin work.

When a worker who has completed storage work visually recognizes the shelf address identification information attached to the storage shelf, the HMC mounted on the HMD acquires the shelf address identification information for that product. The information processing device compares and verifies the shelf address identification information with storage location information obtained by accessing a database (DB).

When changing a storage location, the worker actually moves the product's storage location while giving instructions to the HMD to change the product's storage location. When the worker visually checks the product's individual product identification information and then looks at the shelf address identification information associated with the changed storage location, the HMC mounted on the HMD acquires the product's individual product identification information and the changed storage location information. The information processing device receives the individual product identification information and the changed storage location information, and accesses the DB to update the data.

Japanese Patent Application Laid-Open No. 2015-124023

In warehouse product management systems, identification information such as barcodes is often attached to storage shelves and products. By having workers read each piece of identification information in turn, the location where the worker stored the product and the product stored in that location can be determined.

Patent Document 1 proposes technology that links individual product identification information with shelf address identification information when a worker handles an item, but it is not possible to obtain the worker's location within the warehouse in real time. There is a need for technology that can provide real-time guidance on the route from the worker's current location to the product's storage location on the storage shelf.

In consideration of the above-mentioned problems, the purpose of this disclosure is to provide an information processing device, information processing system, information processing method, and program that can provide real-time guidance on the route from a worker's current location to the storage location of an item on a storage shelf.

The information processing device disclosed herein includes an acquisition unit that continuously acquires images captured by a work terminal over time and acquires multiple item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; a spatial location information generation unit that estimates the relative three-dimensional positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes and generates three-dimensional spatial location information; a spatial map creation unit that integrates the spatial location information to create a spatial map; and a route information generation unit that uses the spatial map to generate route information that provides guidance on a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

The information processing system disclosed herein includes a work terminal that generates images including item identification codes assigned to items and storage location identification codes assigned to each storage location in warehouse equipment by capturing images continuously over time, and an information processing device to which the images are input. The information processing device includes an acquisition unit that continuously acquires the images over time and acquires multiple item identification codes and storage location identification codes, a spatial location information generation unit that estimates the relative three-dimensional positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes to generate three-dimensional spatial location information, a spatial map creation unit that integrates the spatial location information to create a spatial map, and a route information generation unit that uses the spatial map to generate route information that provides guidance on a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

The information processing method disclosed herein involves a computer performing the following processes: acquiring images captured by a work terminal in succession over time, and acquiring multiple item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; estimating the relative three-dimensional positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes, and generating three-dimensional spatial location information; integrating the spatial location information to create a spatial map; and generating route information using the spatial map to guide users along a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

The program disclosed herein causes a computer to perform the following processes: continuously acquiring images captured by a work terminal over time, and acquiring multiple item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; estimating the relative three-dimensional positions of the items and the work terminal in real time based on the item identification codes and storage location identification codes, and generating three-dimensional spatial location information; integrating the spatial location information to create a spatial map; and generating route information that uses the spatial map to guide the user on a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

This disclosure makes it possible to provide real-time guidance on the route from the worker's current location to the storage location of an item on a storage shelf.

FIG. 1 is a diagram illustrating a configuration of an information processing device according to the present disclosure. 1 is a flowchart of an information processing method according to the present disclosure. 1 is a block diagram illustrating a configuration of an information processing system according to the present disclosure. 1 is a block diagram showing the overall configuration of an information processing system according to the present disclosure. FIG. 1 is a diagram illustrating an example of a warehouse to which an information processing system is applied. FIG. 5 is a block diagram showing the configuration of the work terminal of FIG. 4 . FIG. 5 is a block diagram showing the configuration of the management terminal of FIG. 4. FIG. 5 is a block diagram showing the configuration of the server in FIG. 4 . FIG. 10 is a diagram illustrating an example of layout information. FIG. 10 is a diagram illustrating an example of product identification information. FIG. 10 is a diagram illustrating an example of spatial position information. FIG. 10 is a diagram showing a flow of preparation for product management operations in the information processing system. FIG. 10 is a diagram showing a flow in which a manager issues a picking work instruction to a worker. 10 shows a workflow in which a worker operates a work terminal and views work information. This shows the internal processing flow of the system that is carried out while the worker is working.

Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each drawing, identical or corresponding elements are designated by the same reference numerals, and duplicate descriptions will be omitted as necessary for clarity.

This disclosure relates to technology for managing items in warehouses and supporting worker work. The following embodiment illustrates an example in which a worker patrols a warehouse, storing products on shelves, and retrieving (picking) products.

Generally, a warehouse management system (WMS) is used to manage products in a warehouse. In many cases, storage shelves are affixed with shelf position identification codes such as barcodes that indicate the shelf's location within the warehouse. Products are also affixed with product identification codes such as barcodes (JAN (Japanese Article Number) codes).

In the comparative example, a worker sequentially reads the shelf location identification code and product identification code using a handheld terminal equipped with a barcode reader, and the management system links the product to the location of the storage shelf where the product is stored. However, this method requires a lot of work from the worker, and has the disadvantage that the information on the location where the product is stored is not pinpointed. When there are multiple products around a single storage shelf, the task of selecting a specific product from among them is left to the worker.

Furthermore, the location of the worker can be determined from the position within the warehouse of the shelf location identification code read by the worker, but this merely indicates the location where the worker was when handling the product. If it is necessary to use augmented reality (AR) or mixed reality (MR) technology to pinpoint the location from the worker's current position to the storage location of the product, it is not appropriate to use the position within the warehouse of the shelf location identification code read by the worker as the location of the worker.

Furthermore, many handheld terminals lack real-time capabilities, meaning that data can only be downloaded when connected to a dock attached to a management device. This presents a barrier to warehouse managers being able to give timely instructions to workers.

In a comparative example, there are products that use a smartphone's camera function to read barcodes and QR codes (registered trademark) to manage products, but these are merely a substitute for handheld terminals. In other words, the smartphone is only used to link "storage shelves" with "products," whether it is to read one barcode at a time at any time, or multiple barcodes at the same time.

The first objective of this embodiment is to provide real-time guidance on the route from the worker's current location to the product's storage location on the storage shelf.

Furthermore, in the comparative example, the storage location of a product is determined by the shelf location identification code, which means that the actual location of the product cannot be pinpointed. For example, a widely used method is to attach RFID (radio frequency identification) tags or BLE (Bluetooth Low Energy) tags to products to identify their location. However, such tags are expensive, and some products require battery replacement, which increases the cost of simple devices. Furthermore, there is the drawback that the cost-saving effect is limited due to the need to attach tags to products correctly and to constantly manage tags to check for dead batteries, etc.

A second objective of this embodiment is to pinpoint the location where a product is stored without using devices such as tags.

Embodiment 1.
An example configuration of an information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the information processing device 1 according to the present disclosure. As shown in Fig. 1, the information processing device 1 includes an acquisition unit 2, a spatial position information generation unit 3, a spatial map creation unit 4, and a route information generation unit 5. The information processing device 1 identifies a storage location in a warehouse where a product is to be stored.

Here, a storage location refers to a location within a warehouse where a product is stored. For example, if a warehouse is equipped with multiple shelves, a storage location can refer to a specific storage space on the shelf. A storage space on a shelf can refer to a space for storing products separated by shelves or the like. Each storage location is assigned shelf location identification information (storage location identification information) using an identification code such as a barcode or QR code (registered trademark), which is necessary to identify the location of the storage location.

In addition, each product handled by the worker is assigned product identification information (item identification information) using an identification code such as a barcode or QR code (registered trademark). The product identification information may be a JAN code. The product identification information includes information necessary to identify the storage location of the product. Note that the product identification information may also include information for uniquely identifying the product.

The product identification information may include, by way of example, at least some of the following information:
- Product name - Product height/width/depth dimensions - Product weight - Product shape - Shipper information - Information indicating the product storage location specified by the shipper - Product entry date - Planned product storage period information - Expiration date management information for managing product expiry dates, best-before dates, use-by dates, etc. - Invoice number, tracking number, etc. - Product quantity information - Information on the number of products packed in one pallet

The information processing device 1 can communicate with external devices in various ways via a network. Specifically, the information processing device 1 can be connected via a communication line such as the Internet to work terminals carried by workers who perform tasks such as storing and picking goods in a warehouse. However, the communication line is not limited to the Internet, and may be a combination of the Internet and another communication line, or a communication line other than the Internet. Communication methods that can be used include, for example, wireless communication such as 4G (Generation), 5G, local 5G, Wi-Fi (registered trademark), and LTE (Long Term Evolution), or wired communication. However, the communication method is not limited to these examples.

By taking continuous photographs over time using a work terminal, a worker can generate an image that includes the product identification information attached to the product and the shelf position identification information attached to each storage position on a shelf within the warehouse. For example, a worker can move around the warehouse while taking photographs of the surrounding environment using the work terminal. Note that an "image that includes product identification information and shelf position identification information" can be an image in which both product identification information and shelf position identification information are included in a single image, or an image in which multiple images taken in succession include product identification information and shelf position identification information.

The acquisition unit 2 acquires images captured by the work terminal in a time-series manner. As a result, the acquisition unit 2 acquires multiple pieces of product identification information and shelf location identification information. In other words, the acquisition unit 2 may acquire an image that simultaneously includes product identification information and shelf location identification information within a single image, or may acquire product identification information and shelf location identification information by sequentially acquiring multiple images that are time-series.

The spatial position information generation unit 3 estimates the relative three-dimensional positions of the product and the work terminal in real time based on the product identification information and shelf position identification information contained in the acquired image, and generates three-dimensional spatial position information. Note that since the position of each storage shelf in the warehouse is determined in advance, the three-dimensional spatial position information indicating the shelf position can be stored in advance in a memory unit (not shown).

The spatial position information generation unit 3 can estimate the three-dimensional position of the work terminal that read the identification codes from the positions of the identification codes indicating product identification information and shelf location identification information within the image. The spatial position information generation unit 3 can also identify products based on the product identification information contained in the image, and estimate the position of the identified product relatively from the positions of the identification codes indicating product identification information and shelf location identification information within the image, and the estimated position of the work terminal. In other words, the spatial position information generation unit 3 can pinpoint the actual position of a product.

The methods used to estimate the location of the work terminal and the location of the product are not particularly limited. For example, a method can be used in which information about the size of the identification code and the location where the identification code is attached is registered in advance, and coordinates are estimated based on the position and distortion of the arrangement of pixels representing the identification code or storage shelf in the image. Alternatively, a method can be used in which photogrammetry is performed from the frame differences between consecutive captured images. SLAM (Simultaneous Localization and Mapping) can be used, which uses a 3D sensor to estimate self-location and create a spatial map.

The spatial map creation unit 4 creates a spatial map by integrating the generated spatial position information. For example, the spatial map creation unit 4 creates a spatial map by integrating and overlaying the identification codes indicating product identification information and the positions of the identification codes indicating shelf position identification information contained in the image, the estimated position of the work terminal, and the estimated position of the identified product. For example, the spatial map creation unit 4 can make adjustments to unify the coordinate system and align the positions of each identification code, work terminal, and product to create a three-dimensional spatial map.

The route information generation unit 5 generates route information using a spatial map to guide the route from the location of the work terminal to the storage location of the product identified based on the product identification information. The storage location of the product to be worked on can be identified based on the product identification information described above. For example, if the work to be performed by the worker is product storage work, the route information generation unit 5 displays on the spatial map the storage location for storing the product to be worked on.

The generated route information is output to the work terminal. This allows the worker to carry out their work while checking a spatial map that overlays multiple pieces of information, including the estimated locations of the product and work terminal, and information on the storage location of the product being worked on.

Next, the processing performed by the information processing device 1 will be described with reference to FIG. 2. FIG. 2 is a flowchart of the information processing method according to the present disclosure. First, the information processing device 1 acquires images captured by the work terminal in succession over time, and acquires multiple items of item identification information attached to the items and shelf position identification information attached to each storage position in the warehouse equipment (S11). Then, based on the item identification information and shelf position identification information, the information processing device 1 estimates the three-dimensional relative positions of the items and the work terminal in real time, and generates three-dimensional spatial position information (S12).

The information processing device 1 then integrates the spatial position information to create a spatial map (S13). The information processing device 1 then generates route information using the spatial map to guide the route from the location of the work terminal to the storage location of the item to be worked on, identified based on the product identification information (S14). In this way, the information processing device 1 makes it possible to obtain route information that guides the route from the worker's current location to the location of the item to be worked on in real time, without the need for the cumbersome task of reading identification codes, simply by capturing images containing identification codes indicating product identification information and shelf position identification information while the worker is moving.

The information processing device 1 includes a processor, memory, and storage device, which are not shown in the figure. The storage device stores a program that causes a computer to execute each process of the management method according to embodiment 1. The processor loads the program from the storage device into memory and executes the program. In this way, the processor realizes the functions of the acquisition unit 2, spatial position information generation unit 3, spatial map creation unit 4, and route information generation unit 5.

Each component of the information processing device 1 may be implemented using dedicated hardware. Furthermore, some or all of the components of each device may be implemented using general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be implemented using a single chip, or multiple chips connected via a bus. Some or all of the components of each device may be implemented using a combination of the above-mentioned circuits, etc., and a program. Furthermore, processors such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), FPGAs (Field-Programmable Gate Arrays), and quantum processors (quantum computer control chips) may be used.

Furthermore, when some or all of the components of the information processing device 1 are realized by multiple devices, circuits, etc., the multiple devices, circuits, etc. may be centrally or decentralized. FIG. 3 is a block diagram showing the configuration of an information processing system 101 according to the present disclosure. As shown in FIG. 3, the information processing system 101 includes an acquisition unit 102, a spatial position information generation unit 103, a spatial map creation unit 104, and a route information generation unit 105. As described above, the detailed operation of each component of the information processing system 101 corresponds to the operation of each component of the information processing device 1. The devices, circuits, etc. that realize the components of FIG. 3 may be realized as a client-server system, a cloud computing system, or other system connected via a communications network. Furthermore, the functions of the information processing device 1 may be provided in a SaaS (Software as a Service) format.

Embodiment 2.
4 is a block diagram showing the overall configuration of an information processing system 100 according to the present disclosure. The information processing system 100 includes a server 10, a work terminal 20, and a management terminal 30. The server 10, the work terminal 20, and the management terminal 30 are each connected via a network N. Here, the network N is a wired or wireless communication line, for example, the Internet.

Figure 5 is a diagram showing an example of a warehouse W to which the information processing system 100 is applied. Here, route information is generated based on images taken by the work terminals 20 carried by workers in the warehouse W.

As shown in Figure 5, warehouse W is equipped with multiple storage shelves 40. Each storage shelf 40 is equipped with multiple shelves 41 for storing products. In the example shown in Figure 5, a shelf position identification code 42 is attached to the left end of each shelf 41. In other words, the space separated by each shelf 41 becomes a storage space for storing products 43. Note that the storage space of each shelf 41 may be divided, and a shelf position identification code 42 may be attached to each divided space. Products 43 are placed in the storage positions of each shelf 41. Each product 43 has a product identification code 44 attached to it.

Carrying a work terminal 20, a worker stores products brought into the warehouse in various storage locations, and retrieves designated products from storage locations according to instructions on slips and other documents. While moving around the warehouse W, the worker can use the work terminal 20 carried by the worker to continuously capture images containing the shelf location identification code 42 and product identification code 44. The information processing system 100 uses the images containing the shelf location identification code 42 and product identification code 44 sent from the work terminal 20 to generate route information that provides real-time guidance on the route from the worker's current location to the location of the product being worked on.

<Work terminal 20>
The work terminal 20 is an information terminal that transmits and receives data. The work terminal 20 is, for example, a smartphone or tablet, and outputs information about the product to be worked on to the worker. Fig. 6 is a block diagram showing the configuration of the work terminal 20 in Fig. 4.

The work terminal 20 includes an imaging unit 21, a storage unit 22, a memory 23, a communication unit 24, an input/output unit 25, and a processing unit 26. The imaging unit 21 is an imaging device that takes images in accordance with the control of the processing unit 26. The imaging unit 21 continuously takes images including a shelf position identification code 42 and a product identification code 44. These images are input to the server 10.

The memory unit 22 is an example of a storage device that includes non-volatile memory such as flash memory or an SSD (Solid State Drive). The memory unit 22 stores a program that implements processes including sending captured images and notifying workers of received information. The processes of notifying workers may include (1) displaying the work content to be performed by the worker, (2) displaying detailed information about the product that the worker is working on (work target information), and (3) displaying route information that guides the route from the location of the work terminal to the storage location of the target product within the warehouse W.

The memory 23 is a volatile storage device such as RAM (Random Access Memory), and is a storage area for temporarily storing information while the processing unit 26 is operating. The communication unit 24 is a communication interface with the network N. The communication unit 24 may also establish a short-range wireless communication connection with the server 10 and the management terminal 30 to communicate. Various standards for short-range wireless communication can be applied here, such as Bluetooth (registered trademark), BLE (Bluetooth Low Energy), and UWB (Ultra Wide Band).

The input/output unit 25 includes a display device and an input device, such as a touch panel. The processing unit 26 is a processor that controls the hardware of the work terminal 20. The processing unit 26 loads a program from the storage unit 22 into the memory 23 and executes it. In this way, the processing unit 26 realizes the functions of the transmission/reception processing unit 261 and the display processing unit 262.

The transmission/reception processing unit 261 transmits an image including the shelf position identification code 42 and product identification code 44 to the server 10. The transmission/reception processing unit 261 also receives various information from the server 10 that is necessary for the worker to perform the work. The display processing unit 262 displays the various received information on the display device so that it can be viewed by the worker.

The display device can display, for example, route information received from the server 10, guiding the route from the location of the work terminal to the location of the product to be worked on. The worker can move to the location of the product to be worked on while visually checking the displayed route. The display device can also display the work content to be performed by the worker and the work target information, as described above. The work content to be performed by the worker may include, for example, storing the product on a storage shelf or picking the product from a storage shelf. The work target information may include the product name, size, weight, quantity, etc. of the product to be worked on.

Note that the method of presenting route information to workers is not limited to visual methods such as displaying it on a display device. For example, route information may be presented to workers by voice. Therefore, the input/output unit 45 may include a speaker, etc.

In addition, workers can use an input device to input information about the work they have performed as needed. For example, when a worker has finished picking up a specified item, they can input completion information indicating that the work has been completed. For example, a checkbox for inputting whether the work has been completed may be displayed on the touch panel screen of the work terminal 20. The completion information is sent to the server 10. This allows the manager to confirm that the worker has completed the work, and makes it possible to assign new work to the worker who is now free.

<Management terminal 30>
The management terminal 30 is an information terminal used by the manager of the warehouse W, and transmits and receives data to and from the server 10 via the network N by wireless communication. The management terminal 30 is, for example, a PC, a smartphone, a tablet, etc. FIG. 7 is a block diagram showing the configuration of the management terminal 30 of FIG. 4 .

The management terminal 30 includes a storage unit 31, memory 32, a communication unit 33, an input/output unit 34, and a processing unit 35. The management terminal 30 mainly issues instructions to and manages the server 10. The storage unit 31 is an example of a storage device that includes non-volatile memory such as flash memory or an SSD (Solid State Drive). The storage unit 31 stores programs that have executed processes, including processes that determine the work content to be performed by workers. The memory 32 is a storage area such as RAM for temporarily storing information when the processing unit 35 is operating. The communication unit 33 is a communication interface with the network N.

The input/output unit 34 may include, for example, a display device and input devices such as a keyboard and a mouse. The input/output unit 34 may also be a touch panel that integrates a display device and an input device. The input/output unit 34 may also include a known code reader. The manager can use the input/output unit 34 to input information identifying the products to be received. The information about the products to be received may be used when instructing a worker to store the products in the storage shelves. The manager can also input information identifying the products to be shipped. The information about the products to be shipped may be used when instructing a worker to remove the products from the storage shelves. Such product identification information is used to update the product information database, which will be described later.

The processing unit 35 is a processor that controls the hardware of the management terminal 30. The processing unit 35 loads a program from the storage unit 31 into the memory 32 and executes it. As a result, the processing unit 35 realizes the functions of the work instruction generation unit 351, product management unit 352, layout management unit 353, and spatial map management unit 354.

The work instruction generation unit 351 generates work instructions including worker information that identifies the worker who will perform the work and the work content for that worker, and outputs these to the server 10. When creating work instructions for the worker to store received products on a storage shelf, the work instruction generation unit 351 can, for example, identify the storage location for the products based on the product identification information of the received products (information related to the product size, instructions from the shipper, etc.).

If a storage shelf has multiple storage spaces of different sizes, the work instruction generation unit 351 may identify a storage space with a size that is small compared to the size of the product as the storage location for the product. The work instruction generation unit 351 may also identify a storage location for the product to be stored, taking into account the sizes of products already stored on the shelf. The work instruction generation unit 351 may also identify a location that further satisfies information regarding the product's weight as the storage location. For example, if a shelf is made up of multiple levels, the work instruction generation unit 351 may identify the bottom level as the storage location for the product when the weight of the product is greater than a predetermined value.

The product management unit 352 outputs the above-mentioned product identification information to the server 10 and updates the product information database. The layout management unit 353 outputs layout management information entered by the manager, such as the arrangement of storage shelves 40 within the warehouse W and the size allocation of storage space on each storage shelf 40, to the server 10 and updates the layout database, which will be described later. The spatial map management unit 354 updates the spatial position information DB 114, which will be described later.

<Server 10>
The server 10 is an information processing device that realizes functions such as a function of generating route information that provides guidance on the route from the current location of the worker to the location where the product should be stored (route information generation function), a function of pinpointing the location of the product (location identification function), and a function of determining the validity of the relative locations of the storage shelf, product, worker terminal, etc. Note that the server 10 may be composed of multiple servers, and each functional block may be realized by multiple computers.

Figure 8 is a block diagram showing the configuration of the server 10 in Figure 4. As shown in Figure 8, the server 10 includes a storage unit 11, a memory 12, a communication unit 13, and a processing unit 200. The storage unit 11 is an example of a storage device such as a hard disk or flash memory. The storage unit 11 stores a program 111, a layout DB (database) 112, a product information DB 113, and a spatial position information DB 114. The program 111 is a computer program that implements part of the above-mentioned route information generation function, position identification function, and validity determination function.

Layout DB 112 manages layout information that indicates the arrangement of structures such as aisles and storage shelves within the warehouse. Figure 9 is a diagram showing an example of layout information. Layout information may include the attachment positions of barcodes that indicate shelf position identification information within the warehouse. Layout information is, for example, a plan view showing the aisles of the warehouse. As shown in Figure 9, the warehouse may be divided into multiple areas. "Scale" is used to match the scale of the layout information with that of the image input from the work terminal 20. For example, "scale" can indicate the unit length in the layout information.

The layout of aisles in each area can be represented by data indicating x and y coordinate values, with (0,0) on the plane coordinate system as the origin. This origin position information is used when overlaying the estimated position information of products and work terminals stored in the spatial position information DB 114. The layout information is also used in processing by the association determination unit 240, described below, such that the position of the work terminal (worker) must be in an aisle, and the position of products must be on a storage shelf other than an aisle. The layout information can also be used when the route information generation unit 232, described below, generates route information and determines that workers cannot pass through areas other than aisles.

Product information DB113 manages various information related to products. Product information DB113 can store the product identification information described above. Figure 10 is a diagram showing an example of product identification information. As shown in Figure 10, the product identification information includes general product information such as the bar code attached to each product, the shipper, and the date of arrival/departure.

The spatial position information DB 114 manages spatial position information that indicates the relative three-dimensional positions of products and work terminals 20, estimated by the spatial position information generation unit 220 of the processing unit 200, which will be described later. Figure 11 is a diagram showing an example of spatial position information. As shown in Figure 11, the area and X, Y, and Z coordinate values of the estimated position of each work terminal 20 and each product are shown. The spatial position information also includes the barcode attached to each product. In other words, the spatial position information DB 114 stores linking information with the layout DB 112 and product information DB 113, as well as coordinate information in three-dimensional space for the products and work terminals.

The layout DB 112, product information DB 113, and spatial position information DB 114 are managed via the aggregation unit 230, described below, primarily via the position information integration unit 231. The reason they are managed via the aggregation unit 230 is that, for example, the product information DB 113 is updated in response to input from the product management unit 352 of the management terminal 30, and is also rewritten when a worker completes picking or storage work, requiring coordinated information updates. Note that the configurations of the layout DB 112, product information DB 113, and spatial position information DB 114 are not limited to the examples described above.

The memory 12 is a volatile storage device such as RAM, and is a storage area for temporarily storing information while the processing unit 200 is operating. The communication unit 13 is a communication interface with the network N. The processing unit 200 is a processor that controls each component of the server 10. The processing unit 200 loads the program 111 from the storage unit 11 into the memory 12 and executes the program 111. In this way, the processing unit 200 realizes the functions of the acquisition unit 210, spatial position information generation unit 220, aggregation unit 230, and association determination unit 240.

The acquisition unit 210 includes an image analysis unit 211 and a code reading unit 212. Images captured by the imaging unit 21 of the work terminal 20 are input to the image analysis unit 211 in a continuous time series. The image analysis unit 211 analyzes the input images and automatically detects the position of the barcode. The code reading unit 212 reads the number (a string of numbers) contained in the detected barcode. This allows the acquisition unit 210 to acquire multiple pieces of product identification information attached to the product and shelf position identification information attached to each storage position on the storage shelf 40.

The spatial position information generation unit 220 includes a work terminal 3D position estimation unit 221, a product identification unit 222, and a product 3D position estimation unit 223. The work terminal 3D position estimation unit 221 estimates the three-dimensional position of the work terminal 20 that captured the image from the position of the barcode in the image acquired by the acquisition unit 210. The product identification unit 222 checks whether the barcodes acquired by the acquisition unit 210 include a barcode that indicates product identification information for the product. The product identification unit 222 compares the product identification information acquired from the image with the product identification information in the product information DB 113 to identify the product.

The product 3D position estimation unit 223 estimates the position of the product identified by the product identification unit 222 in three dimensions from the position of the work terminal 20 and the position of the barcode in the image acquired by the acquisition unit 210. In other words, the spatial position information generation unit 220 estimates the relative three-dimensional positions of the product and the work terminal based on the product identification information and shelf position identification information acquired from the image input from the work terminal 20, and generates three-dimensional spatial position information.

The three-dimensional position estimation method used by the work terminal 3D position estimation unit 221 and the product 3D position estimation unit 223 is not particularly limited. As mentioned above, for example, a method can be used in which information regarding the size of the identification code and the attachment position of the identification code is registered in advance, and coordinates are estimated based on the position and distortion of the arrangement of pixels representing the identification code or storage shelf in the image. Alternatively, a method can be used in which photogrammetry is performed from the frame differences between consecutive acquired images. SLAM, which uses a 3D sensor to estimate self-position and create a spatial map, can also be used.

The aggregation unit 230 includes a location information integration unit 231, a route information generation unit 232, and a work information generation unit 233. The location information integration unit 231 integrates information from the layout DB 112, product information DB 113, and spatial position information DB 114, and overlays the estimated positions of products, work terminals, etc. Hereinafter, the information from each database integrated and overlaid on each other will be referred to as a "spatial map."

In other words, the location information integration unit 231 functions as a spatial map creation unit that integrates information from each database to create a spatial map. A "spatial map" represents products, work terminals 20, storage shelves 40, shelves 41, aisles, etc., using coordinate values in an XYZ coordinate system. Therefore, the location information integration unit 231 can identify the storage location of the work object as coordinate information on the spatial map. In this way, the location information integration unit 231 can integrate and overlay multiple pieces of information from each database, which were not managed simultaneously in the comparative example, to derive pinpoint three-dimensional location information for products and work terminals (workers).

The route information generation unit 232 generates route information using a spatial map to guide the route from the location of the work terminal to the location of the product identified based on the product identification information. Specifically, when the route information generation unit 232 receives a work instruction from the management terminal 30, including the above-mentioned worker information and work content, it uses a spatial map to calculate the route required for the worker to perform the work. Specifically, the route information generation unit 232 uses the work target information, including the product to be worked on and its quantity, from the product information DB 113, the estimated positions of the work terminal 20 and the product to be worked on from the spatial position information DB 114, and the aisle information within the warehouse from the layout DB 112, to determine the route the worker will take when performing the work, taking into account the accurate positions of both the work terminal 20 and the product to be worked on. The route information generation unit 232 can generate route information using coordinate information in the XYZ coordinate system described above.

The location information integration unit 231 can update the spatial location information DB 114 based on the barcode location information input from the spatial location information generation unit 220 and the estimated location information of the product and work terminal 20. The accuracy of updating the spatial location information DB 114 can be further improved based on the validity determination results of the two types of location information input from the association determination unit 240. The association determination unit 240 will be described in detail later.

The work information generation unit 233 transmits to the work terminal 20 work information necessary for the worker to perform the work, such as work instructions provided by the management terminal 30, route information calculated by the location information integration unit 231, and a spatial map obtained by integrating each database. For example, the work instructions may include work object information indicating detailed information about the identified work target item and the quantity of work target item to be picked up. Based on the work information, the work terminal 20 displays the work content to be performed by the worker, the route to be taken when performing the work, and the work object information.

The route information may be information that displays on a spatial map the route from the location of the work terminal 20 to the storage location of the item to be worked on, identified based on the item identification information. The route information may be displayed superimposed on an image captured by the work terminal 20 using at least one of the following display methods: text display, augmented reality display, and mixed reality display. The route information may also include stopping position information that indicates the worker's stopping position, which is displayed on the spatial map when the work terminal 20 is located a predetermined distance from the storage location of the item to be worked on.

The association determination unit 240 includes a relative position validity determination unit 241 and a time series validity determination unit 242. The relative position validity determination unit 241 compares the current spatial map with a spatial map that is temporally earlier than the current spatial map to determine the validity of the relative positional relationships between the product, the work terminal 20, the product identification information, and the shelf position identification information. Specifically, the relative position validity determination unit 241 determines whether there are any abnormalities in the associations of relative positions, such as the positions of the aisle and the worker, the positions of shelves, the positions of shelves and products, and the positions of products, based on the position of the barcode in the image, the estimated positions of the work terminal 20 and the products estimated by the spatial position information generation unit 220, and the database information integrated and superimposed by the aggregation unit 230.

The time series validity determination unit 242 compares the current spatial map with a spatial map taken chronologically before the current spatial map to determine the validity of the positional relationship in the time series. Specifically, the time series validity determination unit 242 determines whether the time series information of the worker's movements, i.e., the barcode information read from the temporally consecutive images taken by the worker while moving, contains any abnormal content. The processing of the association determination unit 240 makes it possible to pinpoint and ensure highly accurate data on the two pieces of positional information for the worker and the product.

Here, the flow of the information processing method according to the present disclosure will be described with reference to Figures 12 to 15. Here, the flow of the picking work will be described. Figure 12 is a diagram showing the flow of preparation for product management operations in the information processing system 100.

The manager operates the input/output unit 34 to input layout information indicating the arrangement of structures within the warehouse and the locations within the warehouse where barcodes indicating shelf position identification information are attached (S001). The manager also sets the origin of the coordinate system of the spatial map (S0002). Note that the origin of the coordinate system of the spatial map does not need to be set by the manager; it may be automatically set to a corner of the map, for example.

Then, the manager operates the input/output unit 34 to input product identification information (barcode, product name, quantity, etc.), which is data linked to information necessary to identify the products handled by the worker (S0003). This forms the basis of the spatial map. As the spatial map continues to be updated in subsequent flows, the location of the worker terminal (worker) and the location of the products are managed. Note that step S0003 is repeatedly performed during operation, and the product identification information is always kept up to date. Step S0003 may also be automated, for example, based on product inventory information.

Figure 13 shows the flow in which a manager issues picking instructions to a worker. First, the manager operates the input/output unit 34 to specify one or more types of products to be worked on and the required quantity (S101). The manager may select products to be worked on from a list, or may select them in a video game-like manner by operating in a digital twin space that is a three-dimensional spatial map.

Next, the manager operates the input/output unit 34 to decide which worker to issue the work instruction to (S102). The manager may specify a specific worker, or may automatically assign a worker who is in a position where it is easy to pick the product specified in S101, or a worker who has free time. The work instruction, including the instruction content and worker information, is processed by the aggregation unit 230. The route information generation unit 232 generates a route using a spatial map to guide the worker from the location of the work terminal to the storage location of the product to be worked on, identified based on the product identification information (S103). Thereafter, the work information generation unit 233 transmits work information necessary for the worker to perform the work, including the work instruction and route information, to the work terminal 20 (S104).

The worker holding the work terminal that received the work information can then carry out the picking work in accordance with the work content. The processing of the information processing system 100 at this time can be divided into a flow in which the worker operates the work terminal 20 to view the work information (Figure 14), and the system's internal processing flow that takes place during this time (Figure 15).

Figure 14 shows a worker workflow in which a worker operates the work terminal 20 to view work information. First, the received work information is displayed on the display device of the work terminal 20 carried by the worker (S201). The worker can then operate the input/output unit 25 to accept the work instruction (S202). If the work instruction includes the task of picking up multiple items, a flow that is repeated until one item is acquired (S220) and a flow that is repeated until all items are acquired (S210) are executed.

In the flow (S220) that is repeated until one product is acquired, route information is first displayed on the display device of the work terminal 20 (S221). The worker can then begin moving along the displayed route (S222). As the worker moves, the internal processing described below (Figure 15) is repeatedly executed, and the estimated position of the work terminal 20 in the spatial position information DB 114 is updated, thereby repeatedly updating the route information.

When the worker moves to the location of the product to be picked, the internal processing shown in FIG. 15 updates the estimated location of the work terminal 20 to a location within a predetermined processing distance from the estimated location of the product to be picked, and the flow exits S220. The input/output unit 25 then displays work target information indicating the product to be picked and the required quantity (S211). The worker can check the displayed information and perform the picking. At this time, if the worker needs detailed information such as the date of entry, the worker may operate the input/output unit 25 to request the server 10 to access the product information DB 113 and read the detailed information (S212). If detailed information is requested (S212, YES), the work terminal 20 can obtain and display the detailed information from the server 10 (S213). At this time, the server 10 may also read a work procedure manual or the like and send it to the work terminal 20.

When the worker has completed all tasks displayed on the input/output unit 25, he or she can input completion information through the input/output unit 25 (S214). This allows the worker to move on to the next item to be worked on. When all items to be worked on have been picked, this flow ends. Note that the information processing system 100 may continue route guidance until the picked up items are taken to the specified location.

Next, referring to Figure 15, we will explain the internal system processing that takes place while the worker is working. Figure 15 shows the flow of internal system processing that takes place while the worker is working. While the worker is working, the imaging unit 21 of the acquisition unit 2 continuously captures images and continues to send them to the server 10. The acquisition unit 210 of the server 10 continuously acquires images over time and analyzes the barcodes in the images (S301).

If identification information, such as product identification information or shelf location identification information, cannot be obtained from the barcode (S302, NO), image acquisition and analysis of the barcode in the image (S301) continue. If identification information can be obtained from the barcode (S302, YES), the work terminal 3D position estimation unit 221 estimates the position of the work terminal 20 using all identification information in the image (S303).

Then, the association determination unit 240 determines the validity of the estimated position of the work terminal 20 (S304). The association determination unit 240 denies the validity if multiple barcodes with impossible combinations are read based on the estimated position of the work terminal 20 and the relative positional relationship between the barcodes attached to the products in the image and the barcodes attached to the shelves 41, or if a comparison of the time series information indicates that the worker moved at a speed that would normally be impossible or to an impossible height. If the estimated position of the work terminal 20 is not valid (S304, NO), the data is discarded (S305).

If the estimated position of the work terminal 20 is valid (S304, YES), the position information integration unit 231 updates the position of the work terminal 20 in the spatial map in the spatial position information DB 114 (S306). The position information integration unit 231 can update the spatial map, for example, each time an image is acquired from the work terminal 20. In response, the route information generation unit 232 updates the route information (S307) and outputs it to the work terminal 20. The work terminal 20 can display the updated route information (S221 in FIG. 14).

If the position of the work terminal 20 is valid and the product barcode has been acquired (S308, YES), the product 3D position estimation unit 223 estimates the product position (S309). If the product barcode has not been acquired (S308, NO), the process returns to S301. As in the case of the work terminal 20 (S304), the association determination unit 240 determines the validity of the estimated product position (S310). If the estimated product position is invalid (S310, NO), that is, if the package is not where it should be, the association determination unit 240 can issue invalid position information to the management terminal 30 (S311).

If the estimated product position is valid (S310, YES), it is confirmed whether the product for which product identification information has been obtained is the product to be worked on as specified in FIG. 13 (S312). If this product is a product to be worked on (S312, YES), the work information generation unit 233 notifies the work terminal 20 of the work target information (S313, S211). If this product is not a product to be worked on (S312, NO), the process returns to S301.

So far, we have explained an example of picking work. In the above-mentioned picking work, the route information indicates the route from the location of the work terminal to the target product, based on the barcode indicating the product identification information. In storage work, the route information may also indicate the route from the location of the work terminal to the package storage location, based on the barcode indicating the shelf location identification information that indicates the shelf location. In this case, after the target product is stored, the validity of whether the product has been stored in the specified location is determined, and if there are no problems, the work is completed.

As explained above, according to the present disclosure, by simply continuing to capture images with a work terminal, it is possible to use barcodes attached to items or within the warehouse to confirm the location of the worker's terminal or items in three-dimensional coordinates, thereby guiding the worker from their current location to the work location.

The present disclosure has the following seven advantages, for example:
(1) Not only can it be determined that a product is located in the vicinity of a specific storage shelf identified by shelf position identification information, but it can also be determined not only by the human eye but also by a system that the product is located in a pinpoint position with high precision in three dimensions.
(2) By being able to pinpoint product location and product identification information in three-dimensional space without the need for repeated, cumbersome barcode reading, warehouse operations can be carried out efficiently without workers having to search for products around storage shelves.

(3) By reducing the work of reading barcodes, enabling storage locations to be specified graphically, and automatically verifying the validity of the location while workers are working, human errors such as failing to read barcodes can be reduced, and product loss can be reduced.

(4) By providing work instructions using images, such as route information, rather than text, even new workers who are unfamiliar with the layout of the warehouse can complete their work in a time similar to that of veteran workers.
(5) Since there is little written information in the work instructions, even foreign workers can carry out their work without any miscommunication.

(6) Even if a product is placed at the back of a storage shelf, which is not managed by shelf position identification information and is in a blind spot that is difficult for the human eye to find, the location of the product can be grasped graphically, making it easy to find.
(7) It is possible to determine whether products that do not require topping or products that have storage conditions that prohibit stacking on top are placed correctly, and whether products that have expired are being stored, while the worker is performing other tasks.

Other Embodiments The work terminal 20 may be, for example, a smartphone with a camera or smart glasses with a camera. The work terminal 20 may also be a combination of a wearable camera and a small terminal such as a smartphone. The work terminal 20 may also be a picking cart used to pick products from storage locations, to which a web camera and a tablet are attached.

In addition to the images captured by the work terminals described herein, barcode monitoring using fixed cameras installed in the warehouse may also be utilized in combination. The system described herein can also be integrated with existing indoor positioning technology to improve the positioning accuracy of work terminals and products.

In the above-described embodiment, location information is managed using three-dimensional coordinates, but it is also possible to combine this with technology that identifies product storage locations using three-dimensional coordinate information and existing shelf location identification information. The system disclosed herein continuously performs inventory, so it can also be used to detect product loss (theft). The system may also be installed on a patrol robot and used as an unmanned surveillance device.

The system disclosed herein is applicable to material management and picking work by workers in logistics warehouses and raw material warehouses where workers patrol the warehouse. The system disclosed herein can also be used to track the location of patrol robots patrolling the warehouse and to grasp product packages. The system disclosed herein can contribute to more efficient back-of-house inventory management and stocking in retail stores such as supermarkets. The system disclosed herein is suitable, for example, for work sites where the types of inventory in a store change frequently and where workers, such as part-timers and casual workers, are frequently replaced. The system disclosed herein also makes it possible for part-timers to retrieve prescription medications at dispensing pharmacies.

In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include RAM, read-only memory (ROM), flash memory, SSD or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

The present disclosure has been described above with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various modifications that would be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

The drawings are merely examples for illustrating one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing may be combined with features or steps shown in one or more other figures, for example, to create an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to illustrate an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.
(Appendix A1)
an acquisition unit that acquires images taken by the work terminal in succession over time and acquires a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility;
a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information;
a spatial map creation unit that creates a spatial map by integrating the spatial position information;
a route information generating unit that generates route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
Equipped with
Information processing device.
(Appendix A2)
a determination unit that compares a current spatial map with a spatial map that is temporally older than the current spatial map to determine at least one of the validity of a relative positional relationship between the item, the work terminal, the item identification code, and the storage location identification code, and the validity of a positional relationship in time series;
10. The information processing device according to claim 1,
(Appendix A3)
the route information is information that displays on the spatial map a route from the position of the work terminal to a storage location of the item identified based on the item identification code;
An information processing device according to appendix A1 or A2.
(Appendix A4)
the route information includes stop position information indicating a stop position of the worker to be displayed on the spatial map when the work terminal is located a predetermined distance from the storage position of the item;
An information processing device according to Appendix A3.
(Appendix A5)
the route information is displayed superimposed on an image captured by the work terminal in accordance with at least one of a text display, an augmented reality display, and a mixed reality display;
An information processing device according to appendix A3 or A4.
(Appendix A6)
the spatial map creation unit identifies the storage position of the item as coordinate information on the spatial map;
The route information generation unit generates the route information using the coordinate information.
An information processing device according to any one of appendices A1 to A5.
(Appendix A7)
the spatial map creation unit updates the spatial map every time the image is acquired from the work terminal.
An information processing device according to any one of appendices A1 to A6.
(Appendix B1)
a work terminal that takes continuous photographs in time to generate images including an item identification code attached to the item and a storage location identification code attached to each storage location in the warehouse facility;
an information processing device to which the image is input;
Including,
The information processing device includes:
an acquisition unit that acquires the images successively in time and acquires a plurality of the item identification codes and the storage location identification codes;
a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information;
a spatial map creation unit that creates a spatial map by integrating the spatial position information;
a route information generating unit that generates route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
Equipped with
Information processing system.
(Appendix C1)
The computer
A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility;
a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information;
a process of integrating the spatial location information to create a spatial map;
a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
To execute
Information processing methods.
(Appendix D1)
A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility;
a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information;
a process of integrating the spatial location information to create a spatial map;
a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
to the computer,
program.

Some or all of the elements (e.g., configurations and functions) described in Appendix A1 through Appendix A7 that are dependent on Appendix A1 (device) may also be dependent on Appendix B1 (information processing system), Appendix C1 (method), and Appendix D1 (program) in the same dependency relationship as Appendix A1 through Appendix A7. Some or all of the elements described in any appendix may be applied to various hardware, software, recording means for recording software, systems, and methods.

REFERENCE SIGNS LIST 1 Information processing device 2 Acquisition unit 3 Spatial position information generation unit 4 Spatial map creation unit 5 Route information generation unit 10 Server 11 Storage unit 12 Memory 13 Communication unit 111 Program 112 Layout DB
113 Product information DB
114 Spatial location information DB
20 Work terminal 21 Imaging unit 22 Storage unit 23 Memory 24 Communication unit 25 Input/output unit 26 Processing unit 261 Transmission/reception processing unit 262 Display processing unit 30 Management terminal 30 Management terminal 31 Storage unit 32 Memory 33 Communication unit 34 Input/output unit 35 Processing unit 351 Work instruction generation unit 352 Product management unit 353 Layout management unit 354 Spatial map management unit 40 Storage shelf 41 Shelf 42 Shelf position identification code 43 Product 44 Product identification code 100 Information processing system 101 Information processing system 102 Acquisition unit 103 Spatial position information generation unit 104 Spatial map creation unit 105 Route information generation unit 200 Processing unit 210 Acquisition unit 211 Image analysis unit 212 Code reading unit 220 Spatial position information generation unit 221 Work terminal 3D position estimation unit 222 Product identification unit 223 Product 3D position estimation unit 230 Aggregation unit 231 Position information integration unit 232 Route information generation unit 233 Work information generation unit 240 Association determination unit 241 Relative position validity determination unit 242 Time series validity determination unit W Warehouse

Claims (10)

an acquisition unit that acquires images taken by the work terminal in succession over time and acquires a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility;
a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information;
a spatial map creation unit that creates a spatial map by integrating the spatial position information;
a route information generating unit that generates route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
Equipped with
Information processing device. a determination unit that compares a current spatial map with a spatial map that is temporally older than the current spatial map to determine at least one of the validity of a relative positional relationship between the item, the work terminal, the item identification code, and the storage location identification code, and the validity of a positional relationship in time series;
The information processing device according to claim 1 . the route information is information that displays on the spatial map a route from the position of the work terminal to a storage location of the item identified based on the item identification code;
The information processing device according to claim 1 . the route information includes stop position information indicating a stop position of the worker to be displayed on the spatial map when the work terminal is located a predetermined distance from the storage position of the item;
The information processing device according to claim 3 . the route information is displayed superimposed on an image captured by the work terminal in accordance with at least one of a text display, an augmented reality display, and a mixed reality display;
The information processing device according to claim 3 . the spatial map creation unit identifies the storage position of the item as coordinate information on the spatial map;
The route information generation unit generates the route information using the coordinate information.
The information processing device according to claim 1 . the spatial map creation unit updates the spatial map every time the image is acquired from the work terminal.
The information processing device according to claim 1 . a work terminal that takes continuous photographs in time to generate images including an item identification code attached to the item and a storage location identification code attached to each storage location in the warehouse facility;
an information processing device to which the image is input;
Including,
The information processing device includes:
an acquisition unit that acquires the images successively in time and acquires a plurality of the item identification codes and the storage location identification codes;
a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information;
a spatial map creation unit that creates a spatial map by integrating the spatial position information;
a route information generating unit that generates route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
Equipped with
Information processing system. The computer
A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility;
a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information;
A process of integrating the spatial location information to create a spatial map;
a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
To execute
Information processing methods. A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility;
a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information;
A process of integrating the spatial location information to create a spatial map;
a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code;
to the computer,
program.