WO2025173507A1 - Information processing method, information processing device, and program - Google Patents

Abstract

Provided is an information processing method capable of appropriately generating and outputting manufacturing rule information. An information processing device performs the following (1) to (6). (1) Acquiring image information obtained by imaging a situation in which a plurality of manufacturing steps are sequentially executed on a product. (2) Creating, on the basis of the image information, graphic information including information indicating the plurality of manufacturing steps to be executed on a product and information indicating an execution order of the plurality of manufacturing steps. (3) Displaying the graphic information on a display device. (4) Acquiring, from an input device, correction information for the graphic information inputted by the user. (5) Generating, on the basis of the graphic information and the correction information, manufacturing rule information including the plurality of manufacturing steps related to the product and a rule of an execution order of the plurality of manufacturing steps. (6) Outputting the manufacturing rule information.

Description

Information processing method, information processing device, and program

This disclosure relates to an information processing method, an information processing device, and a program.

Patent Document 1 discloses a work data management system related to background technology. This work data management system classifies work performed by workers at a work site into multiple chronologically-sequenced detailed processes based on work data including video data of the work site, and displays the video data and the chronologically-sequenced detailed processes.

Japanese Patent No. 6824838

The work data management system related to the background art described above does not consider generating and outputting manufacturing rule information to be used in factory simulations such as CPS (Cyber-Physical System) that optimize manufacturing operations.

The present disclosure aims to provide an information processing method, information processing device, and program that can appropriately generate and output manufacturing rule information that includes multiple manufacturing processes related to a product and rules for the execution order of those multiple manufacturing processes, thereby enabling optimization of manufacturing operations through factory simulations using the manufacturing rule information.

An information processing method according to one aspect of the present disclosure is an information processing method for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, in which an information processing device performs the following steps (1) to (6): (1) Obtain image information obtained by photographing the situation in which the multiple manufacturing processes are performed in sequence on the product. (2) Based on the image information, create graphic information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes. (3) Display the graphic information on a display device. (4) Obtain correction information for the graphic information entered by a user from an input device. (5) Generate manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes based on the graphic information and the correction information. (6) Output the manufacturing rule information.

An information processing device according to another aspect of the present disclosure is an information processing device for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, and performs the following: Specifically, the information processing device acquires image information obtained by photographing the situation in which the multiple manufacturing processes are performed in sequence on the product. Based on the image information, the information processing device creates graphic information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution sequence of the multiple manufacturing processes. The information processing device displays the graphic information on a display device. Acquires, from an input device, correction information for the graphic information input by a user. Based on the graphic information and the correction information, the information processing device generates manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution sequence of the multiple manufacturing processes. The information processing device outputs the manufacturing rule information.

According to the present disclosure, it is possible to appropriately generate and output manufacturing rule information that includes multiple manufacturing processes related to a product and rules for the execution order of those multiple manufacturing processes, thereby making it possible to optimize manufacturing operations through factory simulations using the manufacturing rule information.

1 is a diagram illustrating a simplified overall configuration of an information processing system according to an embodiment of the present disclosure. 1 is a flowchart illustrating a flow of processing executed by an information processing device according to an embodiment of the present disclosure. FIG. 10 is a diagram showing a simplified example of analysis of image information. FIG. 10 is a simplified diagram showing an example of graphic information displayed on a display device. FIG. 10 is a simplified diagram illustrating an example of manufacturing rule information generated when no correction information is input. FIG. 10 is a simplified diagram showing an example of graphic information displayed on the display device after correction of deletion. FIG. 10 is a simplified diagram showing an example of manufacturing rule information generated when correction information for deletion is input. FIG. 10 is a simplified diagram showing an example of graphic information displayed on the display device after correction by adding a branching condition. FIG. 10 is a simplified diagram illustrating an example of manufacturing rule information generated when correction information for adding a branch condition is input. FIG. 10 is a simplified diagram showing an example of graphic information displayed on a display device after the division has been corrected. FIG. 10 is a simplified diagram showing an example of manufacturing rule information generated when division correction information is input. FIG. 10 is a simplified diagram showing an example of graphic information displayed on a display device after integration correction. FIG. 10 is a simplified diagram showing an example of manufacturing rule information generated when integration correction information is input. FIG. 10 is a simplified diagram showing an example of graphic information displayed on a display device. FIG. 10 is a simplified diagram illustrating an example of graphical information displayed on a display device after additional modifications. FIG. 10 is a simplified diagram showing an example of graphic information displayed on a display device after integration correction. FIG. 10 is a simplified diagram illustrating an example of graphic information to which proposal information has been added. FIG. 10 is a simplified diagram showing another example of graphic information displayed on the display device. FIG. 10 is a simplified diagram showing another example of graphic information displayed on the display device. FIG. 10 is a simplified diagram showing another example of graphic information displayed on the display device. FIG. 10 is a simplified diagram showing another example of graphic information displayed on the display device. 10 is a flowchart illustrating a flow of processing according to a modified example executed by an information processing device according to an embodiment of the present disclosure.

(Findings that form the basis of this disclosure)
In order to optimize manufacturing operations in manufacturing factories, it is desirable to build a factory simulator using a CPS (Cyber-Physical System), etc. Building a highly accurate factory simulator makes it possible to discover manufacturing operation methods that improve production efficiency, verify resilience to fluctuations in demand or supply, and verify the load on manufacturing equipment or manufacturing processes when additionally manufacturing new products or changing manufacturing rules for existing products, without affecting the operation of the actual factory.

However, not all manufacturing plants have accumulated or managed sufficient manufacturing rule information to build a factory simulator. It is particularly difficult to obtain detailed information about processes or tasks that are not mechanized, making it difficult to build a highly accurate factory simulator.

In order to solve this problem, the inventors discovered that by using image information obtained by photographing the process in which multiple manufacturing processes are performed in sequence on a product, and by also taking into account correction information entered by a user such as an operator, it is possible to appropriately generate manufacturing rule information that includes multiple manufacturing processes related to a product and the rules for the order in which those multiple manufacturing processes are performed, and this led to the present disclosure.

Next, each aspect of this disclosure will be described.

The information processing method according to a first aspect of the present disclosure is an information processing method for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, in which an information processing device performs the following steps (1) to (6): (1) Obtain image information obtained by photographing the situation in which the multiple manufacturing processes are performed in sequence on the product. (2) Based on the image information, create graphic information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution sequence of the multiple manufacturing processes. (3) Display the graphic information on a display device. (4) Obtain correction information for the graphic information entered by a user from an input device. (5) Generate manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution sequence of the multiple manufacturing processes based on the graphic information and the correction information. (6) Output the manufacturing rule information.

According to the first aspect, it is possible to appropriately generate and output manufacturing rule information, including multiple manufacturing processes related to a product and rules for the execution order of those multiple manufacturing processes, based on graphic information and correction information entered by the user. As a result, it is possible to optimize manufacturing operations through factory simulation using the manufacturing rule information.

In the information processing method according to the second aspect of the present disclosure, in the first aspect, the correction information may include information for deleting or changing a manufacturing process included in the graphic information.

According to the second aspect, manufacturing rule information can be appropriately generated based on modification information that includes information for deleting or changing manufacturing processes.

In the information processing method according to the third aspect of the present disclosure, in the first or second aspect, the modification information may include information that adds a branching condition from a manufacturing process included in the graphic information to multiple subsequent manufacturing processes.

According to the third aspect, manufacturing rule information can be appropriately generated based on modification information that includes information for adding branching conditions.

In the information processing method according to the fourth aspect of the present disclosure, in any one of the first to third aspects, the manufactured products may include multiple manufactured products with different attributes, and the correction information may include information for dividing or integrating the graphic information according to the attributes.

According to the fourth aspect, manufacturing rule information can be appropriately generated based on modification information that includes information for dividing or integrating graphic information.

In the information processing method according to the fifth aspect of the present disclosure, in any one of the first to fourth aspects, the manufactured products include multiple manufactured products with different attributes, and in creating the graphic information, the multiple manufactured products are classified into multiple groups based on their attributes, and the graphic information is created for each group.

According to the fifth aspect, graphic information is created for each group, eliminating the need for the user to classify groups.

In the information processing method according to a sixth aspect of the present disclosure, in any one of the first to fifth aspects, the image information may include manufacturing equipment that performed each manufacturing process, and the graphic information may further include information indicating at least one piece of manufacturing equipment that performed each manufacturing process.

According to the sixth aspect, the graphic information includes information indicating the manufacturing equipment, making it possible to generate and output more detailed manufacturing rule information.

In the information processing method according to the seventh aspect of the present disclosure, in the sixth aspect, the correction information may include information for adding manufacturing equipment capable of executing the manufacturing process included in the graphic information.

According to the seventh aspect, manufacturing rule information can be appropriately generated based on modification information that includes information on adding manufacturing equipment.

In the information processing method according to the eighth aspect of the present disclosure, in the sixth or seventh aspect, the correction information may include information for integrating multiple pieces of manufacturing equipment corresponding to the manufacturing processes included in the graphic information.

According to the eighth aspect, manufacturing rule information can be appropriately generated based on modification information that includes information on integrating multiple pieces of manufacturing equipment.

In an information processing method according to a ninth aspect of the present disclosure, in any one of the sixth to eighth aspects, the manufactured products include a plurality of manufactured products with different attributes, and in creating the graphic information, proposal information is created based on the attributes to propose the addition of manufacturing equipment capable of executing the manufacturing processes included in the graphic information, and the proposal information is added to the graphic information, and the revision information includes information indicating whether or not to adopt the proposal.

According to the ninth aspect, proposal information proposing the addition of manufacturing equipment is added to the graphic information, making it easier for users to add manufacturing equipment.

The information processing method according to a tenth aspect of the present disclosure is any one of the first to ninth aspects, in which the graphical information includes nodes representing each manufacturing process and links connecting multiple nodes, and in creating the graphical information, the display mode of the corresponding nodes is set based on the processing time required for each manufacturing process.

According to the tenth aspect, the display mode of the node is set based on the time required for processing in the manufacturing process, allowing the user to easily determine the time required for processing in the manufacturing process.

In the information processing method according to the eleventh aspect of the present disclosure, in the tenth aspect, when creating the graphic information, it is preferable to further set the display mode of the corresponding link based on the movement time of the product between manufacturing processes.

According to the eleventh aspect, the display mode of the link is set based on the time it takes for the product to travel between manufacturing processes, allowing the user to easily determine the time it takes for the product to travel between manufacturing processes.

In the information processing method according to a twelfth aspect of the present disclosure, in any one of the first to eleventh aspects, the graphic information further includes processing time distribution information indicating a distribution of the time required for processing in the manufacturing process. Furthermore, processing time setting information indicating a setting value for the time required for processing in the manufacturing process set by a user based on the processing time distribution information is acquired from the input device, and the manufacturing rule information is generated based on the graphic information, correction information, and processing time setting information.

According to the twelfth aspect, the user can easily set a setting value for the processing time required based on the processing time distribution information. Furthermore, manufacturing rule information can be appropriately generated based on the graphic information, correction information, and processing time setting information.

In the information processing method according to a thirteenth aspect of the present disclosure, in the twelfth aspect, the graphic information further includes travel time distribution information indicating a distribution of the time required for moving products between manufacturing processes. Furthermore, travel time setting information indicating a setting value for the time required for moving the products between manufacturing processes, set by the user based on the travel time distribution information, is acquired from the input device, and the manufacturing rule information is generated based on the graphic information, correction information, processing time setting information, and travel time setting information.

According to the thirteenth aspect, the user can easily set a setting value for the time required for movement based on movement time distribution information. Furthermore, manufacturing rule information can be appropriately generated based on graphic information, correction information, processing time setting information, and movement time setting information.

The information processing method according to a fourteenth aspect of the present disclosure, in any one of the first to thirteenth aspects, may further create difference information indicating the difference between the manufacturing rule information and a manufacturing process performed within a predetermined period in the past, and display the difference information on the display device.

According to the fourteenth aspect, the difference information is displayed on the display device, allowing the user to easily check the difference information.

An information processing device according to a fifteenth aspect of the present disclosure is an information processing device for generating manufacturing rule information for a product manufactured through multiple manufacturing processes, and acquires image information obtained by photographing the situation in which the multiple manufacturing processes are performed in sequence on the product. Based on the image information, the information processing device creates graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution sequence of the multiple manufacturing processes. The information processing device displays the graphical information on a display device. The information processing device acquires correction information for the graphical information input by a user from an input device. Based on the graphical information and correction information, the information processing device generates manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution sequence of the multiple manufacturing processes. The information processing device outputs the manufacturing rule information.

According to the fifteenth aspect, it is possible to appropriately generate and output manufacturing rule information, including multiple manufacturing processes related to a product and rules for the execution order of those multiple manufacturing processes, based on graphic information and correction information entered by the user. As a result, it is possible to optimize manufacturing operations through factory simulation using the manufacturing rule information.

A program according to a sixteenth aspect of the present disclosure is a program for causing an information processing device to execute processing to generate manufacturing rule information for a product manufactured through multiple manufacturing processes, the processing acquiring image information obtained by photographing the situation in which the multiple manufacturing processes are performed in sequence on the product. The processing creates, based on the image information, graphical information including information indicating the multiple manufacturing processes to be performed on the product and information indicating the execution order of the multiple manufacturing processes. The processing displays the graphical information on a display device. The processing acquires, from an input device, correction information for the graphical information input by a user. The processing generates, based on the graphical information and correction information, manufacturing rule information including the multiple manufacturing processes for the product and rules for the execution order of the multiple manufacturing processes. The processing outputs the manufacturing rule information.

According to the sixteenth aspect, it is possible to appropriately generate and output manufacturing rule information, including multiple manufacturing processes related to a product and rules for the execution order of those multiple manufacturing processes, based on graphic information and correction information entered by a user. As a result, it is possible to optimize manufacturing operations through factory simulation using the manufacturing rule information.

The present disclosure can also be realized as a program that causes a computer to execute each of the characteristic components included in such a method or apparatus, or as a system operated by this program. It goes without saying that such a computer program can also be distributed on a computer-readable, non-transitory recording medium such as a CD-ROM, or via a communications network such as the Internet.

(Embodiments of the present disclosure)
Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Elements with the same reference numerals in different drawings indicate the same or corresponding elements. Furthermore, the components, the arrangement positions of the components, the connection forms, the order of operations, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. The present disclosure is limited only by the claims. Therefore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept of the present disclosure are not necessarily required to achieve the objectives of the present disclosure, but are described as constituting more preferred forms.

FIG. 1 is a simplified diagram showing the overall configuration of an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 may be a computer terminal or the like located in a manufacturing factory where products are manufactured. Products are manufactured through multiple manufacturing processes in the manufacturing factory. The manufacturing processes may include a processing process, an assembly process, an inspection process, etc.

The information processing system 1 includes an information processing device 11, a storage device 12, an input device 13, a display device 14, a communication device 15, and a photographing device 16.

The information processing device 11 is configured to include a processor such as a CPU (Central Processing Unit). The storage device 12 is configured to include a HDD (Hard Disk Drive), SSD (Solid State Device), or other semiconductor memory. The input device 13 is configured to include a mouse, keyboard, or touch panel. The display device 14 is configured to include a liquid crystal display or an organic EL (Electro Luminescence) display. The communication device 15 is configured to include a communication module compatible with any communication standard such as IP (Internet Protocol). The imaging device 16 is configured to include a camera installed on the ceiling or wall of the manufacturing factory, or above a work space where humans work within the manufacturing factory. Multiple cameras may be installed within the manufacturing factory. Also, a camera may be installed for each piece of manufacturing equipment. Furthermore, the camera may be a wide-angle camera or a 360-degree camera. The imaging device 16 captures images of multiple manufacturing processes being performed in sequence on a product, and outputs image information 32 of the moving images obtained by the capture. The image information 32 is stored in the storage device 12. The input device 13 and display device 14 may be integrated into a touch panel display.

The storage device 12 stores a program 31, image information 32, and graphic information 40. The storage device 12 includes a computer-readable non-volatile storage medium, and the program 31 is stored in this storage medium.

The information processing device 11 has an acquisition unit 21, a processing unit 22, a display control unit 23, and an output unit 24 as functions realized by the information processing device 11 executing the program 31 read from the storage device 12. Details of the processing content of these functions will be described later.

Figure 2 is a flowchart showing the flow of processing executed by the information processing device 11.

First, in step S01, the acquisition unit 21 acquires the image information 32 by reading it from the storage device 12.

Next, in step S02, the processing unit 22 creates graphic information 40 (see Figure 4, described below) based on the image information 32 acquired in step S01 and stores it in the storage device 12.

FIG. 3 is a simplified diagram illustrating an example of analysis of image information 32. The image information 32 includes multiple pieces of image information _32-1 to _32-3 of moving images. The processing unit 22 analyzes each piece of image information _32-1 to _32-3 , thereby clustering each piece of image information _32-1 to _32-3 into multiple time-series manufacturing processes. As a first example, the processing unit 22 inputs the image information _32-1 to _32-3 into a machine-learned convolutional neural network and obtains the results of the intermediate layer, thereby extracting feature quantities for each image frame. As a second example, the processing unit 22 inputs the image information _32-1 to _32-3 into an object detection model or a skeleton estimation model, thereby extracting feature quantities such as object type and its coordinates, or body part type and its coordinates, for each image frame. The processing unit 22 then performs clustering using the extracted feature quantities. As a third example, the processing unit 22 performs clustering by calculating the similarity between a set of temporally adjacent images and defining the division time for dividing the video based on the magnitude of the similarity.

The manufacturing processes include manufacturing processes performed using manufacturing equipment and manufacturing processes performed manually by workers. In the example shown in FIG. 3 , image information _32-1 is classified into multiple chronological manufacturing processes (processes A, B, and C), image information _32-2 is classified into multiple chronological manufacturing processes (processes A, D, and C), and image information _32-3 is classified into multiple chronological manufacturing processes (processes A, B, and C). Note that each classified process may be a manufacturing process such as a processing process, an assembly process, or an inspection process. Alternatively, each classified process may be a work procedure included in a single manufacturing process. For example, if the manufacturing process is a welding process, each classified process may be a work procedure for adjusting the position of a product, a work procedure for moving a welding machine to a predetermined position, or a work procedure for performing welding.

The image information corresponding to each process may include product identification information that identifies the product, manufacturing process identification information that identifies the manufacturing process performed on the product, manufacturing equipment identification information that identifies the manufacturing equipment that performed the manufacturing process, and time information that indicates the time the manufacturing process was performed. This information may be extracted from images captured by the imaging device 16. This information may also be extracted from manufacturing history information stored as a database of log data that indicates the operating history of the manufacturing equipment. Furthermore, this information may be extracted by reading tags such as RFID or two-dimensional codes attached to the product or worker.

Product identification information includes attribute information indicating the attributes of the product. The attribute information includes a lot ID indicating the lot number of the product and a product number ID indicating the product's product number. The attribute information may also include a worker ID identifying the worker. Manufacturing process identification information includes a process ID indicating the identification number of the manufacturing process. Manufacturing equipment identification information includes an equipment ID indicating the identification number of the manufacturing equipment. Time information includes a start time indicating the time when the manufacturing process started and an end time indicating the time when the manufacturing process ended.

Next, in step S03, the display control unit 23 causes the display device 14 to display the graphic information 40 created in step S02.

Figure 4 is a simplified diagram showing an example of graphical information 40 displayed on the display device 14. The graphical information 40 includes information indicating multiple manufacturing processes to be performed on a product and information indicating the execution sequence of the multiple manufacturing processes. In the example shown in Figure 4, the graphical information 40 represents a directed graph including nodes representing each manufacturing process and links that directly or indirectly connect the multiple nodes. Note that the graphical information 40 is not limited to a directed graph and may be a Gantt chart or the like.

In the example shown in Figure 4, the nodes include a start node NS, multiple nodes NA-ND corresponding to multiple manufacturing processes (processes A-D), and an end node NE. The links include a link LSA pointing from the start node NS to node NA, a link LAB pointing from node NA to node NB, a link LBC pointing from node NB to node NC, a link LAD pointing from node NA to node ND, a link LDC pointing from node ND to node NC, and a link LCE pointing from node NC to the end node NE. In addition, the graphic information 40 includes an icon 38 labeled "Edit" and an icon 39 labeled "Confirm."

Next, in step S04, the processing unit 22 determines whether the user has input correction information from the input device 13.

If the "Edit" icon 38 is not selected and the "Confirm" icon 39 is selected instead, this indicates that no correction information has been entered by the user. If no correction information has been entered (step S04: NO), then in step S06, the processing unit 22 generates manufacturing rule information based on the graphic information 40, including multiple manufacturing processes related to the product and rules for the execution order of the multiple manufacturing processes.

Figure 5 is a simplified diagram showing an example of manufacturing rule information 50 generated when no correction information is input. Manufacturing rule information 50 indicates a process path of "A, B, C" or a process path of "A, D, C." The process path of "A, B, C" corresponds to the links LAB and LBC shown in Figure 4, and indicates a rule that multiple manufacturing processes are executed in the order of process A → process B → process C. The process path of "A, D, C" corresponds to the links LAD and LDC shown in Figure 4, and indicates a rule that multiple manufacturing processes are executed in the order of process A → process D → process C.

Next, in step S07, the output unit 24 outputs the manufacturing rule information 50 generated in step S06. The output manufacturing rule information 50 may be input into a factory simulator using a CPS (Cyber-Physical System) or the like, or may be stored in any storage device. The manufacturing rule information 50 may be written in a programming language that can be input into a factory simulator, or may be in the data format of master data registered in a factory database or the like as management information for manufacturing rules. Alternatively, the manufacturing rule information 50 may be in the format of a work procedure manual used by workers to carry out manufacturing.

On the other hand, the user can select the "Edit" icon 38 and then edit the graphical information 40 by operating the input device 13. Editing the graphical information 40 indicates that the user has entered edit information. Editing the graphical information 40 includes deleting or changing the manufacturing processes included in the graphical information 40. Deleting a manufacturing process includes, for example, deleting process D. Changing a manufacturing process includes, for example, replacing process D with another process. In the case of an exceptional manufacturing history resulting from an unusual operation, such as the production of a sample product, deleting process D can remove it from the manufacturing rules. The user can delete or change the node ND, for example, by moving the mouse cursor over the node ND to be edited and right-clicking to display a menu that displays "Delete" or "Edit." The following example describes a case where the user edits the node ND by deleting process D. When the user moves the mouse cursor over the node ND to be edited, the display control unit 23 may display on the display device 14 a representative image showing a characteristic operation in the manufacturing process corresponding to the node ND. The representative image is created by the processing unit 22 based on the image information 32.

If correction information has been input (step S04: YES), then in step S05, the processing unit 22 acquires the correction information from the input device 13. The processing unit 22 then executes steps S02 to S04 again, reflecting the correction content.

Figure 6 is a simplified diagram showing an example of the graphic information 41 displayed on the display device 14 after the deletion correction. As a result of the correction to delete process D, node ND and links LAD and LDC have been deleted from the graphic information 40 shown in Figure 4.

When the "OK" icon 39 is selected after the corrections are complete, the processing unit 22 then generates manufacturing rule information, including multiple manufacturing processes related to the product and rules for the execution order of those multiple manufacturing processes, based on the graphic information 40 and the correction information, in step S06.

Figure 7 is a simplified example of manufacturing rule information 51 generated when deletion correction information is input. As a result of the correction to delete process D, the process route "A, D, C" is deleted from the manufacturing rule information 50 shown in Figure 5, and manufacturing rule information 51 contains only the process route "A, B, C."

Next, in step S07, the output unit 24 outputs the manufacturing rule information 51 generated in step S06. The output manufacturing rule information 51 may be input into a factory simulator using a CPS or the like, or may be stored in any storage device. The manufacturing rule information 51 may be written in a programming language that can be input into a factory simulator, or may be in the data format of master data registered in a factory database or the like as management information for manufacturing rules. Alternatively, the manufacturing rule information 50 may be in the format of a work procedure manual used by workers to carry out manufacturing.

According to this embodiment, when correction information is input by the user, the information processing device 11 can appropriately generate and output manufacturing rule information 51, which includes multiple manufacturing processes related to the product and rules for the execution order of those multiple manufacturing processes, based on the graphic information 40 and the correction information. As a result, it becomes possible to optimize manufacturing operations through factory simulation using the manufacturing rule information 51.

Furthermore, the information processing device 11 can appropriately generate manufacturing rule information 51 based on modification information that includes information for deleting or changing manufacturing processes.

Various variations of the above embodiment are described below. The variations described below can be applied in any combination.

(First Modification)
The modification information may include information for adding a branching condition from a certain manufacturing process included in the graphic information 40 to a plurality of subsequent manufacturing processes. In the example of the graphic information 40 shown in Fig. 4, a branch from a node NA corresponding to a process A to a node NB corresponding to a process B or a node ND corresponding to a process D is included. For example, the user can add a branching condition from the node NA to the node ND by moving the mouse cursor over the link LAD to which the branching condition is to be added and then right-clicking to display a menu, thereby selecting "Add Branching Condition."

Figure 8 is a simplified example of the graphical information 42 displayed on the display device 14 after modification to add a branching condition. A branching condition "if process A. abnormal" has been added to the link LAD. This branching condition can be selected from columns included in the log data, or even if it is not included in the log data, new information associated with the log data for the lot or equipment, etc., can be added. This branching condition means that if process A is abnormal, branch from node NA to node ND.

Figure 9 is a simplified diagram showing an example of manufacturing rule information 52 generated when modification information for adding a branch condition is input. As a result of a modification that adds a branch condition from node NA to node ND, manufacturing rule information 52 is output from output unit 24, indicating that if an abnormality occurs in process A, the process path "A, D, C" will be adopted, and otherwise the process path "A, B, C" will be adopted.

According to this modified example, the information processing device 11 can appropriately generate manufacturing rule information 52 based on modification information that includes information for adding branch conditions.

(Second Modification)
The correction information may include information for dividing or integrating the graphic information 40 according to the attributes of the product. An example in which the attribute is a product number ID will be described below. However, the attribute may also be a lot ID or a manufacturing date and time, etc. The graphic information 40 shown in FIG. 4 includes information related to all product number IDs. The user can divide the graphic information 40 by product number ID, for example, by moving the mouse cursor over the start node NS and right-clicking in that state to display a menu and selecting "Split by product number."

Figure 10 is a simplified diagram showing an example of graphical information 43 displayed on display device 14 after the division has been corrected. Graphic information 43 includes graphical information 43X corresponding to product number X, graphical information 43Y corresponding to product number Y, and graphical information 43Z corresponding to product number Z. In other words, graphical information 40 is divided into graphical information 43X, 43Y, and 43Z according to the product number ID.

Figure 11 is a simplified diagram showing an example of manufacturing rule information 53 generated when division correction information is input. By dividing the graphic information 40 by product number ID, the output unit 24 outputs manufacturing rule information 53, which indicates that the process route "A, B, C" is adopted for product number X, the process route "A, D, C" is adopted for product number Y, and the process route "A, B, C" is adopted for product number Z.

Furthermore, in the example shown in Figure 10, the graphic information 43X corresponding to product number X and the graphic information 43Z corresponding to product number Z are the same. The user can integrate graphic information 43Z with graphic information 43X, for example, by using the mouse to specify the range of graphic information 43Z and then dragging and dropping it onto graphic information 43X.

12 is a simplified diagram showing an example of graphic information 44 displayed on display device 14 after the integration correction. Graphic information 44 includes graphic information 441 corresponding to group G1 including part numbers X and Z, and graphic information ₄₄₂ corresponding to group G2 including part number Y. In other words, graphic information 43X and 43Z are integrated into graphic information ₄₄₁ according to _the part number ID.

13 is a simplified diagram showing an example of manufacturing rule information 54 generated when integration correction information is input. By integrating the graphic information 43X and 43Z into the graphic information ₄₄₁ , the output unit 24 outputs the manufacturing rule information 54, which indicates that the process route of "A, B, C" is adopted for the group G1 and the process route of "A, D, C" is adopted for the group G2.

In addition, for groups where there is a large difference between the part number properties, etc. and the manufacturing rule information 53, 54, the processing unit 22 may display an alert on the display device 14 when displaying the graphic information 43, 44. For example, an alert may be displayed when separate data on part number properties, such as the types of parts to be assembled or parameters of the manufacturing process, are stored and parts are divided into different groups despite having similar properties. Alternatively, an alert may be displayed when parts are combined into the same group despite having significantly different properties. The user may accept or delete the displayed alert.

According to this modified example, manufacturing rule information 53 can be appropriately generated based on modification information that includes information for dividing graphic information 40 according to the attributes of the product. Furthermore, simplified manufacturing rule information 54 can be appropriately generated based on modification information that includes information for integrating multiple pieces of graphic information 43 according to the attributes of the product. This is particularly effective in manufacturing factories with a large variety of products.

(Third Modification)
In the second modified example, the graphic information is divided or integrated by user operation, but the processing unit 22 may classify multiple products into multiple groups G1 and G2 based on attribute information, and automatically create graphic information 44 ₁ and 44 ₂ for each group G1 and G2 instead of graphic information 40 and propose them to the user.

The processing unit 22 classifies multiple product numbers into multiple groups based on the similarity of the graph structures in the graphic information 43 for each product number. As a first example, the processing unit 22 calculates the number of operations required to convert a directed graph corresponding to a certain product number into a directed graph corresponding to another product number as the similarity. The processing unit 22 then classifies multiple product numbers into multiple groups by performing clustering using a similarity matrix. As a second example, the processing unit 22 vectorizes the process path for each product number using a vector that indicates, with a binary value of "0" or "1," whether or not each process is included in the process path. The processing unit 22 then classifies multiple product numbers into multiple groups by performing clustering using the vectorized process path.

The user inputs whether or not to adopt the graphic information 44 ₁ and 44 ₂ for the groups G1 and G2 proposed by the processing unit 22 by operating the input device 13. Alternatively, the user may further modify the graphic information 44 ₁ and 44 ₂ for the groups G1 and G2 proposed by the processing unit 22 by the division or integration operation described in the second modified example.

According to this modification, the information processing device 11 automatically creates the graphic information 44 ₁ and 44 ₂ for the groups G1 and G2, eliminating the need for the user to perform group classification operations, improving user convenience. This is particularly effective in manufacturing factories with a large variety of products.

(Fourth Modification)
In the embodiment, the graphic information 40 includes nodes NA to ND representing each manufacturing process, but instead of the nodes NA to ND, it may include a node representing at least one piece of manufacturing equipment that performs each manufacturing process.

Figure 14 is a simplified diagram showing an example of graphic information 45 displayed on display device 14. Processing unit 22 creates graphic information 45 by identifying manufacturing equipment based on manufacturing equipment identification information included in image information corresponding to each process. Graphic information 45 includes nodes NA1 and NA2 indicating the manufacturing equipment (equipment U1 and U2) that performs process A, nodes NB1 to NB3 indicating the manufacturing equipment (equipment V1 to V3) that performs process B, and nodes NC1 and NC2 indicating the manufacturing equipment (equipment W1 and W2) that performs process C. Graphic information 45 also includes links that connect nodes NA1 and NA2, nodes NB1 to NB3, and nodes NC1 and NC2 as necessary.

In this modified example, the modification information may include information for adding manufacturing equipment capable of executing a manufacturing process. After selecting the "Modify" icon 38, the user can modify the graphic information 45 by operating the input device 13. For example, the user can add nodes and links by moving the mouse cursor to the location where they want to add a node, right-clicking in that state, and selecting "Add" from the menu that appears.

Figure 15 is a simplified diagram showing an example of graphical information 46 displayed on the display device 14 after additional modifications. Node NB4, which corresponds to equipment V4 capable of executing process B, has been added to the graphical information 45 shown in Figure 14. Additionally, link LU2V4 connecting node NA2 and node NB4, and link LV4W2 connecting node NB4 and node NC2 have been added to the graphical information 45 shown in Figure 14.

The modification information may also include information for integrating multiple pieces of manufacturing equipment included in the graphic information 46. In the example shown in FIG. 15, equipment V3 and V4 share both upstream equipment U2 and downstream equipment W2. The user can integrate node NB4 into node NB3, for example, by using the mouse to specify the range of node NB4 and then dragging and dropping it onto node NB3.

Figure 16 is a simplified example of graphical information 47 displayed on the display device 14 after the integration has been corrected. Graphical information 47 includes node NB34, which is the integration of nodes NB3 and NB4 shown in Figure 15; link LU2V34, which is the integration of links LU2V3 and LU2V4 shown in Figure 15; and link LV34W2, which is the integration of links LV3W2 and LV4W2 shown in Figure 15.

According to this modified example, the graphic information 45 includes information indicating the manufacturing equipment, so the information processing device 11 can generate and output more detailed manufacturing rule information that includes the order of the manufacturing equipment instead of the manufacturing process.

Furthermore, according to this modified example, manufacturing rule information corresponding to the graphic information 46 can be appropriately generated based on modification information that includes information on adding manufacturing equipment.

Furthermore, according to this modified example, simplified manufacturing rule information corresponding to the graphic information 47 can be appropriately generated based on modification information that includes information on integrating multiple pieces of manufacturing equipment. This is particularly effective in manufacturing plants with a large variety of manufacturing equipment.

(Fifth Modification)
In the fourth variant, node NB4 was added by user operation, but the processing unit 22 may automatically create manufacturing equipment capable of executing the manufacturing process included in the graphic information 45 based on the attribute information and manufacturing equipment identification information and propose it to the user.

The processing unit 22 creates proposal information proposing the addition of manufacturing equipment capable of executing the manufacturing process included in the graphic information 45 based on the similarity between the product numbers and the similarity between the manufacturing equipment, and adds the proposal information to the graphic information 45. For example, if equipment V3 is being used for a certain product number, the processing unit 22 determines that equipment V4, whose similarity with equipment V3 is equal to or greater than a predetermined value, is also likely to be usable for that product number, and creates proposal information proposing the addition of equipment V4.

Figure 17 is a simplified diagram showing an example of graphical information 48 in which proposal information has been added to graphical information 45. Graphic information 48 includes proposal information shown by dashed lines in Figure 17. The proposal information includes node NB4 corresponding to facility V4. The proposal information also includes link LU2V4 connecting node NA2 and node NB4, and link LV4W2 connecting node NB4 and node NC2. The user can input, as revision information, from input device 13 whether or not to adopt the proposal shown in the proposal information.

According to this modified example, proposal information proposing the addition of manufacturing equipment is included in the graphic information 48, making it easier for users to add manufacturing equipment and improving user convenience.

(Sixth Modification)
The processing unit 22 may set different display modes for corresponding nodes based on the time required for processing in each manufacturing process when creating the graphic information 40. Furthermore, the processing unit 22 may set different display modes for corresponding links based on the travel time of products between manufacturing processes when creating the graphic information 40. The travel time includes the transport time for transporting products between manufacturing devices and the waiting time or residence time until processing by the manufacturing device starts after the transport is completed.

Figure 18 is a simplified diagram showing an example of graphic information 40 displayed on the display device 14. The longer the processing time required for each manufacturing process, the thicker the line width of the edge at the corresponding node is displayed. Also, the longer the movement time of the product between manufacturing processes, the thicker the line width of the corresponding link is displayed. Note that the processing unit 22 may set the display mode of the node or link to differ depending on the occurrence frequency, etc., instead of the required time. Also, the processing unit 22 may set the display mode to differ depending on the display color, etc., instead of the line width.

In this modified example, the display mode of the nodes is set based on the required time or frequency of occurrence of processing in the manufacturing process, allowing the user to visually and easily determine the required time or frequency of occurrence of processing in the manufacturing process.

In addition, the display mode of the links is set based on the time it takes for products to travel between manufacturing processes or their frequency of occurrence, so users can easily visually determine the time it takes for products to travel between manufacturing processes or their frequency of occurrence, etc.

(Seventh Modification)
The graphic information 40 may further include processing time distribution information that indicates the distribution of the time required for processing in each manufacturing process.

19 and 20 are simplified diagrams showing another example of the graphical information 40 displayed on the display device 14. When the user moves the mouse cursor over node ND, for example, the processing unit 22 creates processing time distribution information K1 for process D corresponding to node ND, as shown in FIG. 19. The display control unit 23 displays the created processing time distribution information K1 in a pop-up on the display device 14. The horizontal axis of the processing time distribution information K1 indicates time, and the vertical axis indicates occurrence frequency. When the user inputs a product number specification, the processing unit 22 creates processing time distribution information K2, which indicates the distribution of processing times for each specified product number, as shown in FIG. 20. The display control unit 23 displays the created processing time distribution information K2 in a pop-up on the display device 14. The processing time distribution information K2 shown in FIG. 20 includes two pieces of distribution information for two product numbers. The user inputs representative values for each piece of distribution information as the setting values V1 and V2 for the processing times. The representative value may be the average value, mode, median, or the top m% (m is arbitrary) of values, and the position of each value may be specified on the screen by a user operation, or each value may be calculated by statistical processing by the processing unit 22.

The acquisition unit 21 acquires processing time setting information indicating the setting values V1 and V2 from the input device 13. The processing unit 22 generates manufacturing rule information based on the graphic information 40, the correction information, and the processing time setting information. The manufacturing rule information may also include processing time distribution information K2. The processing time distribution information K2 included in the manufacturing rule information may be defined using a function formula representing a normal distribution approximated by maximum likelihood estimation, etc.

Similarly, the graphical information 40 may further include movement time distribution information that indicates the distribution of the time required to move products between manufacturing processes.

21 is a simplified diagram showing another example of the graphic information 40 displayed on the display device 14. When the user moves the mouse cursor over a link LAD, for example, the processing unit 22 creates travel time distribution information corresponding to the link LAD. When the user inputs a product number, the processing unit 22 creates travel time distribution information K3, which indicates the distribution of travel times for each specified product number, as shown in FIG. 21. The display control unit 23 displays the created travel time distribution information K3 as a pop-up on the display device 14. The travel time distribution information K3 shown in FIG. 21 includes two pieces of distribution information for two product numbers. The user inputs representative values for each piece of distribution information as set values V3 and V4 for travel times. The representative value may be the average, mode, median, or the top m% (m is arbitrary) value. The position of each value may be specified on the screen by the user's operation, or each value may be calculated by statistical processing by the processing unit 22.

The acquisition unit 21 acquires travel time setting information indicating the setting values V3 and V4 from the input device 13. The processing unit 22 generates manufacturing rule information based on the graphic information 40, correction information, processing time setting information, and travel time setting information. The manufacturing rule information may also include travel time distribution information K3. The travel time distribution information K3 included in the manufacturing rule information may be defined using a function formula representing a normal distribution approximated by maximum likelihood estimation, etc.

According to this modified example, the user can easily set the setting values V1 and V2 for the processing time required based on the processing time distribution information K2. Furthermore, the information processing device 11 can appropriately generate manufacturing rule information based on the graphic information 40, correction information, and processing time setting information.

Furthermore, according to this modified example, the user can easily set the setting values V3 and V4 related to the travel time required based on the travel time distribution information K3. Furthermore, the information processing device 11 can appropriately generate manufacturing rule information based on the graphic information 40, correction information, processing time setting information, and travel time setting information.

Furthermore, according to this modified example, the manufacturing rule information includes set values for the processing time required for each product number in each manufacturing process, and set values for the travel time required for each product number between manufacturing processes, making it possible to build a highly accurate factory simulator that includes the processing time and travel time required, which are essential for simulation.

(Eighth Modification)
The processing unit 22 may create difference information indicating the difference between the manufacturing rule information and the manufacturing process performed within a specified period in the past, and the display control unit 23 may display the created difference information on the display device 14.

FIG. 22 is a flowchart showing the flow of processing in a modified example executed by the information processing device 11.

In step S08, the processing unit 22 compares the manufacturing rule information created in step S06 with the log data of manufacturing processes executed within the most recent specified period. The processing unit 22 extracts log data that violates the manufacturing rule information and creates difference information that summarizes the extracted log data in a list format.

Next, in step S09, the display control unit 23 causes the display device 14 to display the difference information created in step S08.

In this modified example, the difference information is displayed on the display device 14, allowing the user to easily check the difference information. As a result, it becomes possible to determine the cause of the violation for log data that violates manufacturing rule information.

(Ninth Modification)
The correction information may include information for changing notation such as "Process A" included in the graphic information 40 to the actual name of each process such as "Work to remove jigs and work-in-progress" or "Processing using jigs." This makes it easier for the user to intuitively understand.

(Tenth Modification)
The modification information may include information for dividing or merging steps in the graphic information 40. The graphic information 40 is constructed based on the clusters proposed by the processing unit 22.

For example, the user can move the mouse cursor over node NA, then right-click to display a menu, select "Split Process," and select the time of division, thereby splitting process A in the graphic information 40 into two processes. At this time, the processing unit 22 may further cluster the data of each image included in process A and present the user with a suitable position for division.

Alternatively, the user can move the mouse cursor over node NA, right-click in that state, select "Merge Processes" from the menu that appears, and then select another process (for example, process B) as the process to be merged, thereby merging process A and process B in the graphic information 40 to create a new process A. At this time, the processing unit 22 may present the user with the similarity between process A and the other process. This similarity is calculated, for example, based on the similarity between the central images of each process.

According to this modified example, even if the proposed clustering results are inaccurate, the inaccurate clustering results can be corrected, and as a result, accurate manufacturing rule information can be output for each process.

(Eleventh Modification)
The output unit 24 may have a function of outputting the image data for each process unit corrected by the user in Modification 10. Furthermore, the processing unit 22 may perform clustering on new data using a machine learning model trained using this image data as training data.

According to this modified example, the accuracy of clustering can be improved by using a machine learning model trained using the target image data itself.

As another variation, the display control unit 23 may display the programming code of the manufacturing rule information on the display device 14. This makes it easier for developers who are familiar with programming to intuitively understand it.

This disclosure is widely applicable to manufacturing management systems for building factory simulators, etc.

REFERENCE SIGNS LIST 1 Information processing system 11 Information processing device 12 Storage device 13 Input device 14 Display device 15 Communication device 16 Photography device 21 Acquisition unit 22 Processing unit 23 Display control unit 24 Output unit 31 Program 32, 32 ₁ , 32 ₂ , 32 ₃ Image information 40 to 48 Graphic information 50 to 54 Manufacturing regulation information

Claims (16)

An information processing method for generating manufacturing rule information for a product manufactured through a plurality of manufacturing processes, comprising:
The information processing device
Acquiring image information obtained by photographing a situation in which a plurality of manufacturing processes are performed on a product in order;
creating, based on the image information, graphic information including information indicating a plurality of manufacturing steps to be executed on the product and information indicating the order in which the plurality of manufacturing steps are executed;
Displaying the graphic information on a display device;
acquiring correction information for the graphic information input by a user from an input device;
generating manufacturing rule information including a plurality of manufacturing processes related to the product and rules for the order of execution of the plurality of manufacturing processes based on the graphic information and the correction information;
outputting the manufacturing rule information;
Information processing methods. the modification information includes information for deleting or changing a manufacturing process included in the graphic information;
The information processing method according to claim 1 . the modification information includes information for adding a branching condition from a manufacturing process included in the graphic information to a plurality of subsequent manufacturing processes;
The information processing method according to claim 1 . the products include a plurality of products with different attributes;
the modification information includes information for dividing or integrating the graphic information according to the attribute;
The information processing method according to claim 1 . the products include a plurality of products with different attributes;
In creating the graphic information, the plurality of products are classified into a plurality of groups based on the attributes, and the graphic information is created for each group.
The information processing method according to claim 1 . The image information includes manufacturing equipment that performed each manufacturing process,
the graphic information further includes information indicating at least one manufacturing facility that performs each manufacturing process;
The information processing method according to claim 1 . the modification information includes information for adding manufacturing equipment capable of executing the manufacturing process included in the graphic information;
The information processing method according to claim 6. the modification information includes information for integrating a plurality of manufacturing facilities corresponding to the manufacturing process included in the graphic information;
The information processing method according to claim 6. the products include a plurality of products with different attributes;
In creating the graphic information, proposal information is created based on the attributes, proposing the addition of manufacturing equipment capable of executing the manufacturing process included in the graphic information, and the proposal information is added to the graphic information;
The modification information includes information indicating whether or not the proposal is to be adopted.
The information processing method according to claim 6. the graphic information includes nodes representing each manufacturing process and links connecting the plurality of nodes;
In creating the graphic information, a display mode of a corresponding node is set based on the required processing time of each manufacturing process.
The information processing method according to claim 1 . In the creation of the graphic information, a display mode of the corresponding link is set based on the movement time of the product between the manufacturing processes.
The information processing method according to claim 10. the graphic information further includes processing time distribution information indicating a distribution of processing times in the manufacturing process;
Furthermore, processing time setting information indicating a setting value related to a required time for processing in a manufacturing process, which is set by a user based on the processing time distribution information, is acquired from the input device;
In generating the manufacturing rule information, the manufacturing rule information is generated based on the graphic information, the correction information, and the processing time setting information.
The information processing method according to claim 1 . the graphic information further includes movement time distribution information indicating a distribution of the time required for movement of the product between manufacturing processes;
Furthermore, the method acquires, from the input device, travel time setting information indicating a setting value related to the time required for the transportation of the products between the manufacturing processes, which is set by a user based on the travel time distribution information;
In generating the manufacturing rule information, the manufacturing rule information is generated based on the graphic information, the correction information, the processing time setting information, and the movement time setting information.
The information processing method according to claim 12. moreover,
creating difference information indicating a difference between the manufacturing rule information and a manufacturing process executed within a predetermined period in the past;
displaying the difference information on the display device;
The information processing method according to claim 1 . An information processing device for generating manufacturing rule information regarding a product manufactured through a plurality of manufacturing processes,
Acquiring image information obtained by photographing a situation in which a plurality of manufacturing processes are performed on a product in order;
creating, based on the image information, graphic information including information indicating a plurality of manufacturing steps to be executed on the product and information indicating the order in which the plurality of manufacturing steps are executed;
Displaying the graphic information on a display device;
acquiring correction information for the graphic information input by a user from an input device;
generating manufacturing rule information including a plurality of manufacturing processes related to the product and rules for the order of execution of the plurality of manufacturing processes based on the graphic information and the correction information;
outputting the manufacturing rule information;
Information processing device. A program for causing an information processing device to execute processing for generating manufacturing rule information related to a product manufactured through a plurality of manufacturing processes,
The process comprises:
Acquiring image information obtained by photographing a situation in which a plurality of manufacturing processes are performed on a product in order;
creating, based on the image information, graphic information including information indicating a plurality of manufacturing steps to be executed on the product and information indicating the order in which the plurality of manufacturing steps are executed;
Displaying the graphic information on a display device;
acquiring correction information for the graphic information input by a user from an input device;
generating manufacturing rule information including a plurality of manufacturing processes related to the product and rules for the order of execution of the plurality of manufacturing processes based on the graphic information and the correction information;
outputting the manufacturing rule information;
program.