WO2025109882A1 - Analysis device, analysis method, program, learned machine learning model, and machine learning method - Google Patents

Abstract

This analysis device comprises: an acquisition unit that acquires time-series data representing at least the operation state of a first movable part and the operation state of a second movable part of a work machine having the first movable part and the second movable part; a first operation state classification unit that classifies the operation state of the first movable part on the basis of the time-series data; a division unit that divides the time axis of the time-series data into a plurality of sections on the basis of the classification result of the operation state of the first movable part; a second operation state classification unit that classifies the operation state of the second movable part for each of the sections on the basis of the time-series data; and a work classification unit that classifies work by the work machine on the basis of the classification result of the operation state of the first movable part and the classification result of the operation state of the second movable part.

Description

Analysis device, analysis method, program, trained machine learning model, and machine learning method

This disclosure relates to an analysis device, an analysis method, a program, a trained machine learning model, and a machine learning method. This application claims priority to Japanese Patent Application No. 2023-196595, filed on November 20, 2023, the contents of which are incorporated herein by reference.

Patent Document 1 discloses a technology that, when an operator operates a wheel loader to perform a cyclical operation (loading operation in this example) that repeats each of the operation modes of unladen forward movement, excavation, loaded reverse movement, loaded forward movement, loading, and unladen reverse movement, distinguishes each operation mode included in the cyclical operation based on a numerical value that indicates the state of the wheel loader.

International Publication No. 2022/244632

Incidentally, wheel loaders perform multiple types of work, including loading, sorting of excavated material, and shoveling. In addition, with work machines such as wheel loaders, there is a need to efficiently analyze what type of work the work machine is performing based on time-series data that indicates the operating status of the work machine.

The present disclosure has been made in consideration of the above circumstances, and aims to provide an analysis device, an analysis method, a program, a trained machine learning model, and a machine learning method that can efficiently classify work based on time-series data that represents the operating status of a work machine.

The analysis device disclosed herein includes an acquisition unit that acquires time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having a first movable part and a second movable part, a first operating state classification unit that classifies the operating state of the first movable part based on the time series data, a division unit that divides the time axis of the time series data into a plurality of sections based on the classification result of the operating state of the first movable part, a second operating state classification unit that classifies the operating state of the second movable part for each of the sections based on the time series data, and a work classification unit that classifies work performed by the work machine based on the classification result of the operating state of the first movable part and the classification result of the operating state of the second movable part.

The analysis method disclosed herein also includes the steps of acquiring time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having a first movable part, classifying the operating state of the first movable part based on the time series data, dividing the time axis of the time series data into a plurality of sections based on the classification result of the operating state of the first movable part, classifying the operating state of the second movable part for each section based on the time series data, and classifying work performed by the work machine based on the classification result of the operating state of the first movable part and the classification result of the operating state of the second movable part.

The program disclosed herein also causes a computer to execute the steps of acquiring time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having a first movable part and a second movable part, classifying the operating state of the first movable part based on the time series data, dividing the time axis of the time series data into a plurality of sections based on the classification results of the operating state of the first movable part, classifying the operating state of the second movable part for each section based on the time series data, and classifying work performed by the work machine based on the classification results of the operating state of the first movable part and the classification results of the operating state of the second movable part.

The trained machine learning model of the present disclosure is a machine learning model that inputs predetermined input information based on time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, and outputs output information representing the result of classifying the operating state of the second movable part, the input information including a plurality of predetermined feature quantities related to the operating state of the second movable part for each of the sections obtained by dividing the time axis of the time series data into a plurality of sections based on the classification result of the operating state of the first movable part, and is machine-trained using the input information labeled with the type of the operating state of the second movable part.

The machine learning method disclosed herein is a machine learning method for a machine learning model that inputs predetermined input information based on time series data representing at least the operating state of a first movable part and the operating state of a second movable part of a work machine having the first movable part and the second movable part, and outputs output information representing the result of classifying the operating state of the second movable part, the input information including a plurality of predetermined feature quantities related to the operating state of the second movable part for each of the sections obtained by dividing the time axis of the time series data into a plurality of sections based on the classification result of the operating state of the first movable part, and machine learning the machine learning model using the input information labeled with the type of the operating state of the second movable part.

The analysis device, analysis method, program, trained machine learning model, and machine learning method disclosed herein classify the operating state of the second moving part for each of a plurality of sections into which time series data is divided based on the classification result of the first moving part, and therefore can efficiently classify the operating state of the second moving part. Therefore, the work of a work machine having a first moving part and a second moving part can be efficiently classified based on the time series data.

FIG. 1 is a schematic diagram illustrating a configuration example of an analysis system according to an embodiment of the present disclosure. 1 is a side view showing an example configuration of a work machine according to an embodiment of the present disclosure. FIG. FIG. 2 is a schematic diagram illustrating an example of the configuration of a time-series data file according to an embodiment of the present disclosure. 4 is a schematic diagram showing an example of a behavior discrimination flag of a traveling device (undercarriage of a work machine) according to an embodiment of the present disclosure. FIG. FIG. 13 is a diagram illustrating an example of time-series data related to behavior determination of a traveling device according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of time-series data related to behavior determination of a traveling device according to an embodiment of the present disclosure. FIG. 1 is a diagram illustrating an example in which time-series data is divided into a plurality of intervals according to an embodiment of the present disclosure. 11 is a diagram showing an example of time-series data relating to behavior determination of a work machine according to an embodiment of the present disclosure. FIG. FIG. 9 is a diagram illustrating an example of an operating state of a work machine in section (k) illustrated in FIGS. 7 and 8 . FIG. 9 is a diagram illustrating an example of an operating state of a work machine in a section (k+1) illustrated in FIGS. 7 and 8 . FIG. 9 is a diagram illustrating an example of an operating state of a work machine in a section (k+2) illustrated in FIGS. 7 and 8 . FIG. 9 is a diagram illustrating an example of an operating state of a work machine in a section (k+3) illustrated in FIGS. 7 and 8 . FIG. 9 is a diagram illustrating an example of an operating state of a work machine in a section (k+4) illustrated in FIGS. 7 and 8 . FIG. 11 is a diagram illustrating an example of feature amounts according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of feature amounts of a section (k) according to an embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of feature amounts according to an embodiment of the present disclosure. 11 is a schematic diagram showing an example of a behavior determination flag of a work machine according to an embodiment of the present disclosure. FIG. FIG. 13 is a diagram illustrating an example of feature amounts in a section (k+1) according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of feature amounts in a section (k+2) according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of feature amounts in a section (k+3) according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of feature amounts in a section (k+4) according to an embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of teacher data according to an embodiment of the present disclosure. FIG. 1 is a schematic diagram illustrating an example of task classification according to an embodiment of the present disclosure. FIG. 1 is a schematic diagram for explaining an example of task classification according to an embodiment of the present disclosure. FIG. 9 is a schematic diagram for explaining an example of task classification of sections (k) to (k+4) shown in FIGS. 7 and 8. FIG. 13 is a diagram illustrating an example of a classification result according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a result of a detailed analysis according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a result of a detailed analysis according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a result of a detailed analysis according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a result of a detailed analysis according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a result of a detailed analysis according to an embodiment of the present disclosure. 10 is a flowchart illustrating an example of the operation of an analysis device according to an embodiment of the present disclosure.

Below, an embodiment of the present disclosure will be described with reference to the drawings. Note that the same or corresponding components in each drawing will be designated by the same reference numerals and their description will be omitted as appropriate.

(Analysis system)
FIG. 1 is a schematic diagram showing a configuration example of an analysis system according to an embodiment of the present disclosure. The analysis system SY1 shown in FIG. 1 includes a wheel loader 1 as an example of a work machine to be analyzed, a data management server 20 that is communicatively connected to the wheel loader 1 via a communication network NW, and an analysis device 30. The wheel loader 1 includes a vehicle body control device 100 and an operation information recording device 101. The vehicle body control device 100 inputs sensor information measured by a plurality of sensors included in the wheel loader 1, and controls each part of the wheel loader 1 according to an operation of an operator. The operation information recording device 101 records time series data representing the operating state of the wheel loader 1, such as a predetermined control signal and sensor information generated by the vehicle body control device 100, for a certain period (or a certain amount of data), and transmits the time series data to the data management server 20 via the communication network NW. The data management server 20 accumulates the time series data received from the operation information recording device 101 as a time series data file 21. In this embodiment, the analysis device 30 is provided separately from the data management server 20, but is not limited to this. The analysis device 30 may be provided within the data management server 20 .

(Work machine)
Here, the wheel loader 1 will be described with reference to Fig. 2. Fig. 2 is a side view showing an example configuration of a work machine (wheel loader 1) according to an embodiment of the present disclosure. As shown in Fig. 2, the wheel loader 1 has a vehicle body 2, a cab 3, a traveling device 4, and a working implement 10. The wheel loader 1 travels on the traveling device 4 at a work site. The wheel loader 1 performs work at the work site using the working implement 10. The wheel loader 1 can perform work such as excavation work, loading work, transporting work, and snow removal work using the working implement 10.

The cab 3 is supported by the vehicle body 2. Inside the cab 3, a driver's seat where the operator sits, operating devices, a display input unit, etc. are arranged.

The traveling device 4 has a power transmission device, rotatable wheels 5, etc., not shown. The wheels 5 support the vehicle body 2. The wheel loader 1 is capable of traveling on the road surface (or ground) RS by the traveling device 4. Note that in FIG. 2, only the left front wheel 5F and rear wheel 5R are shown. The power transmission device transmits the driving force of a power source, which will be described later, to the wheels 5. In this embodiment, the power transmission device has a transmission with four forward speeds, neutral, and four reverse speeds (speed stages). Note that the traveling device 4 is one example configuration of the "traveling section" of this disclosure.

The work machine 10 is supported by the vehicle body 2. The work machine 10 is composed of a bucket 12, which is an example of a work tool, and a movable support part 17 that changes the position and attitude of the bucket 12. In the example shown in FIG. 2, the movable support part 17 includes a boom 11, a boom cylinder 13, a bucket cylinder 14, a bell crank 15, and a link 16.

The boom 11 is supported rotatably on the vehicle body 2, and moves up and down in response to the extension and contraction of the boom cylinder 13. The boom cylinder 13 is an actuator that generates power to move the boom 11, with one end connected to the vehicle body 2 and the other end connected to the boom 11. When an operator operates a boom operation device (not shown), the boom cylinder 13 extends and contracts. This causes the boom 11 to move up and down. The boom cylinder 13 is, for example, a hydraulic cylinder. Note that in this embodiment, the angle of the boom 11 is expressed as shown by arrow A1, with the horizontal direction being 0 [deg], and a positive value being when pointing upward and a negative value being when pointing downward.

The bucket 12 has a cutting edge 12T and is a work tool for excavating and loading soil and other objects to be excavated. The bucket 12 is rotatably connected to the boom 11 and one end of a link 16. The other end of the link 16 is rotatably connected to one end of a bell crank 15. The center of the bell crank 15 is rotatably connected to the boom 11, and the other end is rotatably connected to one end of a bucket cylinder 14. The other end of the bucket cylinder 14 is rotatably connected to the vehicle body 2. The bucket 12 is operated by the power generated by the bucket cylinder 14. The bucket cylinder 14 is an actuator that generates power for moving the bucket 12. When the operator operates a bucket operating device (not shown), the bucket cylinder 14 expands and contracts. This causes the bucket 12 to swing. The bucket cylinder 14 is, for example, a hydraulic cylinder. The cutting edge 12T has a shape such as a crest or flat blade, and is attached to the end of the bucket 12 in a replaceable manner. In this embodiment, the angle of the bucket 12 (the angle of the direction in which the cutting edge 12T faces (the cutting edge direction)) is expressed as a value, as shown by the arrow A2, with the horizontal direction being 0 degrees, and when it faces upwards it is positive and when it faces downwards it is negative.

In this embodiment, the position of the bucket 12c in which the cutting edge 12T faces downward is called the dump position. The dump position is, for example, a position in which the excavated material in the bucket 12 can be loaded onto a transport vehicle, hopper, etc. (soil discharge position). The position of the bucket 12a in which the cutting edge 12T faces upward is called the tilt position (holding position). The tilt position is, for example, a position in which the excavated material can be held in the bucket 12 (transport position). The position of the bucket 12b in which the cutting edge 12T faces horizontally (including substantially horizontally) with the road surface RS is called the excavation position (or the traveling position during excavation). The excavation position is, for example, a position when starting to excavate an excavation target such as soil or sand or traveling toward the excavation target (or a position suitable for starting to excavate or traveling). The position of the bucket 12 in which the cutting edge 12T is in contact with the road surface RS with the boom 11 lowered is called the ground contact position. The wheel loader 1, for example, places the bucket 12 in an excavation position (or a position in which the cutting edge 12T is lower than the road surface RS from the excavation position) and travels forward to start excavating an object to be excavated that is located in front. Note that with the wheel loader 1, the excavation position can also be called the horizontal position because the cutting edge direction is substantially horizontal to the road surface RS.

The wheel loader 1 includes a power source, a PTO (Power Take Off), a hydraulic pump, a control valve, an operation device, a display input unit, and other components (not shown). The power source generates a driving force for operating the wheel loader 1. Examples of the power source include an internal combustion engine and an electric motor. The PTO transmits at least a portion of the driving force of the power source to the hydraulic pump. The PTO distributes the driving force of the power source to the traveling device 4 and the hydraulic pump. The hydraulic pump is driven by the power source and discharges hydraulic oil. At least a portion of the hydraulic oil discharged from the hydraulic pump is supplied to each of the boom cylinder 13 and the bucket cylinder 14 via a control valve. The control valve controls the flow rate and direction of the hydraulic oil supplied from the hydraulic pump to each of the boom cylinder 13 and the bucket cylinder 14. The work machine 10 operates using the hydraulic oil from the hydraulic pump.

The operating device is disposed inside the cab 3. The operating device is operated by an operator. The operator operates the operating device to adjust the travel direction and travel speed of the wheel loader 1, switch between forward and reverse, and operate the work implement 10. The operating device includes, for example, a steering wheel, a shift lever, an accelerator pedal, a brake pedal, and operating devices for operating the boom 11 and bucket 12 of the work implement 10.

The wheel loader 1 also includes sensors such as a GNSS (Global Navigation Satellite System) receiver, an engine RPM sensor, a vehicle speed sensor, a fuel consumption sensor, a work equipment load sensor, a boom angle sensor, and a bucket angle sensor. The GNSS receiver has two receivers (antennas), each of which acquires position information (latitude, longitude, and altitude). The orientation of the wheel loader 1 can be calculated based on the two pieces of position information. The engine RPM sensor detects the rotation speed of the engine (power source). The vehicle speed sensor detects the traveling speed of the wheel loader 1. The fuel consumption sensor detects the fuel consumption of the wheel loader 1. The work equipment load sensor detects, for example, the weight (load) of the excavated material held in the bucket 12. The boom angle sensor detects the angle of the boom 11. The bucket angle sensor detects the angle of the bucket 12.

(Relationship between the first movable portion and the second movable portion of the present disclosure)
As described in the section of the means for solving the problem, the working machine to be analyzed by the analysis device in the present disclosure has a first movable part and a second movable part. The first movable part is, for example, a traveling part (traveling body), and corresponds to the traveling device 4 in the present embodiment. The second movable part is, for example, a working machine, and corresponds to the working machine 10, the boom 11, and the bucket 12 in the present embodiment. In addition, when the working machine to be analyzed is, for example, a hydraulic excavator, the first movable part corresponds to the rotating part (rotating body), and the second movable part corresponds to the working machine (bucket, arm, and boom). Alternatively, the first movable part corresponds to the rotating part and the traveling part, and the second movable part corresponds to the working machine. Alternatively, the first movable part corresponds to the traveling part, and the second movable part corresponds to the rotating part and the working machine. In addition, examples of the working machine in the present disclosure include a dump truck, a bulldozer, and the like.

(Time series data)
FIG. 3 is a schematic diagram showing an example of the configuration of a time-series data file according to an embodiment of the present disclosure. The time-series data file 21 shown in FIG. 3 includes work machine information 22, operator information 23, and time-series data 24. The work machine information 22 includes, for example, information such as individual identification information, model, and manufacturing date of the wheel loader 1. The operator information 23 includes identification information of the operator of the wheel loader 1. The time-series data 24 is time-series data representing the operating state of each part of the wheel loader 1. In the example shown in FIG. 3, the time-series data 24 includes data representing date and time, position information, engine speed, gear shift, vehicle speed, fuel consumption, load, boom angle, bucket angle, etc. at a predetermined sampling period (for example, every second). Each time-series data file 21 can be created, for example, for each time from when the engine key is turned on to when it is turned off, or can be created in a form divided into multiple files for each predetermined time. For example, the gear shift and vehicle speed are examples of data representing the operating state of the first movable part, and the boom angle and bucket angle are examples of data representing the operating state of the second movable part. In this case, the time series data 24 is data that at least represents the operating state of a first movable part (traveling device 4) and a second movable part (work implement 10) of a work machine (wheel loader 1) having the first movable part and the operating state of the second movable part.

(Configuration of analysis device)
The analysis device 30 can be configured using a computer such as a personal computer or a tablet terminal. The analysis device 30 includes the following units as a functional configuration including a combination of hardware such as a computer and peripheral devices of the computer, and software executed by the computer. That is, the analysis device 30 includes an acquisition unit 31, a first operation state classification unit 32, a division unit 33, a second operation state classification unit 34, a trained machine learning model 35, a task classification unit 36, a detailed analysis unit 37, a memory unit 38, and an output unit 39 as a functional configuration.

The acquisition unit 31 acquires a time series data file 21 including the time series data 24 to be analyzed, and stores it in the storage unit 38. The time series data file 21 may be acquired automatically from the data management server 20 and stored in the storage unit 38, or may be acquired manually and stored in the storage unit 38.

The first operating state classification unit 32 classifies the operating state of the traveling device 4 (first movable part) based on the time series data 24. FIG. 4 is a schematic diagram showing an example of a behavior discrimination flag of the traveling device 4 (the undercarriage of the work machine; in this embodiment, the traveling device 4 is also simply referred to as the traveling device) according to an embodiment of the present disclosure. As shown in FIG. 4, the first operating state classification unit 32 classifies the behavior of the traveling device into four types (stop, forward, reverse, or neutral coasting), and sets a behavior discrimination flag (1, 2, 3, or 4) of the traveling device for each type. Here, neutral coasting is a state in which the gear shift is in neutral and the vehicle is traveling while lightly applying the brake, for example, to limit the vehicle speed. In the example shown in FIG. 4, when the vehicle speed value is "0" [km/h], the operating state of the traveling device 4 is classified as "stop" regardless of the value of the gear shift. Also, if the vehicle speed value is not "0", forward gears 1-4 (F1-F2) are "forward", reverse gears 1-4 (R1-R2) are "reverse", and neutral (N) is "neutral coasting".

5 and 6 are diagrams showing examples of time series data relating to behavior discrimination of a traveling device according to an embodiment of the present disclosure. In Figs. 5 and 6, the horizontal axis is time, the vertical axis (left side) is vehicle speed and gear position, and the vertical axis (right side) is behavior discrimination flag of the traveling device, and the changes in the vehicle speed, gear position, and value of the behavior discrimination flag of the traveling device are shown. Note that the vehicle speed is shown as an absolute value regardless of whether the vehicle is moving forward or backward, and the gear position is shown as +1 to +4 for forward gears 1 to 4, 0 for neutral, and -1 to -4 for reverse gears 1 to 4. In the example shown in Fig. 5, the vehicle speed becomes 0 at 10 seconds and 15 seconds, and the value of the behavior discrimination flag of the traveling device is 1 ("stop"). If this continues, the behavior will be frequently determined to be "stop" when switching between forward and reverse. Therefore, in the example shown in FIG. 6, a correction process is performed so that the vehicle is not stopped if the vehicle does not stop continuously. The vehicle speed is 0 at 10 seconds and 15 seconds, but if the time when the speed is 0 does not continue (for example, if the speed is 0 for only one sampling), the value of the behavior discrimination flag for the traveling device is not set to 1 ("stopped"), but is changed to the behavior value before the vehicle speed became 0.

The division unit 33 divides the time axis of the time series data 24 into a plurality of sections based on the classification result of the operating state of the traveling device 4 (first movable part) by the first operating state classification unit 32. FIG. 7 is a diagram showing an example of dividing the time series data into a plurality of sections according to an embodiment of the present disclosure. In the example shown in FIG. 7, the division unit 33 divides the time axis of the time series data so that a period classified as "forward", "reverse" or "stop" by the first operating state classification unit 32 becomes one section. Note that the period classified as "neutral coasting" is combined with the previous classification period and included in the previous type of period. For the classification result of the period (0 to 42 seconds) shown in FIG. 6, the division unit 33 divides the period (0 to 42 seconds) into five sections, section (k) to section (k+4), as shown in FIG. 7 (k is any natural number). Note that in this embodiment, each section is divided every second, and the last time of each section is the first time of the next section. For this reason, for example, in the period from 0 to 43 seconds, the time 43 seconds is classified as the next period (k+5) not shown. As will be described later, the period (0 to 42 seconds) is classified as loading work, and in the example shown in FIG. 7, the period from period (k) to period (k+4) is one loading work period, and period (k) corresponds to the forward movement (1), period (k+1) to period (k+2) to period (2), period (k+3) to period (k+4) to period (2).

The second operating state classification unit 34 classifies the operating state of the working machine 10 (second moving part) for each section based on the time series data 24. The second operating state classification unit 34 is a machine learning model that inputs input information including a plurality of predetermined feature amounts related to the operating state of the working machine 10 (second moving part) for each section, and outputs output information representing the result of classifying the operating state of the working machine 10 (second moving part), and classifies the operating state of the working machine 10 (second moving part) using a trained machine learning model 35 that has been trained in a supervised manner using input information labeled with the type of operating state of the working machine 10 (second moving part) as training data. FIG. 8 is a diagram showing an example of time series data related to behavior discrimination of the working machine 10 (note that in this embodiment, the working machine 10 is also simply referred to as the working machine) according to an embodiment of the present disclosure.

The trained machine learning model 35 is composed of, for example, a combination of a program that performs calculations from input to output and weighting coefficients (parameters) used in the calculations. Furthermore, the coefficients used in the calculations are optimized by machine learning so that the desired solution is output for a large amount of input data. In this embodiment, it has been confirmed that good accuracy can be obtained by using the k-nearest neighbor method as the algorithm for the machine learning model. However, there are no limitations specific to this embodiment regarding the algorithm for the machine learning model.

In Figure 8, the horizontal axis is time, the vertical axis (left side) is vehicle speed, boom angle, and bucket angle, and the vertical axis (right side) is the traveling gear behavior discrimination flag, and the change in the values of the vehicle speed, boom angle, bucket angle, and traveling gear behavior discrimination flag is shown. The values on the time axis and the vehicle speed and traveling gear behavior discrimination flag values are the same as those in Figure 7. Figures 9 to 13 are diagrams showing examples of the operating state of the work machine 10 in sections (k) to (k+4) shown in Figures 7 and 8. The horizontal axis is the boom angle, and the vertical axis is the bucket angle, and points based on the boom angle and bucket angle at the same time within the target section are plotted.

FIG. 14 is a diagram showing an example of a feature quantity according to an embodiment of the present disclosure. Note that a feature quantity is a numerical value that quantitatively represents the characteristics of the object of analysis. In this embodiment, four types of feature quantities are defined. For example, for each plot for each section shown in FIGS. 9 to 13, the first is the "area" (FV1), which can be expressed by the maximum and minimum values of the boom angle and the maximum and minimum values of the bucket angle for the purpose of grasping the position of the work machine 10. The second is the "entrance/exit coordinates" (FV2), which can be expressed by the initial and final values of the boom angle for the section and the initial and final values of the bucket angle for the section for the purpose of grasping the movement. The third is the "center of gravity coordinates" (FV3), which can be expressed by the average value of the boom angle and the average value of the bucket angle for the purpose of grasping the movement within the area. The fourth is the "number of peaks" (FV4), which can be expressed as the number of peak-like changes (such as a local maximum value or a change in the slope of the curve exceeding a predetermined threshold) in the curve connecting the plots of the boom angle and bucket angle over time, for the purpose of grasping repetition within the area. FIG. 15 is a diagram showing an example of a feature amount for section (k) according to an embodiment of the present disclosure. FIG. 16 is a diagram showing an example of a feature amount according to an embodiment of the present disclosure. FIG. 15 is a diagram in which figures explaining feature amounts FV1 to FV3 have been added to FIG. 9. FIG. 16 also shows an example including a peak (an example during warm-up operation).

17 is a schematic diagram showing an example of a behavior discrimination flag of the work machine according to an embodiment of the present disclosure. The second operating state classification unit 34 sets the value of the behavior discrimination flag of the work machine shown in FIG. 17 for the relevant section as a result of classifying the operating state of the work machine 10 (second movable part). In addition, in this embodiment, the trained machine learning model 35 inputs the four types of feature amounts shown in FIG. 14 and outputs the value of the behavior discrimination flag of the work machine shown in FIG. 17. In the example shown in FIG. 17, the behavior of the work machine is classified into 13 types. The correspondence between the behavior discrimination flag of the work machine and each of the contents of "section", "posture" and "movement" of the "bucket", and "posture" and "movement" of the "boom" shown in FIG. 17 is, for example, a rough criterion for labeling input information in supervised learning. Note that the "leveler" is in a state in which the function of automatically adjusting to a horizontal state is being used. FIGS. 18 to 21 are diagrams showing examples of feature amounts of sections (k+1) to (k+4) according to an embodiment of the present disclosure. 18 to 21 are diagrams in which figures explaining the feature quantities FV1 to FV3 are added to FIGS. 10 to 13. When the feature quantities shown in FIG. 15 are input, the trained machine learning model 35 outputs, for example, the behavior discrimination flag "20" of the work machine. The behavior discrimination flag "20" of the work machine corresponds to the behavior of the work machine in the "scooping" operation. The "scooping" operation is, for example, an operation in which the boom 11 is lowered while moving forward to scoop the excavated material into the bucket 12, and then the bucket 12 is scooped up. When the feature quantities shown in FIGS. 18, 19, 20, and 21 are input, the trained machine learning model 35 outputs, for example, the behavior discrimination flags "31", "51", "12", and "41" of the work machine.

FIG. 22 is a diagram showing an example of training data according to an embodiment of the present disclosure. The training data (data set) 351 shown in FIG. 22 includes multiple sets of data that associate the initial value, maximum value, minimum value, average value, number of peaks, and final value of the boom angle in the target section with the initial value, maximum value, minimum value, average value, number of peaks, and final value of the bucket angle and the value of the behavior determination flag of the work machine (this is the label value).

The trained machine learning model 35 of this embodiment can be understood as follows. That is, the trained machine learning model 35 is a machine learning model that inputs predetermined input information based on time series data representing at least the operating state of the first movable part and the operating state of the second movable part of a work machine having a first movable part and a second movable part, and outputs output information representing the result of classifying the operating state of the second movable part, and the input information includes a plurality of predetermined feature amounts related to the operating state of the second movable part for each section obtained by dividing the time axis of the time series data into a plurality of sections based on the classification result of the operating state of the first movable part, and is a trained machine learning model that has been machine-learned using input information labeled with the type of the operating state of the second movable part. The second movable part is, for example, a work machine having a plurality of partial movable parts, and the plurality of feature amounts can include one or more values that represent the respective movable amounts of the plurality of partial movable parts that correspond to each other on the time axis by respective coordinate values on a plurality of different coordinate axes. Here, the plurality of partial movable parts correspond to, for example, the boom 11 and the bucket 12. In this case, if the work machine includes three or more partial movable parts (e.g., a bucket, an arm, and a boom), the coordinate axes can include one or more values represented by each coordinate value on the three or more coordinate axes. Furthermore, the one or more values represented by each coordinate value can include values corresponding to the initial value, maximum value, minimum value, average value, or final value of each coordinate value for each section. Furthermore, the multiple feature amounts can include a value representing the number of repeated operations of the second movable part that occur for each section.

The work classification unit 36 classifies the work performed by the wheel loader 1 based on the classification results of the operating state of the traveling device 4 (first movable part) and the classification results of the operating state of the work machine 10 (second movable part). The work classification unit 36 classifies the work by using one or more sections as a unit. FIG. 23 is a schematic diagram showing an example of work classification according to an embodiment of the present disclosure. In the example shown in FIG. 23, the types of work are classified into nine types: "loading", "raising", "leveling", "sorting", "moving", "parking", "loading-related work", "sorting-related work", and "other". The work classification flag is information that specifies the type of work. "Loading" is, for example, the work of loading excavated materials onto the bed or hopper of a transport vehicle. "Raising" is the work of piling up excavated materials. "Leveling" is the work of leveling the work site. "Sorting" is, for example, the work of separating materials to be excavated from materials to be excavated. "Moving" is the work of moving a predetermined distance or more. "Stopping" refers to work in which the vehicle stops for a specified period of time or more. "Loading-related work" refers to, for example, loading work in which a single scooping action is usually repeated two or more times. "Sorting-related work" refers to, for example, sorting work in which more sorting actions are performed than usual. "Other" refers to work that does not fall into the above categories.

When the transition of the combination of the behavior of the traveling device and the behavior of the working machine in five consecutive sections becomes the predetermined transition shown in FIG. 24, for example, the work classification unit 36 classifies the five sections as sections for loading work. That is, the work classification unit 36 predetermines the combination of the behavior of the traveling device and the behavior of the working machine in one or more sections corresponding to each type of work, or the transition of the combination, and identifies the work in the section when the combination of the behavior of the traveling device and the behavior of the working machine in one or more sections of the time series data 24, or the transition of the combination, matches. For example, when there is a typical transition of the combination of the behavior of the traveling device and the behavior of the working machine for each section to be classified, the work classification unit 36 preferentially selects the section and determines the type. Next, for sections in which work is not classified, it searches for a section that matches the next typical combination of the behavior of the traveling device and the behavior of the working machine in one or more sections different from the work initially classified, or the transition of the combination, and determines the type if there is a match. The work classification unit 36 classifies the work in all target sections by, for example, repeating the above process. As described above, by giving priority to identifying the type of work that is easy to identify, highly accurate identification is possible. FIG. 25 is a schematic diagram for explaining an example of the work classification of sections (k) to (k+4) shown in FIG. 7 and FIG. 8. For example, the work classification unit 36 determines that the transition of the combination of the traveling device behavior discrimination flag and the work machine behavior discrimination flag for each section divided according to the classification result of the traveling device behavior matches the transition shown in FIG. 24, and classifies the work in the section (k) to (k+4) as loading work.

26 is a diagram showing an example of a classification result according to an embodiment of the present disclosure. The first operating state classification unit 32, the second operating state classification unit 34, and the task classification unit 36 generate time series data 24R in which the values of the traveling device behavior discrimination flag, the work machine behavior discrimination flag, and the task classification flag are set for each piece of time series data (each sampling) as shown in FIG. 26, for example, and store the data in the memory unit 38. Note that the classification result is not limited to the example shown in FIG. 26, and the classification result may be represented, for example, by recording the values of the traveling device behavior discrimination flag, the work machine behavior discrimination flag, and the task classification flag in association with information such as a timestamp of the time series data 24 that identifies each section.

The detailed analysis unit 37 statistically analyzes the time series data 24 for each section based on, for example, the operation of the operator. Figures 27 to 31 are diagrams showing examples of the results of detailed analysis by the detailed analysis unit 37. Figure 27 is a pie chart showing the proportion of work for a specific period, a specific work machine, and a specific operator. Figure 28 is a histogram showing the frequency of the boom angle at the time of earth discharge in loading work for a specific period, a specific work machine, and a specific operator. The detailed analysis unit 37 calculates the frequency of the boom angle for each section where the work classification result is "loading work" and the behavior discrimination flag of the work machine shown in Figure 17 is classified as "50", "51", or "52" (earth discharge). Figure 29 is a scatter diagram showing the relationship between fuel consumption [L/h] and work volume [ton/h] in loading work for a specific period, a specific work machine, and a specific operator. The scatter diagram in Figure 29 includes a regression line. FIG. 30 is a histogram showing the frequency of cycle times in loading operations for a specific period, a specific work machine, and a specific operator. FIG. 31 is a histogram showing the frequency of cycle times in loading operations for a specific period, a specific work machine, and two different operators, allowing comparison between the operators.

The output unit 39 outputs, for example, the analysis results of the detailed analysis unit 37 to a display device, etc.

(Example of analysis device operation)
32 is a flowchart showing an example of the operation of the analysis device according to the embodiment of the present disclosure. When the operator specifies an analysis target and instructs the analysis device 30 to execute the analysis, the acquisition unit 31 first acquires a file of the time series data 24 (step S1). Next, the first operating state classification unit 32 classifies the behavior of the traveling device based on the vehicle speed and the gear for the entire target time of the time series data 24 (step S2). Next, the division unit 33 divides the time axis of the time series data 24 into a plurality of sections based on the classification result of the behavior of the traveling device (step S3). Next, the second operating state classification unit 34 calculates a plurality of feature amounts for each section based on the boom angle and the bucket angle (step S4). Next, the second operating state classification unit 34 classifies the behavior of the work machine for each section using the trained machine learning model 35 (step S5). To train the machine learning model, training data is prepared, and, for example, in the processing of step S5, machine learning is performed to generate a trained machine learning model 35, and then classification processing is performed. Alternatively, the trained machine learning model 35 may be generated in advance, for example, before starting the processing shown in FIG. 32.

Next, the task classification unit 36 classifies the tasks into one or more sections based on the classification results of the traveling gear behavior and the classification results of the work machine behavior (step S6). Next, the detailed analysis unit 37 statistically analyzes the time series data, for example, by section, and the output unit 39 outputs the analysis results (step S7).

(Action and Effects)
According to the analysis device, analysis method, program, trained machine learning model, and machine learning method disclosed herein, the time series data 24 is divided into a plurality of sections based on the classification result of the traveling unit 4 (first movable part), and the operating state of the work machine 10 (second movable part) is classified for each section, so that the operating state of the work machine 10 (second movable part) can be efficiently classified. Therefore, the work of the wheel loader 1 (work machine) having the traveling unit 4 (first movable part) and the work machine 10 (second movable part) can be efficiently classified based on the time series data 24.

Although the embodiments of the present invention have been described above with reference to the drawings, the specific configuration is not limited to the above-mentioned embodiments, and includes design modifications within the scope of the gist of the present invention. In addition, part or all of the programs executed by the computer in the above-mentioned embodiments can be distributed via computer-readable recording media or communication lines.

(Additional Note)
The analysis device, analysis method, program, trained machine learning model, and machine learning method according to the present disclosure can be understood, for example, as follows.

(Appendix 1)
an acquisition unit that acquires time series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
a first operating state classification unit that classifies an operating state of the first movable part based on the time-series data;
a division unit that divides a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part;
a second operating state classification unit that classifies an operating state of the second movable part for each section based on the time series data;
an analysis device comprising: a task classification unit that classifies tasks performed by the work machine based on a classification result of the operating state of the first movable part and a classification result of the operating state of the second movable part.

(Appendix 2)
The analysis device according to claim 1, wherein the task classification unit classifies the tasks into units of one or more of the sections.

(Appendix 3)
The second operating state classification unit is
inputting input information including a plurality of predetermined feature amounts related to the operating state of the second movable part for each of the sections;
A machine learning model that outputs output information representing a result of classifying an operating state of the second movable part,
Using a trained machine learning model that has been machine-learned using the input information labeled with the type of the operating state of the second movable part,
The analysis device according to claim 1 or 2, further comprising: a second movable part configured to: classify an operating state of the second movable part.

(Appendix 4)
The first movable part is a running part,
The analysis device according to any one of appendices 1 to 3, wherein the second movable part is a work machine.

(Appendix 5)
5. The analysis device according to claim 1, further comprising a detailed analysis unit that statistically analyzes the time series data in units of the intervals.

(Appendix 6)
acquiring time series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
classifying an operating state of the first movable part based on the time series data;
Dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part;
classifying an operating state of the second movable part for each of the sections based on the time-series data;
classifying work performed by the work machine based on the classification result of the operating state of the first movable part and the classification result of the operating state of the second movable part.

(Appendix 7)
acquiring time series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
classifying an operating state of the first movable part based on the time series data;
Dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part;
classifying an operating state of the second movable part for each of the sections based on the time-series data;
and a step of classifying work performed by the work machine based on a classification result of the operating state of the first movable part and a classification result of the operating state of the second movable part.

(Appendix 8)
inputting predetermined input information based on time-series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
A machine learning model that outputs output information representing a result of classifying an operating state of the second movable part,
the input information includes a plurality of predetermined feature amounts related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part,
A trained machine learning model that has been machine-learned using the input information in which the type of operating state of the second movable part is labeled.

(Appendix 9)
The second movable part is a work machine having a plurality of partial movable parts,
The trained machine learning model of claim 8, wherein the plurality of feature amounts include one or more values that represent the respective moving amounts of the plurality of partial moving parts that correspond to each other on a time axis as coordinate values on a plurality of different coordinate axes.

(Appendix 10)
The one or more values represented by each coordinate value include a value corresponding to an initial value, a maximum value, a minimum value, an average value, or a final value of each coordinate value for each of the divisions.

(Appendix 11)
The trained machine learning model according to any one of appendices 8 to 10, wherein the plurality of feature amounts include a value representing the number of times the second movable part has been moved repeatedly for each of the sections.

(Appendix 12)
A machine learning method for a machine learning model, the machine learning method comprising: inputting predetermined input information based on time-series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having a first movable part and a second movable part; and outputting output information representing a result of classifying the operating state of the second movable part,
the input information includes a plurality of predetermined feature amounts related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part,
A machine learning method for machine learning the machine learning model using the input information labeled with the type of operating state of the second movable part.

According to each aspect of the present disclosure, the operating state of the second movable part is classified for each of a plurality of sections into which time series data is divided based on the classification result of the first movable part, so that the operating state of the second movable part can be efficiently classified. Therefore, the work of a work machine having a first movable part and a second movable part can be efficiently classified based on the time series data.

1 Wheel loader, 24 Time series data, 30 Analysis device, 31 Acquisition unit, 32 First operation state classification unit, 33 Division unit, 34 Second operation state classification unit, 35 Trained machine learning model, 36 Work classification unit, 37 Detailed analysis unit

Claims (12)

an acquisition unit that acquires time series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
a first operating state classification unit that classifies an operating state of the first movable part based on the time-series data;
a division unit that divides a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part;
a second operating state classification unit that classifies an operating state of the second movable part for each section based on the time series data;
an analysis device comprising: a task classification unit that classifies tasks performed by the work machine based on a classification result of the operating state of the first movable part and a classification result of the operating state of the second movable part. The analysis device according to claim 1 , wherein the task classification unit classifies the tasks into units of one or more of the sections. The second operating state classification unit is
inputting input information including a plurality of predetermined feature amounts related to the operating state of the second movable part for each of the sections;
A machine learning model that outputs output information representing a result of classifying an operating state of the second movable part,
Using a trained machine learning model that has been machine-learned using the input information labeled with the type of the operating state of the second movable part,
The analysis device according to claim 2 , further comprising: a detector configured to detect an operating state of the second movable part. The first movable part is a running part,
The analysis device according to claim 3 , wherein the second movable part is a work machine. The analysis device according to claim 4 , further comprising a detailed analysis unit that statistically analyzes the time series data in units of the intervals. acquiring time series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
classifying an operating state of the first movable part based on the time series data;
Dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part;
classifying an operating state of the second movable part for each of the sections based on the time-series data;
classifying work performed by the work machine based on the classification result of the operating state of the first movable part and the classification result of the operating state of the second movable part. acquiring time series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
classifying an operating state of the first movable part based on the time series data;
Dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part;
classifying an operating state of the second movable part for each of the sections based on the time-series data;
and a step of classifying work performed by the work machine based on a classification result of the operating state of the first movable part and a classification result of the operating state of the second movable part. inputting predetermined input information based on time-series data representing at least an operating state of the first movable part and an operating state of the second movable part of a work machine having a first movable part and a second movable part;
A machine learning model that outputs output information representing a result of classifying an operating state of the second movable part,
the input information includes a plurality of predetermined feature amounts related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part,
A trained machine learning model that has been machine-learned using the input information in which the type of operating state of the second movable part is labeled. The second movable part is a work machine having a plurality of partial movable parts,
The trained machine learning model according to claim 8 , wherein the plurality of feature quantities include one or more values that represent the respective movable amounts of the plurality of partial movable parts that correspond to each other on a time axis as coordinate values on a plurality of different coordinate axes. The trained machine learning model of claim 9 , wherein the one or more values represented by each coordinate value include a value corresponding to an initial value, a maximum value, a minimum value, an average value, or a final value of each coordinate value for each of the sections. The trained machine learning model according to claim 10 , wherein the plurality of feature amounts include a value representing the number of repeated movements of the second movable part that have occurred for each of the sections. A machine learning method for a machine learning model, the machine learning method comprising: inputting predetermined input information based on time-series data representing at least an operating state of a first movable part and an operating state of a second movable part of a work machine having a first movable part and a second movable part; and outputting output information representing a result of classifying the operating state of the second movable part,
the input information includes a plurality of predetermined feature amounts related to the operating state of the second movable part for each of a plurality of sections obtained by dividing a time axis of the time series data into a plurality of sections based on a classification result of the operating state of the first movable part,
A machine learning method for machine learning the machine learning model using the input information labeled with the type of operating state of the second movable part.

Images