CN118829759A - Support equipment, construction machinery and procedures - Google Patents

Abstract

The invention relates to a support device, a construction machine and a program, and provides a technique capable of more easily acquiring data related to a target shape of a construction object. A support device (150) according to one embodiment of the present invention is provided with: a topographic shape acquisition unit (301) that acquires image data relating to the shape of a constructed site in a construction target; a target shape estimating unit (302) that estimates the target shape of the construction object from the image data acquired by the topographic shape acquiring unit (301); and a target shape correction unit (304) for correcting the target shape of the construction object as a result of the estimation by the target shape estimation unit (302) on the basis of the input from the user.

Description

Support equipment, construction machinery and procedures

Technical Field

The present invention relates to a support device for a construction machine, etc.

Background Art

There is known a technology related to equipment guidance and equipment control of a construction machine, which uses data of a target shape of a construction object to support an operator's operation or automatically perform construction (for example, refer to Patent Document 1).

Prior art literature

Patent Literature

Patent Document 1: Japanese Patent Application Publication No. 2016-084663

Summary of the invention

Problems to be solved by the invention

However, in the technology related to equipment guidance and equipment control, it is necessary to prepare data of the target shape of the construction object in advance, or to manually input parameters related to the target shape by an operator, etc. Therefore, it may be difficult to introduce it in a small-scale work site, or increase the workload of the operator.

Therefore, in view of the above-mentioned problems, an object of the present invention is to provide a technology capable of more easily acquiring data related to a target shape of a construction object.

Means for solving problems

In order to achieve the above object, in one embodiment of the present invention, a support device is provided, comprising:

an acquisition unit that acquires data related to a shape of a portion of a construction object on which construction has been completed; and

The estimating unit estimates a target shape of the construction object based on the data acquired by the acquiring unit.

Furthermore, in another embodiment of the present invention, there is provided a construction machine comprising:

an acquisition unit that acquires data related to a shape of a portion of a construction object on which construction has been completed; and

The estimating unit estimates a target shape of the construction object based on the data acquired by the acquiring unit.

Furthermore, in another embodiment of the present invention, a program is provided to enable the support device to execute the following steps:

an acquisition step of acquiring data related to the shape of a portion of the construction object where construction has been completed; and

The inference step is to infer the target shape of the construction object based on the data acquired in the acquisition step.

Effects of the Invention

According to the above-described embodiment, data related to the target shape of the construction object can be acquired more easily.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side view showing an example of a shovel.

FIG. 2 is a plan view showing an example of a shovel.

FIG. 3 is a diagram showing an example of a configuration related to remote control of a shovel.

FIG. 4 is a block diagram showing an example of the hardware configuration of the shovel.

FIG. 5 is a diagram schematically showing an example of a method of estimating a target shape of a construction object in slope work.

FIG. 6 is a diagram schematically showing an example of a method of estimating a target shape of a construction object in ground excavation work.

FIG. 7 is a functional block diagram showing a first example of a functional configuration related to estimation of a target shape of a construction object.

FIG. 8 is a flowchart schematically showing a first example of processing related to estimation of a target shape of a construction object.

FIG. 9 is a diagram showing an example of display content of a display device that displays the topographic shape of the surrounding area of the shovel.

FIG. 10 is a diagram showing an example of display content of a display device that displays the estimation result of the target shape of the construction object.

FIG. 11 is a diagram showing another example of display content of a display device that displays the estimation result of the target shape of a construction object.

FIG. 12 is a diagram showing an example of an operation support system.

FIG. 13 is a diagram showing an example of the hardware configuration of the information processing device.

FIG. 14 is a functional block diagram showing a second example of a functional configuration related to estimation of a target shape of a construction object.

FIG. 15 is a flowchart schematically showing a second example of processing related to estimation of a target shape of a construction object.

DETAILED DESCRIPTION

Hereinafter, embodiments will be described with reference to the drawings.

[Overview of excavator]

First, the outline of a shovel 100 according to the present embodiment will be described with reference to FIGS. 1 to 3 .

FIG. 1 and FIG. 2 are side views showing an example of an excavator 100. FIG. 2 is a top view showing an example of an excavator 100. FIG. 3 is a diagram showing an example of a structure related to remote control of an excavator. In the following, the direction in which the attachment AT extends when the excavator 100 is viewed from above (the upper direction in FIG. 2 ) is sometimes defined as “front” to describe a direction in the excavator 100 or a direction viewed from the excavator 100.

As shown in FIGS. 1 and 2 , the shovel 100 includes a lower traveling body 1 , an upper swing body 3 , an attachment AT including a boom 4 , an arm 5 , and a bucket 6 , and a cab 10 .

The lower traveling body 1 uses crawler tracks 1C to allow the excavator 100 to travel. The crawler tracks 1C include a left crawler track 1CL and a right crawler track 1CR. The crawler tracks 1CL are hydraulically driven by a traveling hydraulic motor 1ML. Similarly, the crawler tracks 1CL are hydraulically driven by a traveling hydraulic motor 1MR. Thus, the lower traveling body 1 can travel automatically.

The upper revolving body 3 is rotatably mounted on the lower traveling body 1 via the revolving mechanism 2. For example, the upper revolving body 3 is revolved relative to the lower traveling body 1 by hydraulically driving the revolving mechanism 2 by a revolving hydraulic motor 2M.

The boom 4 is mounted at the front center of the upper revolving body 3 so as to be able to pitch about a rotation axis in the left-right direction. The arm 5 is mounted at the front end of the boom 4 so as to be able to rotate about a rotation axis in the left-right direction. The bucket 6 is mounted at the front end of the arm 5 so as to be able to rotate about a rotation axis in the left-right direction.

The bucket 6 is an example of an end attachment and is used for excavation work, for example.

The bucket 6 is mounted on the front end of the arm 5 in a manner that allows for appropriate replacement according to the work content of the excavator 100. That is, a bucket different from the bucket 6, such as a relatively large bucket, a bucket for inclined surfaces, a bucket for dredging, etc., may be mounted on the front end of the arm 5 instead of the bucket 6. In addition, a terminal attachment other than a bucket, such as a mixer, a crusher, a crusher, etc., may also be mounted on the front end of the arm 5. In addition, a preliminary attachment such as a quick coupling and a tilt rotator may also be provided between the arm 5 and the terminal attachment.

The boom 4, the arm 5 and the bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8 and a bucket cylinder 9, respectively.

The cab 10 is a control room for an operator to get on and operate the shovel 100. The cab 10 is mounted on the front left side of the upper swing body 3, for example.

For example, the shovel 100 operates driven elements such as the lower traveling body 1 (ie, a pair of left and right crawler tracks 1CL and 1CR), the upper swing body 3, the boom 4, the arm 5, and the bucket 6 according to operations of an operator riding in the cab 10.

Furthermore, the shovel 100 may be configured to be remotely operated from outside the shovel 100, in addition to or instead of being operable by an operator riding in the cab 10. When the shovel 100 is remotely operated, the interior of the cab 10 may be unmanned. The following description is based on the premise that the operator's operation includes at least one of the operation of the operating device 26 by the operator in the cab 10 and the remote operation by an external operator.

For example, as shown in FIG. 3 , the remote operation includes a method of operating the shovel 100 by inputting operation related to the actuator of the shovel 100 in the remote operation support device 300 .

The remote operation support device 300 is, for example, provided in a management center that externally manages the operation of the shovel 100. Furthermore, the remote operation support device 300 may also be a portable operation terminal, in which case the operator can remotely operate the shovel 100 while directly checking the operation status of the shovel 100 from the vicinity of the shovel 100.

The shovel 100 may also transmit an image (hereinafter referred to as a "peripheral image") representing the surrounding state including the front of the shovel 100 based on the camera image output by the camera device 40 described later to the remote operation support device 300, for example, through the communication device 60 described later. Moreover, the remote operation support device 300 may display the image (peripheral image) received from the shovel 100 on the display device. And, similarly, various information images (information screens) displayed on the output device 50 (display device) inside the cab 10 of the shovel 100 may also be displayed on the display device of the remote operation support device 300. Thus, the operator using the remote operation support device 300 can remotely operate the shovel 100 while confirming the display content of the image, information screen, etc. representing the surrounding state of the shovel 100 displayed on the display device. Furthermore, the excavator 100 can also operate the actuator according to the remote operation signal indicating the content of the remote operation received by the communication device 60 from the remote operation support device 300, thereby driving the driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5 and the bucket 6.

Furthermore, in the remote operation, for example, a method of operating the shovel 100 by inputting voice or gesture from outside to the shovel 100 by people around the shovel 100 (e.g., workers), may be included. Specifically, the shovel 100 recognizes voices made by workers around the shovel 100, gestures made by workers, etc., through a voice input device (e.g., microphone), a gesture input device (e.g., camera), etc. mounted on the shovel 100. Furthermore, the shovel 100 may also operate the actuator according to the content of the recognized voice, gesture, etc., thereby driving the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper swing body 3, the boom 4, the arm 5, and the bucket 6.

Furthermore, the excavator 100 can also automatically operate the actuators without depending on the operation contents of the operator. Thus, the excavator 100 can realize a function ("automatic operation function" or "MC (Machine Control) function") of automatically operating at least a part of the driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5 and the bucket 6.

The automatic operation function includes, for example, a function of automatically operating a driven element (actuator) other than the driven element (actuator) of the operation target according to the operation of the operation device 26 by the operator or the remote operation ("semi-automatic operation function" or "operation support type MC function"). In addition, the automatic operation function may include a function of automatically operating at least a part of the plurality of driven elements (actuators) without the operation of the operation device 26 or the remote operation by the operator ("full automatic operation function" or "full automatic type MC function"). In the shovel 100, when the full automatic operation function is effective, the interior of the cab 10 may be unmanned. In addition, the semi-automatic operation function, the full automatic operation function, etc. may include a method of automatically determining the operation content of the driven element (actuator) of the automatic operation target according to a predetermined rule. In addition, the semi-automatic operation function, the full automatic operation function, etc. may also include a method of autonomously determining the operation content of the driven element (actuator) of the automatic operation target according to the judgment result (so-called "autonomous operation function").

Furthermore, the operation of the excavator 100 can also be monitored remotely. In this case, a remote monitoring support device having the same function as the remote operation support device 300 can also be provided. The remote monitoring support device is, for example, the information processing device 200 described later. Thus, a monitor who is a user of the remote monitoring support device can monitor the operation status of the excavator 100 while confirming the peripheral image displayed on the display device of the remote monitoring support device. Furthermore, for example, when it is determined to be necessary from the perspective of safety, the monitor can intervene in the operation performed by the operator of the excavator 100 and make it stop urgently by making a prescribed input using the input device of the remote monitoring support device.

[Hardware structure of excavator]

Next, the hardware configuration of the shovel 100 will be described with reference to FIG. 4 .

FIG. 4 is a block diagram showing an example of the hardware configuration of the shovel 100 .

4 , a double line is used to indicate a path for transmitting mechanical power, a solid line is used to indicate a path for high-pressure hydraulic oil that drives a hydraulic actuator, a dotted line is used to indicate a path for transmitting pilot pressure, and a dotted line is used to indicate a path for transmitting electrical signals.

The excavator 100 includes various components such as a hydraulic drive system related to hydraulic drive of driven elements, an operating system related to operation of driven elements, a user interface system related to information exchange with a user, a communication system related to communication with the outside, and a control system related to various controls.

＜Hydraulic drive system＞

As shown in Fig. 4, the hydraulic drive system of the excavator 100 includes a hydraulic actuator HA that hydraulically drives the driven elements such as the lower traveling body 1 (left and right crawler tracks 1C), the upper swing body 3 and the attachment AT, etc., as described above. In addition, the hydraulic drive system of the excavator 100 involved in this embodiment includes an engine 11, a regulator 13, a main pump 14 and a control valve 17.

The hydraulic actuator HA includes travel hydraulic motors 1ML and 1MR, a swing hydraulic motor 2M, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and the like.

In addition, in the shovel 100, a part or all of the hydraulic actuator HA may be replaced with an electric actuator. That is, the shovel 100 may be a hybrid shovel or an electric shovel.

The engine 11 is a prime mover of the excavator 100 and a main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses diesel as fuel. The engine 11 is, for example, mounted on the rear of the upper slewing body 3. The engine 11 rotates constantly at a preset target speed under direct or indirect control by a controller 30 described later, and drives the main pump 14 and the pilot pump 15.

In addition, other prime movers (for example, electric motors) or the like may be mounted on the shovel 100 instead of the engine 11 or in addition to the engine 11 .

The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter referred to as "tilt angle") according to a control command from the controller 30.

The main pump 14 supplies hydraulic oil to the control valve 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, at the rear of the upper slewing body 3, similarly to the engine 11. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, the stroke length of the piston is adjusted by adjusting the deflection angle of the swash plate by the regulator 13 under the control of the controller 30 to control the discharge flow rate or discharge pressure.

The control valve 17 drives the hydraulic actuator HA according to the operator's operation of the operating device 26, the content of the remote operation, or the operation instruction corresponding to the automatic operation function. The control valve 17 is mounted, for example, on the central part of the upper slewing body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic pipeline, and the working oil supplied from the main pump 14 is selectively supplied to each hydraulic actuator according to the operator's operation or the operation instruction corresponding to the automatic operation function. Specifically, the control valve 17 includes a plurality of control valves (also called "direction switching valves"), which control the flow rate and flow direction of the working oil supplied from the main pump 14 to each hydraulic actuator HA.

＜Operating system＞

As shown in FIG. 4 , the operating system of the shovel 100 includes a pilot pump 15 , an operating device 26 , a hydraulic control valve 31 , a shuttle valve 32 , and a hydraulic control valve 33 .

The pilot pump 15 supplies pilot pressure to various hydraulic devices via the pilot line 25. Like the engine 11, the pilot pump 15 is mounted, for example, at the rear of the upper swing body 3. The pilot pump 15 is, for example, a fixed displacement hydraulic pump and is driven by the engine 11 as described above.

In addition, the pilot pump 15 may be omitted. In this case, the relatively high-pressure hydraulic oil discharged from the main pump 14 can be supplied to various hydraulic devices as pilot pressure after the relatively low-pressure hydraulic oil is reduced in pressure by a predetermined pressure reducing valve.

The operating device 26 is provided near the operator's seat of the cab 10 and is used by the operator to operate various driven elements. Specifically, the operating device 26 is used by the operator to operate the hydraulic actuator HA that drives each driven element, and as a result, the operator can operate the driven element that is the driving target of the hydraulic actuator HA. The operating device 26 includes a pedal device and a joystick device for operating each driven element (hydraulic actuator HA).

For example, as shown in FIG. 4 , the operating device 26 is a hydraulic pilot type. Specifically, the operating device 26 uses the working oil supplied from the pilot pump 15 through the pilot line 25 and the pilot line 25A branched therefrom to output the pilot pressure corresponding to the operation content to the pilot line 27A on the secondary side. The pilot line 27A is connected to an inlet port of the reciprocating valve 32, and is connected to the control valve 17 via the pilot line 27 connected to the outlet port of the reciprocating valve 32. Thus, the pilot pressure corresponding to the operation content related to various driven elements (hydraulic actuator HA) in the operating device 26 can be input to the control valve 17 via the reciprocating valve 32. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26 by the operator or the like.

Furthermore, the operating device 26 may be an electric type. In this case, the pilot line 27A, the reciprocating valve 32, and the hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electrical signal corresponding to the operation content (hereinafter referred to as "operation signal"), and the operation signal is input to the controller 30. Then, the controller 30 outputs a control instruction corresponding to the content of the operation signal to the hydraulic control valve 31, that is, a control signal corresponding to the operation content of the operating device 26. As a result, a pilot pressure corresponding to the operation content of the operating device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26.

Furthermore, the control valve (directional switching valve) built into the control valve 17 for driving each hydraulic actuator HA may be of electromagnetic solenoid type. In this case, the operation signal output from the operating device 26 may be directly input to the control valve 17, that is, the electromagnetic solenoid type control valve.

Furthermore, as described above, a part or all of the hydraulic actuator HA may be replaced with an electric actuator. In this case, the controller 30 may output a control command corresponding to the operation content of the operating device 26 and the content of the remote operation specified by the remote operation signal to the electric actuator or a driver driving the electric actuator. Furthermore, when the shovel 100 is remotely operated, the operating device 26 may be omitted.

The hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) of the operating object of the operating device 26 and for each driving direction of the driven element (hydraulic actuator HA) (for example, the lifting direction and the lowering direction of the boom 4). That is, two hydraulic control valves 31 are provided for each passive hydraulic actuator HA. The hydraulic control valve 31 can be provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and is configured to be able to change its flow area (that is, the cross-sectional area through which the working oil can flow). Thus, the hydraulic control valve 31 can output a prescribed pilot pressure to the pilot line 27B on the secondary side using the working oil of the pilot pump 15 supplied through the pilot line 25B. Therefore, the hydraulic control valve 31 can indirectly act on the control valve 17 with a prescribed pilot pressure corresponding to a control signal from the controller 30 through the reciprocating valve 32 between the pilot line 27B and the pilot line 27. Thus, the controller 30 supplies the pilot pressure according to the operation command corresponding to the automatic operation function from the hydraulic control valve 31 to the control valve 17 , thereby realizing the operation of the shovel 100 based on the automatic operation function.

Furthermore, the controller 30 can control the hydraulic control valve 31, for example, and realize the remote operation of the shovel 100. Specifically, the controller 30 outputs a control signal corresponding to the content of the remote operation specified by the remote operation signal received from the remote operation support device 300 to the hydraulic control valve 31 through the communication device 60. As a result, the controller 30 supplies the pilot pressure corresponding to the remote operation content from the hydraulic control valve 31 to the control valve 17, thereby realizing the operation of the shovel 100 based on the remote operation of the operator.

When the operating device 26 is an electric type, the controller 30 supplies a pilot pressure corresponding to the operation content (operation signal) of the operating device 26 directly from the hydraulic control valve 31 to the control valve 17, thereby realizing the operation of the shovel 100 based on the operator's operation.

The shuttle valve 32 has two inlet ports and one outlet port, and outputs the working oil having the higher pilot pressure of the pilot pressure input to the two inlet ports to the outlet port. The shuttle valve 32 is provided for each driven element (hydraulic actuator HA) of the operating object of the operating device 26 and for each driving direction of the driven element (hydraulic actuator HA). One of the two inlet ports of the shuttle valve 32 is connected to the pilot line 27A on the secondary side of the operating device 26 (specifically, the above-mentioned joystick device and pedal device included in the operating device 26), and the other is connected to the pilot line 27B on the secondary side of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of the corresponding control valve of the control valve 17 through the pilot line 27. The corresponding control valve is a control valve that drives the operating object of the above-mentioned joystick device and pedal device connected to one inlet port of the shuttle valve 32, that is, the hydraulic actuator. Therefore, these shuttle valves 32 can respectively allow the higher pilot pressure of the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure of the pilot line 27B on the secondary side of the hydraulic control valve 31 to act on the pilot port of the corresponding control valve. That is, the controller 30 can control the corresponding control valve independently of the operator's operation of the operating device 26 by outputting a pilot pressure higher than the pilot pressure on the secondary side of the operating device 26 from the hydraulic control valve 31. As a result, the controller 30 can control the operation of the driven elements (lower traveling body 1, upper swing body 3, attachment AT) regardless of the operator's operation state of the operating device 26, thereby realizing a remote operation function and an automatic operation function.

The hydraulic control valve 33 is provided in the pilot line 27A connecting the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured to be able to change its flow path area, for example. The hydraulic control valve 33 operates according to the control signal input from the controller 30. Thus, when the operating device 26 is operated by the operator, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. And, for example, even when the operating device 26 is being operated, the controller 30 can reduce the pilot pressure output from the operating device 26 to be lower than the pilot pressure output from the hydraulic control valve 31. Therefore, the controller 30 can reliably act on the pilot port of the control valve in the control valve 17 by controlling the hydraulic control valve 31 and the hydraulic control valve 33, for example, regardless of the operation content of the operating device 26. Therefore, the controller 30 can more accurately realize the remote control function and the automatic operation function of the shovel 100 by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31, for example.

＜User Interface System＞

As shown in FIG. 4 , the user interface system of the excavator 100 includes an operating device 26 , an output device 50 , and an input device 52 .

The output device 50 outputs various information to the user of the shovel 100 (for example, the operator of the cabin 10 , an external remote operator), people around the shovel 100 (for example, workers, drivers of construction vehicles), and the like.

For example, the output device 50 includes a lighting device and a display device 50A (refer to FIG. 5 ) that output various information in a visual manner. The lighting device is, for example, a warning light (indicator light), etc. The display device 50A is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, etc. For example, as shown in FIG. 2 , the lighting device and the display device 50A can be disposed inside the cockpit 10, and various information can be output to an operator inside the cockpit 10 in a visual manner. Furthermore, the lighting device and the display device 50A can also be disposed on the side of the upper rotating body 3, etc., and various information can be output to workers around the excavator 100 in a visual manner.

Furthermore, for example, the output device 50 includes a sound output device that outputs various information in an auditory manner. The sound output device includes, for example, a buzzer, a speaker, etc. The sound output device may be provided, for example, at least one of the inside and outside of the cab 10, and outputs various information in an auditory manner to the operator in the cab 10 and the people (staff, etc.) around the excavator 100.

Furthermore, for example, the output device 50 may also include a device that outputs various information by a tactile method such as vibration of the operator's seat.

The input device 52 receives various inputs from the user of the shovel 100, and inputs signals corresponding to the received inputs to the controller 30. The input device 52 is provided, for example, inside the cab 10, and receives inputs from the operator or the like inside the cab 10. Furthermore, the input device 52 may be provided, for example, on the side of the upper swing body 3, and receive inputs from the staff or the like around the shovel 100.

For example, the input device 52 includes an operation input device that receives input from the user based on mechanical operation. The operation input device may include a touch panel installed on the display device, a touch pad provided around the display device, a button switch, a joystick, a switch key, a knob switch provided on the operation device 26 (joystick device), etc.

Furthermore, for example, the input device 52 may include a voice input device for receiving a user's voice input. The voice input device may include, for example, a microphone.

Furthermore, for example, the input device 52 may include a gesture input device that receives a gesture input by the user. The gesture input device includes, for example, a camera that captures a state of a gesture performed by the user.

Furthermore, for example, the input device 52 may include a biometric input device for receiving a user's biometric input, such as a fingerprint, an iris, or other biometric information input of the user.

＜Communication system＞

As shown in FIG. 4 , the communication system of the shovel 100 according to the present embodiment includes a communication device 60 .

The communication device 60 is connected to an external communication line and communicates with a device that is separately installed from the excavator 100. In addition to the device located outside the excavator 100, the device that is separately installed from the excavator 100 may also include a portable terminal device (mobile terminal) brought into the cockpit 10 by the user of the excavator 100. The communication device 60 may include, for example, a mobile communication module that complies with standards such as 4G ( ^4th Generation) and 5G ( ^5th Generation). In addition, the communication device 60 may include, for example, a satellite communication module. In addition, the communication device 60 may also include, for example, a WiFi communication module, a Bluetooth (registered trademark) communication module, etc. In addition, the communication device 60 may also include a plurality of communication devices corresponding to the communication line of the connection object.

For example, the communication device 60 communicates with external devices such as the information processing device 200 and the remote operation support device 300 in the construction site through a local communication line built at the construction site. The local communication line is, for example, a mobile communication line based on a local 5G (so-called local 5G) built at the construction site, or a local network (LAN: Local Area Network) based on WiFi6.

Furthermore, for example, the communication device 60 communicates with the information processing device 200 and the remote operation support device 300 located outside the construction site via a wide area communication line including the construction site, i.e., a wide area network (WAN). The wide area network includes, for example, a wide area mobile communication network, a satellite communication network, and the Internet.

＜Control system＞

As shown in Fig. 4 , the control system of the shovel 100 includes a controller 30. Furthermore, the control system of the shovel 100 according to the present embodiment includes an operation pressure sensor 29, an imaging device 40, and sensors S1 to S5.

The controller 30 performs various controls related to the shovel 100 .

The functions of the controller 30 may be implemented by any hardware or any combination of hardware and software. For example, as shown in FIG4 , the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C and an interface device 30D connected via a bus B1.

The auxiliary storage device 30A is a nonvolatile storage member that stores programs to be installed and necessary files, data, etc. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, or the like.

For example, when there is an instruction to start a program, the memory device 30B loads the program in the auxiliary storage device 30A so that the CPU 30C can read it. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

For example, the CPU 30C executes a program loaded into the memory device 30B, and realizes various functions of the controller 30 according to the commands of the program.

The interface device 30D functions as, for example, a communication interface for connecting to a communication line inside the shovel 100. The interface device 30D may include a plurality of different types of communication interfaces depending on the type of communication line to be connected.

Furthermore, the interface device 30D functions as an external interface for reading data from a recording medium and writing data to a recording medium. The recording medium is, for example, a dedicated tool connected to a connector provided inside the cab 10 via a detachable cable. Furthermore, the recording medium may be, for example, a general-purpose recording medium such as an SD memory card or a USB (Universal Serial Bus) memory. Thus, a program for realizing various functions of the controller 30 may be provided, for example, by a portable recording medium and installed in the auxiliary storage device 30A of the controller 30. Furthermore, the program may be downloaded from another computer outside the shovel 100 via the communication device 60 and installed in the auxiliary storage device 30A.

In addition, part of the functions of the controller 30 may be realized by other controllers (control devices). In other words, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot operating device 26, that is, detects the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. The detection signal of the pilot pressure corresponding to the operating state related to each driven element (hydraulic actuator HA) in the operating device 26 detected by the operating pressure sensor 29 is input to the controller 30.

When the operating device 26 is an electric type, the operating pressure sensor 29 can be omitted. This is because the controller 30 can grasp the operating state of each driven element of the operating device 26 based on the operating signal input from the operating device 26.

The camera device 40 acquires an image of the vicinity of the shovel 100. Furthermore, the camera device 40 may acquire (generate) three-dimensional data (hereinafter referred to as "three-dimensional data of the object") indicating the position and shape of an object in the vicinity of the shovel 100 within the camera range (angle of view) based on the acquired image and data related to the distance described later. The three-dimensional data of the object in the vicinity of the shovel 100 is, for example, data indicating coordinate information of a point group on the surface of the object, distance image data, and the like.

For example, as shown in FIG2 , the camera device 40 includes a camera 40F for photographing the front of the upper swing body 3, a camera 40B for photographing the rear of the upper swing body 3, a camera 40L for photographing the left of the upper swing body 3, and a camera 40R for photographing the right of the upper swing body 3. Thus, the camera device 40 can photograph the entire circumference of the shovel 100, that is, a range of 360 degrees in the angular direction when looking down at the shovel 100. In addition, the operator can visually recognize the peripheral images such as the camera images of the cameras 40B, 40L, and 40R and the processed images generated based on the camera images through the output device 50 (display device 50A) and the remote operation display device, and confirm the status of the left, right, and rear of the upper swing body 3. Furthermore, the operator can visually recognize the image captured by the camera 40F and the peripheral images such as the processed image generated based on the image captured by the camera 40F through the remote operation display device, and can remotely operate the shovel 100 while confirming the operation of the attachment AT including the bucket 6. Hereinafter, the cameras 40F, 40B, 40L, and 40R may be collectively or individually referred to as "camera 40X".

The camera 40X is, for example, a monocular camera and may be capable of acquiring data related to distance (depth) in addition to two-dimensional images, such as a stereo camera or a TOF (Time Of Flight) camera (hereinafter collectively referred to as a “3D camera”).

The output data of the camera device 40 (camera 40X) (for example, image data, three-dimensional data of objects around the shovel 100, etc.) is input to the controller 30 through a one-to-one communication line or an in-vehicle network. Thus, for example, the controller 30 can monitor objects around the shovel 100 based on the output data of the camera 40X. Also, for example, the controller 30 can determine the surrounding environment of the shovel 100 based on the output data of the camera 40X. Also, for example, the controller 30 can determine the posture state of the attachment AT shown in the camera image based on the output data of the camera 40X (camera 40F). Also, for example, the controller 30 can determine the posture state of the body (upper swing body 3) of the shovel 100 based on the objects around the shovel 100 based on the output data of the camera 40X.

In addition, part of the cameras 40F, 40B, 40L, and 40R may be omitted. For example, when the excavator 100 is not remotely operated, the cameras 40F and 40L may be omitted. This is because it is relatively easy to confirm the status of the front and left side of the excavator 100 from the operator of the cab 10. Furthermore, a distance sensor may be provided on the upper rotating body 3 instead of the camera device 40 (camera 40X) or on the basis of it. The distance sensor is, for example, installed on the upper part of the upper rotating body 3 to obtain data related to the distance and direction of objects around the excavator 100. Furthermore, the distance sensor may also obtain (generate) three-dimensional data (for example, data of coordinate information of a point group) of objects around the excavator 100 within the sensing range based on the acquired data. The distance sensor is, for example, LIDAR (Light Detection and Ranging). Furthermore, for example, the distance sensor may be a millimeter wave radar, an ultrasonic sensor, an infrared sensor, or the like.

The sensor S1 is mounted on the boom 4, and detects the posture angle (hereinafter referred to as the "boom angle") of the base end corresponding to the connection portion of the boom 4 connected to the upper rotating body 3 around the rotation axis. The sensor S1 includes, for example, a rotary potentiometer, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), etc. The following description may be the same for sensor S2 and sensor S4. In addition, the sensor S1 may also include a cylinder sensor that detects the telescopic position of the boom cylinder 7. The following description may be the same for sensor S2. The detection signal of the boom angle of the sensor S1 is input to the controller 30. Thus, the controller 30 can grasp the posture state of the boom 4.

The sensor S2 is mounted on the boom 5, and detects the posture angle (hereinafter referred to as "the boom angle") of the base end of the boom 5 corresponding to the connection portion connected to the boom 4 around the rotation axis. The detection signal of the boom angle detected by the sensor S2 is input to the controller 30. Thus, the controller 30 can grasp the posture state of the boom 5.

The sensor S3 is mounted on the bucket 6 and detects the posture angle (hereinafter referred to as "arm angle") of the base end of the bucket 6 corresponding to the connection portion connected to the boom 5 around the rotation axis. The detection signal of the boom angle detected by the sensor S3 is input to the controller 30. Thus, the controller 30 can grasp the posture state of the bucket 6.

The sensor S4 detects the tilt state of the machine body (for example, the upper rotating body 3) relative to a predetermined reference plane (for example, a horizontal plane). The sensor S4 is, for example, mounted on the upper rotating body 3, and detects the tilt angles (hereinafter referred to as "front-rear tilt angle" and "left-right tilt angle") of the excavator 100 (that is, the upper rotating body 3) around two axes in the front-rear direction and the left-right direction. The detection signal corresponding to the tilt angle (front-rear tilt angle and left-right tilt angle) detected by the sensor S4 is input to the controller 30. Thus, the controller 30 can grasp the tilt state of the machine body (the upper rotating body 3).

The sensor S5 is mounted on the upper rotating body 3, and outputs detection information related to the rotation state of the upper rotating body 3. The sensor S5 detects, for example, the rotation angular velocity and the rotation angle of the upper rotating body 3. The sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, etc. The detection information related to the rotation state detected by the sensor S5 is input to the controller 30. Thus, the controller 30 can grasp the rotation state such as the rotation angle of the upper rotating body 3.

In addition, when the sensor S4 includes a gyro sensor, a 6-axis sensor, an IMU, etc. that can detect the angular velocity around three axes, the rotation state (for example, the rotation angular velocity) of the upper rotating body 3 can be detected based on the detection signal of the sensor S4. In this case, the sensor S5 can be omitted. Furthermore, when the posture state of the upper rotating body 3, the attachment AT, etc. can be grasped based on the output of the camera device 40 and the distance sensor, at least a part of the sensors S1 to S5 can also be omitted.

[Overview of object shape estimation function]

Next, an overview of a function related to estimating a target shape of a construction object of the shovel 100 (hereinafter referred to as a “target shape estimating function” for convenience) will be described with reference to FIGS. 5 and 6 in addition to FIGS. 1 to 4 .

Fig. 5 is a diagram schematically showing an example of a method for estimating a target shape of a construction object in slope work. Fig. 6 is a diagram schematically showing an example of a method for estimating a target shape in ground excavation work.

The construction object is an object formed on the ground or the like by construction work performed by the shovel 100. Examples of construction work include slope work, ground excavation work, ground leveling work, etc. Examples of the construction object include a horizontal surface, a slope, a trench, a pile of earth, and the like.

The target shape of the construction object refers to the target shape of the construction object that is ultimately expected to be achieved through the operation of the shovel 100. The target shape of the construction object is, for example, a target construction surface formed as a plane. In addition, the target shape of the construction object may also be a target construction surface formed by a curved surface with a predetermined curvature.

In the present embodiment, the excavator 100 (controller 30) infers the target shape of the construction object based on the shape of a portion of the construction object that has been constructed by the excavator operation performed by a skilled person. The inferred target shape of the construction object may be the target shape of the entire construction object, or may be the target shape of a partial area of the construction object that is larger than the area of the portion that has been constructed. The excavator used by the skilled person in the construction work of the construction object may be the excavator 100, or may be another excavator different from the excavator 100. The skilled person is, for example, an operator who has relatively long experience in operating an excavator and has a relatively high experience value related to the operation of the excavator.

For example, as shown in Fig. 5, a partially constructed area 501 is formed in the slope of the construction object during the slope work. As described above, the area 501 is constructed in advance based on the shovel operation performed by a skilled person.

First, the shovel 100 (controller 30 ) uses the imaging device 40 to capture the shape of the region 501 , and acquires shape data of the region 501 based on the output of the imaging device 40 .

Next, the shovel 100 (controller 30 ) estimates a target construction surface 502 corresponding to a target shape of the construction object based on the shape data of the area 501 .

For example, the controller 30 extends the planar shape of the area 501 in the width direction of the slope or copies and arranges it in the width direction, thereby inferring the target construction surface 502 and acquiring data related to the target construction surface 502. Hereinafter, this inference method is sometimes referred to as the "first inference method" of the target shape of the work object.

In addition, the controller 30 may also use a learned model obtained by machine learning using a training data set based on a combination of shape data of a portion of the slope of the construction object where construction has been completed and shape data of a target construction surface of the slope of the construction object. In this case, the controller 30 applies the shape data of the region 501 as input data to the learned model, thereby being able to infer the target construction surface 502 based on the output data of the learned model and obtain data of the target construction surface 502.

And, for example, as shown in Fig. 6, a partially constructed trench 601 is formed in the ground excavation work. As described above, the trench 601 is constructed in advance by the shovel operation performed by a skilled person.

First, the shovel 100 (controller 30 ) uses the imaging device 40 to capture the shape of the groove 601 , and acquires shape data of the groove 601 based on the output of the imaging device 40 .

Next, the shovel 100 (controller 30) estimates a target construction surface 602 corresponding to the target shape of the groove to be constructed based on the shape data of the groove 601. The target construction surface 602 includes a target construction surface corresponding to the side surfaces at both ends of the groove in the width direction, a target construction surface corresponding to the side surfaces at both ends of the groove in the length direction, and a target construction surface corresponding to the bottom of the groove.

For example, the controller 30 estimates the target construction surface 602 by the first estimation method. Specifically, the controller 30 may estimate the target construction surface 602 corresponding to the side surfaces at both ends of the width direction of the groove 601 by extending the shape of the side surfaces at both ends of the width direction of the groove 601 in the length direction of the groove of the construction object, or by copying and arranging in the length direction. Furthermore, the controller 30 may estimate the target construction surface 602 corresponding to the bottom of the groove by extending the shape of the bottom of the groove 601 in the length direction of the groove of the construction object, or by copying and arranging in the length direction. Furthermore, the controller 30 may estimate the target construction surface 602 corresponding to the side surfaces at both ends of the length direction of the groove by directly using the shape of the side surfaces at both ends of the length direction of the groove 601, or by shifting the position in the length direction of the groove of the construction object.

Furthermore, the controller 30 may also infer the target construction surface 602 by the second inference method. Specifically, the controller 30 may also use a learned model obtained by machine learning using a training data set, the training data set being based on a combination of data on the shape of a portion of the groove of the construction object where construction has been completed and data on the shape of the target construction surface of the groove of the construction object. In this case, the controller 30 uses the shape data of the groove 601 as input data and applies the learned model, thereby being able to infer the target construction surface 602 based on the output data of the learned model and obtain data on the target construction surface 602.

Thus, in the present embodiment, the controller 30 can infer the target shape of the construction object based on the shape data of a part of the construction object that has been constructed by the excavator operation performed by a skilled person, and obtain data related to the target shape. Thus, the controller 30 can more easily obtain data related to the target shape of the construction object. Therefore, there is no need to prepare data related to the target shape in advance, so for example, the workload such as the user manually inputting parameters related to the target shape will not be increased. Moreover, for example, even in a small-scale construction site where it is impossible to ensure the system and funds for preparing data related to the target shape in advance, it is possible to obtain data related to the target shape, and adopt technologies related to equipment guidance and equipment control to improve work efficiency.

For example, the target shape inference function is applied to the excavator 100 operated by the operator of the cab 10 or the operator of the remote operation. In this case, the excavator 100 can support the operator's operation using the equipment guidance function and the operation support type MC function based on the data related to the target shape inferred by the target shape inference function. In addition, the target shape inference function can also be applied to the excavator 100 that operates by the fully automatic operation function (fully automatic MC function). In this case, the excavator 100 can automatically perform construction by the fully automatic operation function based on the data related to the target shape inferred by the target shape inference function so that the construction object becomes the target shape.

[First example of functional structure related to estimation of target shape of construction object]

Next, a first example of a functional configuration related to estimation of a target shape of a construction object will be described with reference to FIG. 7 in addition to FIGS. 1 to 6 .

FIG. 7 is a functional block diagram showing a first example of a functional configuration related to estimation of a target shape of a construction object.

As shown in FIG. 7 , the shovel 100 includes a support device 150 .

The support device 150 supports the operation of the shovel 100. The support device 150 includes a controller 30, an imaging device 40, a display device 50A, an input device 52, and a communication device 60.

The controller 30 includes, as functional units, a terrain shape acquisition unit 301 , a target shape estimation unit 302 , a display processing unit 303 , a target shape correction unit 304 , a target shape data storage unit 305 , and a work support control unit 306 .

The terrain shape acquisition unit 301 acquires data related to the terrain shape of a place where a construction object is formed around the excavator 100, including a part of the construction object where construction has been completed, based on the output of the camera device 40 and the distance sensor. The data related to the terrain shape includes, for example, both a part of the construction object where construction has been completed and a part that has not been constructed. The data related to the terrain shape may be, for example, image data or three-dimensional data.

The target shape estimation unit 302 estimates the target shape of the construction object using the first estimation method based on the data acquired by the terrain shape acquisition unit 301 , and acquires data related to the target shape of the construction object.

The display processing unit 303 causes the display device 50A to display the target shape of the construction object as the estimation result of the target shape estimation unit 302. Furthermore, when the shovel 100 is remotely operated or remotely monitored, the display processing unit 303 may transmit the image data of the target shape of the construction object as the estimation result of the target shape estimation unit 302 to the remote operation support device 300 or the remote monitoring support device through the communication device 60. Thus, the display processing unit 303 can cause the display device of the remote operation support device 300 or the remote operation support device to display the target shape of the construction object as the estimation result of the target shape estimation unit 302. Thus, the user (operator) can visually confirm the data related to the target shape of the construction object.

Furthermore, the display processing unit 303 may display the target shape of the construction object as the inference result of the target shape inference unit 302 and the current terrain shape based on the output of the camera device 40 on the display device 50A in a manner that allows comparison (see FIGS. 10 and 11 ). Similarly, the display processing unit 303 may also send an image that allows comparison of the target shape of the construction object as the inference result of the target shape inference unit 302 and the current terrain shape based on the output of the camera device 40 to the remote operation support device 300 and the remote monitoring support device. Thus, the user (operator) can judge whether an appropriate target shape is generated (inferred) by comparing the current terrain shape with the target shape of the construction object.

The target shape correction unit 304 corrects the data related to the target shape of the construction object as the inference result of the target shape inference unit 302 according to the specified input from the user (operator). The input from the user is received by the input device 52. In addition, when the excavator 100 is remotely operated or remotely monitored, the input from the user is received from the remote operation support device 300 and the remote monitoring support device through the communication device 60. For example, the target shape correction unit 304 corrects the data related to the target shape of the construction object according to a total of 6 degrees of freedom, namely, translational movement in the directions along the X-axis, the Y-axis, and the Z-axis, and rotational movement around the X-axis, the Y-axis, and the Z-axis, in a three-dimensional orthogonal coordinate system. In addition, the target shape correction unit 304 can also correct the data related to the target shape of the construction object according to a total of 9 degrees of freedom, namely, enlargement or reduction in the directions along the X-axis, the Y-axis, and the Z-axis, respectively.

The target shape data storage unit 305 stores data related to the target shape of the construction object. Specifically, data related to the target shape of the construction object as the estimation result of the target shape estimation unit 302 or data related to the target shape of the construction object as the correction result of the target shape correction unit 304 may be stored.

The work support control unit 306 performs control for supporting work related to construction of the construction object based on data related to the target shape of the construction object.

For example, the work support control unit 306 performs control related to equipment guidance based on data related to the target shape of the construction object. Specifically, the work support control unit 306 can notify the operator of information such as the distance between the working part such as the tip and back of the bucket 6 and the target shape through the output device 50 such as the display device 50A and the communication device 60.

Furthermore, the work support control unit 306 can perform control related to equipment control (automatic operation function) based on data related to the target shape of the construction object. Specifically, the work support control unit 306 can control the hydraulic control valve 31 to operate the attachment AT, etc., so as to support the operator's operation, or move the working parts such as the tip and back of the bucket 6 on a track along the target shape without relying on the operator's operation.

[First example of processing related to estimation of target shape of construction object]

Next, a first example of processing related to estimation of a target shape of a construction object will be described with reference to FIGS. 8 to 11 in addition to FIGS. 1 to 7 .

FIG8 is a flowchart schematically showing a first example of processing related to the estimation of the target shape of the construction object. FIG9 is a diagram showing an example (screen 900) of the display content of the display device 50A showing the terrain shape around the excavator 100. FIG10 and FIG11 are diagrams showing an example and another example of the display content of the display device 50A that displays the estimation result of the target shape of the construction object. Specifically, FIG10 is a diagram showing the display content of the display device 50A when the target shape of the construction object as the estimation result is appropriate, and FIG11 is a diagram showing the display content of the display device 50A when the target shape of the construction object as the estimation result is inappropriate.

For example, when a predetermined input is received from a user (an operator, a supervisor, etc.) via the input device 52 or the communication device 60, the flowchart of FIG. 8 is started.

As shown in FIG. 8 , in step S102 (an example of an acquisition step), the terrain shape acquisition unit 301 acquires image data of the vicinity of the shovel 100 or three-dimensional data representing the state of the vicinity of the shovel 100 based on the image data based on the output of the imaging device 40 .

If the process of step S102 is completed, the controller 30 proceeds to step S104 .

In step S104 , the display processing unit 303 causes the display device 50A, the remote operation support device 300 , etc. to display an image showing the topographic shape around the shovel 100 forming the construction site based on the data acquired in step S102 (see FIG. 9 ).

For example, as shown in FIG. 9 , images 901 to 903 are displayed on a screen 900 of the display device 50A.

Image 901 is an image showing the terrain shape around the shovel 100 at a location where a construction object (a slope in this example) is formed. Specifically, in image 901, the terrain shape around the shovel 100 is represented by three-dimensional mesh data. Thus, the user can recognize the current terrain shape of the location where the construction object is formed.

Image 902 is an operation icon for executing the process of estimating the target shape of the construction object. Thus, the user can operate the icon of image 902 through the input device 52 or the like to execute the process of estimating the target shape of the construction object (see step S106 of FIG. 8 ).

Image 903 is an operation icon for ending the process of the flowchart of Fig. 8 and returning to a predetermined screen. The same applies to images 1004 and 1104 described later. Thus, the process of the flowchart of Fig. 8 can be stopped midway.

Returning to FIG. 8 , if the process of step S104 is completed, the controller 30 proceeds to step S106 .

In step S106, the controller 30 determines whether an operation for causing it to perform a process of inferring the target shape of the construction object is performed through the input device 52 or the like. If an operation for causing it to perform a process of inferring the target shape of the construction object is performed, the controller 30 proceeds to step S108, and otherwise (for example, if an operation of image 903 is performed, or if no operation is performed for a certain period of time), the controller 30 ends the process of this flowchart.

In addition, the processing of steps S104 and S106 may be omitted.

In step S108 (an example of an estimating step), the target shape estimating unit 302 estimates the target shape of the construction object using the first estimating method based on the data acquired in step S102, and acquires data related to the target shape of the construction object.

If the process of step S108 is completed, the controller 30 proceeds to step S110 .

In step S110 , the target shape of the construction object as the estimation result of step S108 is displayed on the display device 50A, the remote operation support device 300 (display device), or the like (see FIGS. 10 and 11 ).

For example, as shown in Fig. 10 , a screen 1000 includes images 1001 to 1005. Similarly, as shown in Fig. 11 , a screen 1100 includes images 1101 to 1105.

Images 1001 and 1101 are images showing the shape of the terrain around the shovel 100 forming a site of a construction target (a slope in this example), similarly to the image 901 in FIG. 9 .

Images 1002 and 1102 are images showing target shapes (target construction surfaces) of construction objects, respectively.

Images 1003 and 1103 are operation icons for determining a target shape (target construction surface) of a construction object as current display content.

The images 1004 and 1104 are operation icons for shifting the target shape of the construction object from the current display content to a screen for correction.

In the screen 1000, the target shape of the construction object (image 1002) is appropriately estimated with respect to the current terrain shape (image 1001) around the excavator 100. Therefore, the user can determine the target shape of the construction object according to the data of the current display content by operating the icon of the image 1003 through the input device 52, the remote operation support device 300, etc.

On the other hand, in the screen 1100, the target shape of the construction object (image 1102) is inferred in a form that does not match the current shape of the surroundings of the shovel 100. Therefore, the user can operate the operation icon of the image 1104 through the input device 52, the remote operation support device 300, etc., to correct the target shape of the construction object.

Returning to FIG. 8 , if the process of step S110 is completed, the controller 30 proceeds to step S112 .

In step S112, the controller 30 determines whether an operation for correcting the target shape of the construction object has been performed through the input device 52, the remote operation support device 300 (input device), etc. The controller 30 proceeds to step S114 if an operation for correcting the target shape of the construction object has been performed, and proceeds to step S116 if an operation for determining the target shape of the construction object has been performed.

In step S114 , the target shape correction unit 304 corrects the data related to the target shape of the construction object based on the input from the user via the input device 52 or the remote operation support device 300 .

If the process of step S114 is completed, the controller 30 proceeds to step S116 .

In step S116 , the controller 30 determines the target shape, and stores data related to the target shape in the target shape data storage unit 305 .

When the process of step S116 is completed, the controller 30 ends the process of this flowchart.

As described above, in this example, the support device 150 can estimate the target shape of the construction object based on the shape of a portion constructed by the shovel operation performed by a skilled worker, and can more easily acquire data related to the target shape of the construction object.

Furthermore, in this example, the support device 150 can correct the target shape of the construction object as the estimation result based on the input from the user.

[Overview of Operation Support System]

Next, the operation support system SYS will be described with reference to FIG. 12 in addition to FIGS. 1 to 3 .

12 , the shovel 100 may be a component of the operation support system SYS. Specifically, the operation support system SYS includes the shovel 100 and an information processing device 200 .

The operation support system SYS cooperates with the shovel 100 using the information processing device 200 , and performs support related to the operation of the shovel 100 .

The shovel 100 included in the operation support system SYS may be one or a plurality of shovels.

In the operation support system SYS, the shovel 100 is a construction machine that is a target of support related to operation.

The information processing device 200 communicates with the shovel 100 to cooperate with each other and provide support related to the operation of the shovel 100 .

The information processing device 200 is, for example, a server or a management terminal device in a management office located in the construction site of the excavator 100 or a management center located at a location different from the construction site of the excavator 100 for managing the operating status of the excavator 100. The management terminal device may be, for example, a fixed terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile terminal) such as a tablet terminal, a smart phone, or a laptop PC. In the latter case, workers at the construction site, supervisors who supervise the work, managers who manage the construction site, etc. can carry the portable information processing device 200 and move around the construction site. And, in the latter case, the operator can, for example, bring the portable information processing device 200 into the cockpit of the excavator 100.

The information processing device 200 acquires data related to the operating state from the shovel 100, for example. Thus, the information processing device 200 can grasp the operating state of the shovel 100 and monitor whether the shovel 100 has abnormalities, etc. Furthermore, the information processing device 200 can display the data related to the operating state of the shovel 100 on the display device 208 described later for the user to confirm.

The information processing device 200 transmits various data such as programs and reference data used in processing by the controller 30 of the shovel 100 to the shovel 100. The shovel 100 can thereby perform various processes related to the operation of the shovel 100 using various data downloaded from the information processing device 200.

[Hardware structure of the operation support system]

Next, the hardware configuration of the operation support system SYS will be described with reference to FIG. 13 in addition to FIG. 1 to FIG. 4 and FIG. 12 .

In addition, since the hardware configuration of the shovel 100 is the same as that of FIG. 4 , the description thereof will be omitted.

FIG. 5 is a block diagram showing an example of the hardware configuration of the information processing device 200 .

The functions of the information processing device 200 are implemented by arbitrary hardware or arbitrary combination of hardware and software, etc. For example, as shown in FIG5 , the information processing device 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a high-speed computing device 205, a communication interface 206, an input device 207, and a display device 208 connected via a bus B2.

The external interface 201 functions as an interface for reading data from the recording medium 201A and writing data to the recording medium 201A. The recording medium 201A includes, for example, a floppy disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, and a USB memory. Thus, the information processing device 200 can read various data used in processing through the recording medium 201A, store it in the auxiliary storage device 202, or install programs that realize various functions.

Furthermore, the information processing device 200 may acquire various data and programs used in processing from an external device via the communication interface 206 .

The auxiliary storage device 202 stores various installed programs, and stores files and data required for various processes, etc. The auxiliary storage device 202 includes, for example, a HDD (Hard Disc Drive), an SSD (Solid State Disc), a flash memory, and the like.

When there is an instruction to start the program, the memory device 203 reads and stores the program from the auxiliary storage device 202. The memory device 203 includes, for example, a DRAM (Dynamic Random Access Memory) or an SRAM.

The CPU 204 executes various programs loaded from the auxiliary storage device 202 to the memory device 203 , and implements various functions related to the information processing device 200 according to the programs.

The high-speed computing device 205 performs computing at a relatively high speed in conjunction with the CPU 204. The high-speed computing device 205 includes, for example, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or a FPGA (Field-Programmable Gate Array).

In addition, the high-speed computing device 205 may be omitted depending on the required computing processing speed.

The communication interface 206 is used as an interface for connecting to external devices in a communicative manner. Thus, the information processing device 200 can communicate with external devices such as the shovel 100 through the communication interface 206. The communication interface 206 may have multiple communication interfaces depending on the communication method between the connected devices.

The input device 207 receives various inputs from the user.

The input device 207 includes, for example, an operation input device that receives mechanical operation input from the user. The operation input device includes, for example, a button, a switch key, a joystick, etc. In addition, the operation input device includes, for example, a touch panel installed on the display device 208, a touch pad provided separately from the display device 208, etc.

Furthermore, the input device 207 includes, for example, a voice input device capable of receiving voice input from the user. The voice input device includes, for example, a microphone capable of collecting the user's voice.

Furthermore, the input device 207 includes, for example, a gesture input device capable of receiving gesture input from the user. The gesture input device includes, for example, a camera capable of capturing the gesture state of the user.

Furthermore, the input device 207 includes, for example, a biometric input device capable of receiving a biometric input from a user. The biometric input device includes, for example, a camera capable of acquiring image data containing information related to a user's fingerprint or iris.

The display device 208 displays information screens and operation screens to the user. For example, the display device 208 includes the above-mentioned remote operation display device. The display device 208 is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

In addition, the remote operation support device 300 can be implemented by any hardware or any combination of hardware and software, and can adopt the same hardware structure as the information processing device 200. For example, the remote operation support device 300 is the same as the information processing device 200 (Figure 6), and is composed of a computer including a CPU, a memory device, an auxiliary storage device, an interface device, an input device and a display device as the center. The memory device is, for example, SRAM, DRAM, etc. The auxiliary storage device is, for example, HDD, SSD, EEPROM, flash memory, etc. The interface device includes an external interface for connecting to an external recording medium and a communication interface for communicating with the outside such as the excavator 100. The input device includes, for example, a joystick-type operation input device. As a result, the operator can use the operation input device to perform operation input related to the actuator of the excavator 100, and the remote operation support device 300 can use the communication interface to send a signal corresponding to the operation input to the excavator 100. Therefore, the operator can remotely operate the excavator 100 using the remote operation support device.

[Second example of functional structure related to estimation of target shape of construction object]

Next, a second example of the functional configuration related to estimation of the target shape of the construction object will be described with reference to FIG. 14 in addition to FIG. 1 to FIG. 4 , FIG. 12 , and FIG. 13 .

The following description will focus on portions that are different from the first example ( FIG. 7 ), and descriptions of portions that are the same as or correspond to those of the first example may be simplified or omitted.

FIG. 14 is a functional block diagram showing a second example of a functional configuration related to estimation of a target shape of a construction object.

As shown in FIG. 14 , the shovel 100 includes a support device 150 .

Similar to the case of the first example ( FIG. 7 ), the support device 150 includes a controller 30 , an imaging device 40 , a display device 50A, an input device 52 , and a communication device 60 .

The controller 30 includes, as functional units, a terrain shape acquisition unit 301 , a target shape estimation unit 302 , a display processing unit 303 , a target shape correction unit 304 , a target shape data storage unit 305 , a work support control unit 306 , a learned model storage unit 307 , and a transmission unit 308 .

The learned model LM is stored in the learned model storage unit 307 .

The learned model LM is used to estimate the target shape of the construction object by the second estimation method. The learned model LM is distributed from the information processing device 200 .

The target shape inference unit 302 infers the target shape of the construction object using the second inference method based on the data acquired by the terrain shape acquisition unit 301, and acquires data related to the target shape of the construction object. Specifically, the target shape inference unit 302 uses the data acquired by the terrain shape acquisition unit 301 as input data and applies the learned model LM, so that the target shape of the construction object can be inferred based on the output data of the learned model LM.

The transmitting unit 308 transmits, to the information processing device 200 , combined data of the data acquired by the terrain shape acquiring unit 301 and data related to the target shape of the construction object corresponding to the data and stored (registered) in the target shape data storage unit 305 .

The information processing device 200 includes a training data storage unit 2001 , a machine learning unit 2002 , a learned model storage unit 2003 , and a distribution unit 2004 as functional units related to estimation of a target shape of a construction object.

The training data storage unit 2001 stores (registers) training data for generating the learned model LM. In addition, the training data storage unit 2001 may also include training data for relearning or additionally learning the learned model LM to update it. The training data is a combination of data related to the terrain shape including a part of the construction object where construction has been completed and data related to the target shape of the construction object.

For example, the latter training data includes a combination of data acquired by the terrain shape acquisition unit 301 and received from the shovel 100 (transmitting unit 308 ) and data corresponding to the data and related to the target shape of the construction object finally determined.

The machine learning unit 2002 uses the training data set of the training data storage unit 2001 to perform machine learning on a predetermined model, thereby generating a learned model LM.

Furthermore, the machine learning unit 2002 may re-learn or additionally learn the learned model LM using the training data set of the training data storage unit 2001 to update the learned model LM.

The learned model LM generated by the machine learning unit 2002 is stored (registered) in the learned model storage unit 2003. The learned model LM in the learned model storage unit 2003 may be updated by the machine learning unit 2002 performing re-learning or additional learning.

The distribution unit 2004 distributes the learned model LM to the shovel 100 .

[Second Example of Processing Related to Estimation of Target Shape of Construction Object]

Next, a second example of the process related to the estimation of the target shape of the construction object will be described with reference to FIG. 15 in addition to FIGS. 1 to 4 and 12 to 14 .

The following description will focus on portions that are different from the first example ( FIG. 8 ), and descriptions of portions that are the same as or correspond to those of the first example may be simplified or omitted.

As shown in FIG. 14 , the processing of steps S202 and S204 is the same as that of steps S102 and S104 in FIG. 8 , and thus the description thereof is omitted.

If the process of step S204 is completed, the controller 30 proceeds to step S206 .

In step S206, the controller 30 determines whether an operation for executing a process of estimating the target shape of the construction object has been performed through the input device 52, etc., similarly to the case of step S106 in Fig. 8 . If an operation for executing a process of estimating the target shape of the construction object has been performed, the controller 30 proceeds to step S208, and otherwise terminates the process of this flowchart.

In step S208 , the target shape estimating unit 302 estimates the target shape of the construction object using the second estimating method based on the data acquired in step S102 , and acquires data related to the target shape of the construction object.

If the process of step S208 is completed, the controller 30 proceeds to step S210 .

The processing of steps S210 to S216 is the same as that of steps S110 to S116 in FIG. 8 , and thus the description thereof is omitted.

If the process of step S216 is completed, the controller 30 proceeds to step S218 .

In step S218, the sending unit 308 sends the data acquired in step S202 and the data of the target shape of the construction object determined in step S216 to the information processing device 200. Specifically, the sending unit 308 sends to the information processing device 200 a combination of data related to the current topographic shape of the site forming the construction object including a portion of the construction completed in the construction object and data related to the target shape of the construction object finally determined.

When the process of step S216 is completed, the controller 30 ends the process of this flowchart.

In addition, the process of step S216 may be omitted, and the transmission unit 308 may collectively transmit the data of multiple times in this flowchart to the information processing device 200 as a batch process.

Thus, in this example, the support device 150 can estimate the target shape of the construction object using the learned model LM based on data on the current topographic shape of the site forming the construction object including a portion of the construction object where construction has been completed.

Furthermore, in this example, the information processing device 200 can update the learned model LM using training data that is a combination of data related to the terrain shape including a portion of the construction object that has been completed and data related to the target shape of the construction object. Thus, the correction result of the target shape of the construction object by the target shape correction unit 304 can be reflected in the learned model LM. Therefore, the inference accuracy of the target shape of the construction object by the learned model LM can be improved.

[Other embodiments]

Next, other embodiments will be described.

The above-mentioned embodiments may be combined in their contents as appropriate, or may be deformed or modified.

For example, in the above-mentioned embodiment, when a construction operation is performed on a part of a construction object by the shovel 100, the processing related to the construction operation as a preprocessing and the processing related to the estimation of the target shape (refer to FIG. 8 and FIG. 15) can be executed as a series of processing. At this time, in the processing related to the construction operation as a preprocessing or in the processing parallel to the processing, the trajectory of the working part such as the blade tip and the back of the bucket 6 during the construction operation can be recorded in the auxiliary storage device 30A or the like. Thus, the terrain shape acquisition unit 301 can acquire data related to the terrain shape including a part of the construction completed part of the construction object based on the trajectory data of the working part. This is because the trajectory data of the working part can be considered to represent the relative terrain shape observed from the shovel 100 at the construction completed part. And at this time, the target shape estimation unit 302 can estimate the target shape of the construction object based on the trajectory data of the working part. And at this time, the target shape correction unit 304 can correct (correct) the data related to the target shape of the construction object as the estimation result of the target shape estimation unit 302 based on the trajectory data of the working part.

Furthermore, in the above-described embodiment or its modified example, part or all of the functions of the support device 150 may be transferred to the remote operation support device 300 .

Furthermore, in the above-described embodiment or its modified example, part or all of the functions of the support device 150 may be transferred to the information processing device 200 .

Furthermore, in the above-described embodiment and its modified examples, part or all of the functional units of the information processing device 200 related to the target shape estimation function may be transferred to the shovel 100 .

[effect]

Next, the operation of the support device according to this embodiment will be described.

In this embodiment, the support device includes: an acquisition unit that acquires data related to the shape of a portion of the construction object that has been completed; and an inference unit that infers the target shape of the construction object based on the data acquired by the acquisition unit. The support device is, for example, the support device 150. The acquisition unit is, for example, the terrain shape acquisition unit 301. The inference unit is, for example, the target shape inference unit 302.

Thus, data related to the target shape of the construction object can be acquired by only performing construction on a part of the construction object. Therefore, the support device can more easily acquire data related to the target shape of the construction object.

Furthermore, in this embodiment, the inference unit can infer the target shape of the construction object using a learned model obtained by machine learning using training data based on a combination of the shape of a portion of the construction object where construction has been completed and the target shape of the construction object. For example, the learned model is a learned model LM.

Thereby, the support device can acquire data related to the target shape of the construction object based on the shape of a portion of the construction object on which construction has been completed.

Furthermore, in the present embodiment, the inference unit may infer the target shape of the construction object by copying or extending the shape of a portion of the construction object where construction has been completed in a direction corresponding to the construction object.

Thereby, the support device can acquire data related to the target shape of the construction object based on the shape of a portion of the construction object on which construction has been completed.

Furthermore, in this embodiment, the inference unit can infer the target shape of the construction object based on data related to the trajectory of the working part when the construction machine performs construction work on a part of the construction object that has been completed. The construction machine is, for example, an excavator 100, and the working part is the tip and back of the bucket 6 of the excavator 100.

Thereby, the support device can acquire data related to the target shape of the construction object based on the shape of a portion of the construction object on which construction has been completed.

Furthermore, in the present embodiment, the support device may include a display unit that displays the target shape of the construction object as the estimation result of the estimation unit. The display unit is, for example, the display device 50A.

As a result, the support device can allow the user to visually confirm the target shape of the construction object as the estimation result, thereby improving the convenience of the user.

Furthermore, in the present embodiment, the support device may include a correction unit that corrects the target shape of the construction object as the estimation result of the estimation unit based on the input from the user. The correction unit is, for example, the target shape correction unit 304 .

Thus, when the target shape of the construction object is inappropriate as the estimation result, the user can operate the support device to correct the target shape of the construction object.

In this embodiment, the support device may include a correction unit and an updating unit for updating the learned model using a combination of the data acquired by the acquisition unit and the target shape data of the construction object as the correction result of the correction unit as training data. The updating unit is, for example, the machine learning unit 2002.

This allows the correction result of the correction unit to be reflected in the learned model. Therefore, the estimation accuracy of the target shape of the construction object using the learned model can be improved.

Furthermore, in the present embodiment, the construction machine may include the above-mentioned assist device.

As a result, the construction machine can more easily estimate and acquire the target shape of the construction object using the support device.

Although the embodiments have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes can be made within the scope of the gist described in the claims.

Finally, this application claims priority based on Japanese patent application No. 2022-058416 filed on March 31, 2022, and the entire contents of the Japanese patent application are incorporated herein by reference.

Explanation of symbols

1-lower walking body, 1C, 1CL, 1CR-crawler, 1ML, 1MR-travel hydraulic motor, 2-slewing mechanism, 2M-slewing hydraulic motor, 3-upper slewing body, 4-boom, 5-arm, 6-bucket, 7-boom cylinder, 8-arm cylinder, 9-bucket cylinder, 10-cabin, 11-engine, 13-regulator, 14-main pump, 15-pilot pump, 17-control valve, 26-operating device, 29-operating pressure sensor, 30-controller, 40-camera device, 50-output device, 50A-display device, 52-input device, 60-communication device, 100-excavator, 150-support device, 200-information processing device, 206-communication interface, 207-input device, 208-display device, 300-remote operation support device, 301-terrain shape acquisition unit, 302-target shape inference unit, 303-display processing unit, 304-target shape correction unit, 305-target shape data storage unit, 306-operation support control unit, 307-learned model storage unit, 308-sending unit, 2001-training data storage unit, 2002-machine learning unit, 2003-learned model storage unit, 2004-distribution unit, AT-attachment, HA-hydraulic actuator, LM-learned model, S1~S5-sensors, SYS-operation support system.

Claims (9)

1. A support device comprising: an acquisition unit that acquires data related to a shape of a portion of a construction object on which construction has been completed; and The estimating unit estimates a target shape of the construction object based on the data acquired by the acquiring unit. 2. The support device according to claim 1, wherein: The inference unit infers a target shape of the construction object using a learned model obtained by machine learning using training data based on a combination of a shape of a partially constructed portion of the construction object and the target shape of the construction object. 3. The support device according to claim 1, wherein: The estimating unit estimates a target shape of the construction object by copying or extending the shape of a portion of the construction object where construction has been completed in a direction corresponding to the construction object. 4. The support device according to any one of claims 1 to 3, wherein: The estimation unit estimates a target shape of the construction object based on data related to a trajectory of a work site when the construction machine performs construction work on a partially constructed site of the construction object. 5. The support device according to any one of claims 1 to 4, wherein: A display unit is provided for displaying a target shape of a construction object as an estimation result of the estimation unit. 6. The support device according to any one of claims 1 to 5, wherein: A correction unit is provided for correcting a target shape of a construction object as a result of the estimation by the estimation unit based on an input from a user. 7. The support device according to claim 2, comprising: a correction unit that corrects a target shape of the construction object as an inference result of the inference unit based on an input from a user; and The updating unit updates the learned model using a combination of the data acquired by the acquiring unit and the data of the target shape of the construction object as a correction result of the correcting unit as training data. 8. A construction machine comprising: an acquisition unit that acquires data related to a shape of a portion of a construction object on which construction has been completed; and The estimating unit estimates a target shape of the construction object based on the data acquired by the acquiring unit. 9. A program that causes a support device to execute the following steps: an acquisition step of acquiring data related to the shape of a portion of the construction object where construction has been completed; and The inference step is to infer the target shape of the construction object based on the data acquired in the acquisition step.