WO2024262357A1 - Crane device - Google Patents

Abstract

[Problem] To provide a crane device that makes it possible to eliminate blind spots regardless of the form or specification of a device or makes it possible to eliminate blind spots according to a request from an operator. [Solution] A crane device 12 comprises a safety device 17. The safety device 17 includes a wide angle camera 18, a display 84 which is provided on a cabin, and a controller 70. The wide angle camera 18 is provided on the upper surface of a boom and captures images in a range including a region around the tip end part of the boom, a region facing the upper surface of the boom, side regions of the boom, and a rear region of the cabin. The controller 70 determines the type of a monitoring image that is displayed on the display 84, on the basis of first designation information which is stored in a memory 72 according to the specifications and second designation information which indicates the designation of a screen by an operator. The controller 70 then generates monitoring image data that indicates the monitoring image of the determined type on the basis of captured image data which has been output from the wide angle camera 18.

Description

Crane Equipment

The present invention relates to a crane device equipped with a boom that can be rotated, raised, and extended.

Patent Document 1 discloses a crane apparatus that includes a rotating base and a boom mounted on the rotating base. The boom is capable of raising and lowering and extending. A sensor is provided at the tip of the boom as a safety device. The sensor detects obstacles approaching the tip of the boom.

WO2022-092093 publication

Incidentally, crane equipment comes in a variety of types, including mobile types such as rough terrain cranes and all-terrain cranes, as well as fixed types such as tower cranes. Even for the same type of crane equipment, the size and working radius vary depending on the specifications (model), and the blind spot of the operator while working changes. For example, with one model the blind spot is the area on the top of the boom, while with another model the blind spot is the area around the tip of the boom, and in some cases both of these areas are blind spots. In this way, the blind spot that you want to monitor on the boom varies depending on the model of crane equipment.

To ensure work safety, measures are required that allow the operator to monitor the above areas and eliminate blind spots. However, it is difficult to use the above sensors to monitor blind spots, which vary depending on the specifications of the crane equipment. Furthermore, the above areas (blind spots that need to be eliminated) also differ depending on the wishes of the operator who actually operates the boom.

The present invention was made against this background, and its purpose is to provide a crane device that can eliminate blind spots regardless of the type or specifications of the device, or that can eliminate blind spots according to the operator's wishes.

(1) The crane apparatus according to the present invention comprises a rotating base, a cabin mounted on the rotating base and equipped with a control device, a boom that is rotatably mounted on the rotating base and is extendable and retractable, and a safety device having a wide-angle camera attached to the top surface of the boom in the lowered position, a controller, and a display. The controller executes a generation process for generating surveillance image data based on captured image data generated by the wide-angle camera, and a display process for displaying a surveillance image indicated by the surveillance image data on the display.

The wide-angle camera is attached to the top surface of the boom, and can capture images of the area around the tip of the boom, the side area of the boom, the area facing the top surface of the boom, and the rear area around the base end of the boom all at once. The controller generates surveillance image data from the captured image data for an area indicated by settings prestored in memory or an area specified by the operator (generation process), inputs the surveillance image data to the display, and displays the surveillance image on the display (display process). Therefore, the safety device can display a surveillance image of a blind spot on the display, whether the area is an area determined by the model or specifications or an area desired by the operator, whether it is an area around the tip of the boom, a side area of the boom, an area facing the top surface of the boom, or a rear area around the base end of the boom. Therefore, the crane apparatus according to the present invention can eliminate blind spots regardless of the model or specifications, or can eliminate blind spots according to the operator's request.

(2) The controller may further execute a designated information acquisition process for acquiring designated information. The generation process is one or more processes indicated by the designated information among a first process of correcting the captured image data based on a first correction standard corresponding to the tip of the boom shown in the captured image indicated by the captured image data to generate boom-tip monitoring image data, a second process of correcting the captured image data based on a predetermined second correction standard in an area shown in the captured image indicated by the captured image data that is to the side of the boom, a third process of correcting the captured image data based on a predetermined third correction standard in an area shown in the captured image indicated by the captured image data that faces the top of the boom, and a fourth process of correcting the captured image data based on a fourth correction standard corresponding to the base of the boom shown in the captured image indicated by the captured image data to generate rear monitoring image data.

The designated information is, for example, stored in advance in a memory possessed by the controller, or is input to the controller by the operator. The controller reads the designated information from the memory, or receives and acquires the designated information input by the operator. The controller executes one of the first to fourth processes, whichever process is specified by the designated information, as a generation process to generate monitoring image data. Therefore, an appropriate monitoring image according to the type and specifications of the crane apparatus is displayed on the display, or an appropriate monitoring image specified by the operator is displayed on the display.

(3) The crane apparatus according to the present invention may further include a boom hoisting angle sensor that outputs a first detection value indicating the boom hoisting angle. The fourth process includes a process of correcting the captured image data based on the fourth correction standard, and a conversion process of converting a viewpoint based on the boom hoisting angle indicated by the first detection value. The rear monitoring image indicated by the rear monitoring image data is an overhead image showing the rear of the cabin.

The position and height of the wide-angle camera relative to the rotating platform changes as the boom is raised or lowered. A conversion process is carried out to create an overhead image based on the boom hoisting angle, which is a parameter that determines the position and height of the wide-angle camera. Therefore, regardless of the boom hoisting angle, a rear monitoring image that overlooks the rear of the cabin from a fixed position is always displayed on the display.

(4) The crane apparatus according to the present invention may further include a boom hoisting angle sensor that outputs a first detection value indicating the boom hoisting angle. The second process further includes a process of correcting the captured image data based on the second correction standard, and a rotation process of rotating a boom side monitoring image, which is a monitoring image indicated by the boom side monitoring image data, at a rotation angle corresponding to the boom hoisting angle indicated by the first detection value.

The boom side monitoring image tilts according to the boom hoisting angle. The controller rotates the boom side monitoring image according to the boom hoisting angle, correcting the tilt of the boom side monitoring image. Therefore, the crane apparatus according to the present invention makes it easier for the operator to recognize obstacles compared to a case where the tilt of the boom side monitoring image is not corrected.

(5) The crane apparatus according to the present invention may further include a boom hoisting angle sensor that outputs a first detection value indicating the boom hoisting angle. The controller executes the third process based on the designation information indicating execution of the third process, and executes a first enlargement process to enlarge the boom top side monitoring image indicated by the boom top side monitoring image data generated in the third process before executing the display process based on the determination that the boom is being raised based on the change over time in the boom hoisting angle indicated by the first detection value.

When the boom is being raised, there is a risk of contact between the top of the boom and an obstacle. When the boom is being raised, the controller enlarges and displays the boom top monitoring image. Therefore, the crane apparatus according to the present invention makes it easier for the operator to recognize obstacles compared to when the boom top monitoring image is always displayed on the display at a constant size.

(6) The crane apparatus according to the present invention may further include a rotation angle sensor that outputs a second detection value indicating the rotation angle of the rotation table. The boom side monitoring image data includes boom left side monitoring image data indicating the left side of the boom and boom right side monitoring image data indicating the right side of the boom. The controller executes the second process based on the designation information indicating the execution of the second process, executes a second enlargement process of enlarging the boom left side monitoring image indicated by the boom left side monitoring image data generated in the second process before executing the display process based on the determination of a left turn based on the second detection value, and executes a third enlargement process of enlarging the boom right side monitoring image indicated by the boom right side monitoring image data generated in the second process before executing the display process based on the determination of a right turn based on the second detection value.

When the boom is rotating, there is a risk of contact between the side of the boom and an obstacle. The controller enlarges and displays the left-side boom monitoring image when the boom is rotating left, and enlarges and displays the right-side boom monitoring image when the boom is rotating right. Therefore, the crane apparatus according to the present invention makes it easier for the operator to recognize obstacles compared to when the boom side monitoring image is always displayed on the display at a constant size.

(7) The boom may have multiple cylindrical bodies including a base boom. The wide-angle camera is provided on the base boom.

The wide-angle camera is mounted on the base boom, whose position does not change when the boom is extended or retracted, making it easier to establish a wired connection between the wide-angle camera and the controller compared to when the wide-angle camera is mounted on another cylinder whose position changes when the boom is extended or retracted.

(8) The crane apparatus according to the present invention may further include a boom length sensor that outputs a third detection value indicating the extension length of the boom. The boom has a plurality of cylindrical bodies including a base boom. The wide-angle camera is provided on the base boom. The first process includes a process of correcting the captured image data based on the first correction standard, and a fourth enlargement process of enlarging the captured image data by a magnification factor corresponding to the third detection value.

When a wide-angle camera is mounted on the base boom, the tip of the boom moves away from the wide-angle camera as the boom extends. The boom tip monitoring image, which is the monitoring image shown by the boom tip monitoring image data, is enlarged at a magnification rate that corresponds to the extended length of the boom, making it easier for the operator to recognize obstacles approaching the tip of the boom.

(9) The angle of view of the wide-angle camera may be 180 degrees or more.

A wide-angle camera with a viewing angle of 180 degrees or more will be used.

The crane device of the present invention can eliminate blind spots regardless of the type or specifications of the device, or can eliminate blind spots according to the operator's wishes.

FIG. 1 is a functional block diagram of a crane apparatus 12 . FIG. 2 is a schematic side view of the crane vehicle 10 with the boom 22 raised. FIG. 3 is a schematic side view of the crane vehicle 10 in a lowered position with the boom 22 lowered. FIG. 4 is a diagram showing the control device 29 inside the cabin 13. FIG. 5 is a flowchart of the monitoring image display process. Figure 6 (A) is a schematic diagram of an image captured by wide-angle camera 18, Figure 6 (B) is a diagram showing a rear monitoring image, Figure 6 (C) is a diagram showing a monitoring image of the left side of the boom, Figure 6 (D) is a diagram showing a monitoring image of the right side of the boom, Figure 6 (E) is a diagram showing a monitoring image of the boom tip, and Figure 6 (F) is a diagram showing a monitoring image of the top side of the boom.

Below, one embodiment of the present invention will be described. Note that the embodiment described below is merely one example of the present invention, and it goes without saying that the embodiment can be modified as appropriate without changing the gist of the present invention. For example, the order of execution of each process described below can be modified as appropriate without changing the gist of the present invention. Alternatively, some of the processes described below can be omitted as appropriate without changing the gist of the present invention.

FIG. 1 is a functional block diagram of the crane device 12. FIG. 2 is a side view of the crane vehicle 10 with the boom 22 in an upright position. FIG. 3 is a schematic side view of the crane vehicle 10 with the boom 22 in a lowered position.

As shown in Figures 2 and 3, the crane vehicle 10 is a rough terrain crane. However, the crane vehicle 10 may also be an all-terrain crane.

The crane vehicle 10 includes a running body 11, an outrigger device 14 attached to the running body 11, and a crane device 12 and a cabin 13 mounted on the running body 11. Note that the outrigger device 14 is not shown in FIG. 3.

The outrigger device 14 has multiple jacks 24 that contact the ground to stabilize the position of the crane vehicle 10. The crane device 12 is used with the jacks 24 extended and the position of the crane vehicle 10 stabilized.

The crane device 12 has a rotating base 21, a boom 22, a winch 23, a group of sensors 26 (see Figure 1), a group of hydraulic actuators 27 (see Figure 1), a hydraulic supply device 28 (see Figure 1), a control device 29 (see Figure 1), a hook block 32, and a safety device 17 (see Figure 1).

The swivel base 21 is rotatably supported on the running body 11. The rotation axis of the swivel base 21 is aligned vertically (up and down) and is located approximately in the center of the running body 11.

The boom 22 is supported on the rotating base 21 so that it can be raised and lowered. In other words, the boom 22 can be raised and lowered and can be rotated.

The boom 22 includes a base boom 42, an intermediate boom 43, and a top boom 44. The base boom 42, the intermediate boom 43, and the top boom 44 are each rectangular tubular. The intermediate boom 43 is located inside the base boom 42, and the top boom 44 is located inside the intermediate boom 43. In other words, the base boom 42, the intermediate boom 43, and the top boom 44 are nested and form a so-called telescopic structure. The boom 22 expands and contracts as the intermediate boom 43 slides relative to the base boom 42 and the top boom 44 slides relative to the intermediate boom 43. The base boom 42, the intermediate boom 43, and the top boom 44 correspond to the "cylinder" described in the claims. The boom 22 may be composed of two cylinders, or may be composed of four or more cylinders.

The base boom 42 is located on the outermost side. Therefore, the outer surface of the base boom 42 is always exposed to the outside. The rectangular cylindrical base boom 42 has an upper surface 45 which becomes the upper surface when the boom 22 is in a horizontally laid down position (see FIG. 3). A wide-angle camera 18, which will be described later, is provided on the upper surface 45.

The boom 22 is supported on the swivel base 21 by the base boom 42 so that it can be raised and lowered. In other words, the base boom 42 does not move (slide) when the boom 22 is extended or retracted. In other words, the position of the wide-angle camera 18 attached to the base boom 42 does not change when the boom 22 is extended or retracted.

The winch 23 is attached to the base end of the boom 22 or to the rotating base 21. The winch 23 has a drum 56 around which the wire rope 41 (hereinafter referred to as "wire 41") is wound, and a sheave 57 around which the wire 41 is looped. When the winch 23 is driven, the wire 41 is wound onto the drum 56 or unwound from the drum 56. Note that the winch 23, wire 41, sheave 57, drum 56, etc. are not shown in FIG. 3.

The wire 41 is looped around a sheave 57 provided at the base end of the boom 22, a sheave 58 provided at the tip of the boom 22, and a pulley device (not shown). The pulley device has a number of first sheaves (not shown) provided at the tip of the boom 22, and a number of second sheaves (not shown) provided on the hook block 32. Note that the hook block 32 is not shown in FIG. 3.

The hook block 32 comprises a hook body 33, a plurality of the second sheaves rotatably held within the hook body 33, and a hook 34 provided on the underside of the hook body 33.

Two winches 23 may be provided on the crane device 12. A hook block 32, which is a so-called main hook, is connected to a wire 41 of one of the winches 23, and a sub-hook (not shown) is connected to a wire 41 of the other winch 23.

As shown in FIG. 1, the hydraulic actuator group 27 includes a swing motor 51, a hoisting cylinder 52, a telescopic cylinder 53, a hydraulic motor 54, and a jack 24.

The slewing motor 51 is a hydraulic motor that rotates via hydraulic oil supplied from the hydraulic supply unit 28, and rotates the slewing table 21. The hoisting cylinder 52 is a hydraulic cylinder that expands and contracts via hydraulic oil supplied from the hydraulic supply unit 28, and raises and lowers the boom 22. The telescopic cylinder 53 is a hydraulic cylinder that expands and contracts via hydraulic oil supplied from the hydraulic supply unit 28, and extends and lowers the boom 22. The hydraulic motor 54 rotates via hydraulic oil supplied from the hydraulic supply unit 28, and rotates the drum 56 of the winch 23.

The hydraulic supply device 28 includes a hydraulic pump driven by the engine 15 mounted on the traveling body 11, piping connecting the hydraulic pump to the swing motor 51 of the hydraulic actuator group 27, and a hydraulic switching valve provided on the piping. The hydraulic switching valve may be a so-called electromagnetic valve, and is driven by a drive signal input from the controller 70. The swing motor 51, the hoisting cylinder 52, the telescopic cylinder 53, the hydraulic motor 54, and the jack 24 are driven by the electromagnetic valve being driven. In other words, the controller 70 can output a drive signal to swing, hoist, and extend the boom 22, wind or unwind the wire 41, and extend or retract the jack 24.

The sensor group 26 includes a rotation angle sensor 61, a boom length sensor 62, and a boom angle sensor 63.

The swivel angle sensor 61 outputs a detection value corresponding to the swivel angle from the swivel reference position of the swivel base 21. The swivel reference position is the position shown in FIG. 3 where the boom 22 protrudes forward. The swivel angle sensor 61 is, for example, a rotary encoder provided on the swivel shaft of the swivel base 21. The swivel angle sensor 61 outputs a number of pulse signals corresponding to the swivel angle of the swivel base 21 as a detection value. Below, the swivel angle is described as an angle in the counterclockwise direction from the swivel reference position. In other words, the swivel angle increases when the boom 22 swivels to the left and decreases when the boom 22 swivels to the right. The detection value output by the swivel angle sensor 61 corresponds to the "second detection value" described in the claims.

The boom length sensor 62 outputs a detection value corresponding to the length of the boom 22. The detection value output by the boom length sensor 62 may indicate the length from the base end to the tip end of the boom 22, or may indicate the extended length of the boom 22. The boom length sensor 62 may be a sensor that outputs a detection value that directly indicates the boom length, a sensor that outputs a detection value that indicates the extended length of the telescopic cylinder 53, or a sensor that outputs a detection value that indicates the drive time of the telescopic cylinder 53. The detection value output by the boom length sensor 62 corresponds to the "third detection value" described in the claims.

The hoisting angle sensor 63 outputs a detection value corresponding to the hoisting angle of the boom 22. The hoisting angle sensor 63 is, for example, an inclination sensor or a horizontal sensor that outputs an angle with respect to the horizontal plane. Alternatively, the hoisting angle sensor 63 may be a sensor that outputs a detection value indicating the extension length or drive time of the hoisting cylinder 52. The detection value output by the hoisting angle sensor 63 corresponds to the "first detection value" described in the claims.

In the following explanation, the detection values output by the rotation angle sensor 61, boom length sensor 62, and boom hoisting angle sensor 63 are also referred to as "sensor detection values."

The turning angle sensor 61, boom length sensor 62, and elevation angle sensor 63 are connected to the controller 70 by cables. In other words, the sensor detection values output by the turning angle sensor 61 and the like are input to the controller 70. The sensor detection values output by each sensor may also be input to the controller 70 by wireless communication.

Each sensor, such as the turning angle sensor 61, performs detection continuously at a predetermined time interval (so-called sampling period/sampling cycle) and outputs the sensor detection value at the predetermined time interval.

Figure 4 shows the control device 29 inside the cabin 13.

As shown in FIG. 4, the control device 29 is disposed in the cabin 13. The control device 29 includes an operating lever, a foot pedal, and an operating button that are operated by the operator.

The control device 29 is connected to the controller 70 (see FIG. 1) by a signal line (not shown). The operator operates the control device 29 to input instructions to the controller 70 and control the crane device 12.

As shown in FIG. 4, the control monitor device 80 is installed in the cabin 13. As shown in FIG. 1, the control monitor device 80 includes a display 81, a transparent sheet-like touch sensor 82 overlaid on the display 81, and a speaker 83. In other words, the control monitor device 80 is a so-called AML.

The operator inputs (selects) image designation using touch sensor 82. Image designation is information indicating one or more of the rear monitoring image shown in FIG. 6(B), the boom left side monitoring image shown in FIG. 6(C), the boom right side monitoring image shown in FIG. 6(D), the boom tip monitoring image shown in FIG. 6(E), and the boom top side monitoring image shown in FIG. 6(F). The operator inputs designation of one or more of the above five images to be displayed on display 84 through touch sensor 82. The input image designation is stored in memory 72 (see FIG. 1) as second designation information.

As shown in FIG. 1, the safety device 17 includes a wide-angle camera 18, a display 84, and a controller 70.

The wide-angle camera 18 includes a focusing lens, an optical system, multiple image sensors, a control circuit, and a communication interface. The focusing lens is a so-called wide-angle lens or fisheye lens, and the wide-angle camera 18 is a so-called wide-angle camera or fisheye camera.

The optical system has, for example, multiple lenses, and guides light focused by a focusing lens to an imaging element. The multiple imaging elements are image sensors such as CCD or CMOS. The multiple imaging elements are arranged in a matrix (grid). The multiple imaging elements output image data according to the incident light. The image data consists of multiple pixel data and pixel position data. If the captured image is a color image, one piece of pixel data consists of, for example, three color pixel data. The control circuit captures images using the wide-angle camera 18 and generates image data. The communication interface converts the image data into transmission data and outputs it. The control circuit is, for example, a driver IC, and the communication interface is, for example, a communication IC.

As shown in Figures 2 and 3, the wide-angle camera 18 is attached to the upper surface 45 of the base boom 42. The wide-angle camera 18 is positioned away from the base end of the base boom 42. In other words, the wide-angle camera 18 is located at a higher position than the swivel base 21 when the boom 22 is raised (crane operation state). Therefore, the rear monitoring image (see Figure 6 (B)) generated from the image captured by the wide-angle camera 18 shows a wide area behind the cabin 13. In other words, by attaching the wide-angle camera 18 to the upper surface 45 of the boom 22, a wider area behind the cabin 13 can be displayed as an overhead image than when the wide-angle camera 18 is attached to the swivel base 21.

In the example shown in Figures 2 and 3, the wide-angle camera 18 is positioned closer to the tip side than the center of the base boom 42 so that a rear monitoring image (see Figure 6 (B)) that captures a wider range can be generated. However, the wide-angle camera 18 may also be positioned at the center of the base boom 42, or closer to the base end than the center.

The wide-angle camera 18 may be a monochrome camera that outputs image data representing a monochrome image, or a color camera that outputs image data representing a color image. Furthermore, the image data output by the wide-angle camera 18 may be data representing a still image, or video data consisting of multiple frames.

The dashed dotted lines shown in Figures 2 and 3 indicate the imaging range (angle of view) of the wide-angle camera 18. As shown in Figures 2 and 3, the mounting position of the wide-angle camera 18 and the angle of view of the wide-angle camera 18 are determined so that the imaging range of the wide-angle camera 18 includes the tip of the boom 22, the left and right sides of the boom 22, the area facing the top surface 45 of the boom 22, and the rear of the cabin 13. In other words, the imaging range of the wide-angle camera 18 includes all of the areas around the tip of the boom 22, the area on the left side of the boom 22, the area on the right side of the boom 22, the area facing the top surface 45 of the boom 22, and the rear area of the cabin 13.

The angle of view of the wide-angle camera 18 is set to 180 degrees or more. In the example shown in Figures 2 and 3, the angle of view of the wide-angle camera 18 is 220 degrees. Note that the angle of view of a wide-angle camera is generally up to 220 degrees. In other words, the angle of view of the wide-angle camera 18 is set in the range of 180 degrees or more and less than 220 degrees.

As shown in FIG. 1, the wide-angle camera 18 and the controller 70 are connected by a cable 19. The captured image data output by the wide-angle camera 18 is input to the controller 70 via the cable 19. However, the captured image data may also be input to the controller 70 by wireless communication. In that case, a battery and a transmitting antenna are attached to the boom 22, and a receiving antenna connected to the controller 70 by a communication line is installed in the cabin 13. The wide-angle camera 18 captures images using power supplied from the battery and transmits the captured image data from the transmitting antenna. The transmitted captured image data is received by the receiving antenna and input to the controller 70.

The wide-angle camera 18 captures images continuously at a predetermined time interval and outputs captured image data at the predetermined time interval. Alternatively, the wide-angle camera 18 continuously outputs video data consisting of multiple frames.

As shown in FIG. 4, the display 84 is installed in the cabin 13. As shown in FIG. 1, the display 84 is connected to the controller 70 by a signal line. The display 84 displays images indicated by image data input from the controller 70. Specifically, the display 84 displays a rear monitoring image shown in FIG. 6(B), a boom left side monitoring image shown in FIG. 6(C), a boom right side monitoring image shown in FIG. 6(D), a boom tip monitoring image shown in FIG. 6(E), and a boom top side monitoring image shown in FIG. 6(F). The display 84 corresponds to the "display" described in the claims. The above five images may be displayed on the display 81 of the control monitor device 80. In that case, the display 81 corresponds to the "display" described in the claims.

As shown in FIG. 1, the controller 70 includes a CPU 71, which is a central processing unit, a memory 72, a power supply circuit 74, and a communication bus (not shown). The controller 70 is realized, for example, by an IC, a computer, resistors, diodes, coils, capacitors, etc. mounted on a control board. The control board is disposed, for example, in a control box disposed in the cabin 13.

The CPU 71, memory 72, hydraulic supply device 28, turning angle sensor 61 of sensor group 26, wide-angle camera 18, and control device 29 are connected to a communication bus (not shown). A control program 75 executed by CPU 71 (described below) reads data and information from memory 72, stores the data and information in memory 72, and controls the operation of hydraulic actuator group 27. The control program 75 also acquires sensor detection values output by sensor group 26, captured image data output by wide-angle camera 18, and inputs made by the operator to control monitor device 80 and control device 29.

Memory 72 prestores OS 77, which is an operating system, and control program 75 executed by CPU 71. Memory 72 also prestores first designation information, and stores the above-mentioned second designation information indicating image designation input by the operator through control monitor device 80. The first designation information corresponds to the "designation information" described in the claims. The second designation information corresponds to the "designation information" described in the claims. The first designation information and second designation information correspond to the "designation information" described in the claims.

The first designation information, like the second designation information, is information that indicates one or more of the five images. The developer of the crane apparatus 12 stores in the memory 72 the first designation information according to the specifications determined by the model, type, destination, etc. of the crane apparatus 12.

In addition, the memory 72 pre-stores the first correction function, the second correction function, the third correction function, the fourth correction function, the fifth correction function, the first conversion function, the second conversion function, and the third conversion function.

Figure 6(A) is a schematic diagram of an image captured by the wide-angle camera 18. Figure 6(B) is a schematic diagram showing a rear monitoring image, which is an image showing the rear of the cabin 13. Figure 6(C) is a schematic diagram of a boom left side monitoring image showing the left side of the boom 22. Figure 6(D) is a schematic diagram of a boom right side monitoring image showing the right side of the boom 22. Figure 6(E) is a schematic diagram of a boom tip monitoring image showing the periphery of the tip of the boom 22. Figure 6(F) is a schematic diagram of a boom top side monitoring image showing the area facing the top surface 45 of the boom 22.

With reference to FIG. 6, the first correction function, the second correction function, the third correction function, the fourth correction function, the fifth correction function, the first conversion function, the second conversion function, and the third conversion function will be explained.

The first correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate corrected image data showing the rear of the cabin 13. Specifically, the first correction function is a function that discards data of areas other than the area behind the cabin 13, which is a part of the captured image shown by the captured image data, and corrects distortion. The first correction function discards data showing areas other than the area around the fourth correction reference (see FIG. 6(A)), which is the area surrounded by the two-dot chain line in FIG. 6(A), and corrects distortion.

The first correction function receives the captured image data and the hoisting angle detected by the hoisting angle sensor 63 as arguments, and outputs the corrected image data as a return value. To explain in more detail, the height of the wide-angle camera 18 changes according to the hoisting angle of the boom 22. When the height of the wide-angle camera 18 changes, the range shown in the captured image represented by the captured image data changes. The first correction function determines the pixels in the area to be discarded based on the hoisting angle passed to it. In other words, the range shown in the image represented by the generated corrected image data is a fixed range, regardless of the hoisting angle of the boom 22.

The first conversion function is a function that changes the viewpoint in the image represented by the corrected image data. The first conversion function is, for example, a function that performs a projective transformation. The corrected image data and the hoisting angle detected by the hoisting angle sensor 63 are passed as arguments to the first conversion function, and it outputs rear monitoring image data. The rear monitoring image data is an image showing the rear of the cabin 13, and is also a bird's-eye view image. To explain in more detail, the position (viewpoint) of the wide-angle camera 18 changes depending on the hoisting angle of the boom 22. The first conversion function changes the position of the viewpoint based on the hoisting angle passed to it. In other words, the rear monitoring image represented by the generated rear monitoring image data is a bird's-eye view image from the same viewpoint regardless of the hoisting angle of the boom 22.

The second correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom tip monitoring image data showing the periphery of the tip of the boom 22. Specifically, the second correction function is a function that discards data of areas other than the area around the tip of the boom 22 shown by the first correction reference (see FIG. 6(A)), which is a part of the captured image shown by the captured image data, and corrects distortion. The second correction function corrects distortion by discarding data indicating areas other than the periphery of the first correction reference (see FIG. 6(A)), which is an area other than the area surrounded by the dotted line in FIG. 6(A). The second correction function is passed the captured image data as an argument, and outputs the boom tip monitoring image data as a return value.

The second transformation function is a function that enlarges an image. The second transformation function receives image data and the enlargement ratio as arguments, and outputs image data that represents the enlarged image. The second transformation function is, for example, an affine transformation (affine matrix) that performs the enlargement.

The third correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom top side monitoring image data. The boom top side monitoring image data is data representing a boom top side monitoring image that shows the area facing the top surface 45 of the boom 22.

The third correction function is a function that discards data of a part of the captured image represented by the captured image data, other than the area facing the top surface 45 of the boom 22, and corrects distortion. Specifically, the third correction function discards data indicating an area other than the area determined by the third correction standard (see FIG. 6(A)), other than the area surrounded by the dashed line in FIG. 6(A), and corrects distortion. The third correction function receives the captured image data as an argument and outputs the boom top side monitoring image data as a return value. The boom top side monitoring image represented by the boom top side monitoring image data (see FIG. 6(F)) is composed of an image from the viewpoint from the wide-angle camera 18 toward the base end of the boom 22 and an image from the wide-angle camera 18 toward the tip of the boom 22. The boom top side monitoring image may be a single image that combines an image from the viewpoint from the wide-angle camera 18 toward the base end of the boom 22 and an image from the wide-angle camera 18 toward the tip of the boom 22.

The fourth correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom left side monitoring image data that shows the left side of the boom 22. Specifically, the fourth correction function is a function that discards data of a part of the captured image indicated by the captured image data, which is an area other than the area on the left side of the boom 22 indicated by the left second correction reference (see FIG. 6(A)), i.e., an area other than the area to the left of the dotted line in FIG. 6(A), and corrects distortion. The fourth correction function receives the captured image data as an argument, and outputs boom left side monitoring image data as a return value. The left second correction reference corresponds to the "second correction reference" described in the claims.

The fifth correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom right side monitoring image data that shows the right side of the boom 22. Specifically, the fifth correction function is a function that discards data of a part of the captured image indicated by the captured image data, which is an area other than the area on the right side of the boom 22 indicated by the right second correction reference (see FIG. 6(A)), i.e., an area other than the area to the right of the dotted line in FIG. 6(A), and corrects distortion. The fifth correction function receives the captured image data as an argument, and outputs boom right side monitoring image data as a return value. The right second correction reference corresponds to the "second correction reference" described in the claims. The left second correction reference and the right second correction reference correspond to the "second correction reference" described in the claims.

The third transformation function is a function that rotates the image represented by the image data. The third transformation function receives image data and a rotation angle as arguments, and outputs image data representing the rotated image as a return value. The third transformation function is, for example, an affine transformation (affine matrix) that performs rotation.

In FIG. 6(A), the area used to generate the surveillance image is shown in a rectangular shape or the like by dashed lines, dotted lines, dashed lines, and broken lines, but the area used to generate the surveillance image may be a shape other than rectangular.

The control program 75 shown in FIG. 1 is a program that controls the operation of the hydraulic actuator group 27 etc. based on the operation signal input from the control device 29, controls the display on the display 81 of the control monitor device 80, accepts input from the operator through the control monitor device 80, and executes the monitoring image display process described below (see FIG. 5).

The power supply circuit 74 converts the DC voltage supplied from the battery 16 mounted on the running body 11 into a DC voltage of a predetermined voltage value, such as 3.3 V or 5 V, and outputs it. The power supply circuit 74 is, for example, a DC/DC converter (power supply IC) such as a switching regulator. The battery 16 is charged by the engine 15 of the running body 11. The DC voltage output by the power supply circuit 74 is supplied to the CPU 71, the control device 29, the sensor group 26, the wide-angle camera 18, etc. In FIG. 1, the power supply lines from the power supply circuit 74 to the sensor group 26, etc. are omitted from the illustration.

The following describes the processing executed by the controller 70 of the crane vehicle 10. The processing executed by the controller 70 is the processing that the control program 75 causes the CPU 71 to execute.

First, the process in which the controller 70 acquires the second designation information, which is the image designation of the operator, through the control monitor device 80 and stores it in the memory 72 will be described. The controller 70 acquires the first designation information previously stored in the memory 72. The controller 70 causes the type of image indicated by the first designation information to be displayed on the display 81. The operator inputs the type of image that he/she wishes to display on the display 84 from among the displayed types of images through the touch sensor 82. For example, if the first designation information is the above-mentioned five types of images, the operator's options are all of the above-mentioned five types of images. However, of the above-mentioned five types of images, an image that is always displayed on the display 84 without any room for the operator to select may be predetermined. In that case, the information indicating the type of image that is always displayed on the display 84 is included in the first designation information. Furthermore, the image is excluded from the operator's options.

The controller 70 stores in the memory 72 second designation information indicating the type of image designated by the operator using the control monitor device 80.

FIG. 5 is a flowchart of the monitoring image display process executed by the controller 70.

The monitoring image display process will be described with reference to FIG. 5. The controller 70 starts the monitoring image display process, for example, when the control device 29 is powered on, or when the operator makes a specified input to the control device 29 or the control monitor device 80.

The controller 70 acquires the first designation information and the second designation information stored in the memory 72 (S11). If the operator can select all of the types of images indicated by the first designation information, the controller 70 may acquire only the second designation information indicating the type of image designated (selected) by the operator. If the first designation information includes information indicating the type of image that is always displayed on the display 84, the controller 70 acquires both the first designation information and the second designation information. If the operator cannot designate (select) an image, the controller 70 acquires only the first designation information. In the following description, the first designation information and the second designation information are collectively referred to as "designation information". The process of step S11 corresponds to the "designation information acquisition process" described in the claims.

The controller 70 acquires the sensor detection values output by each sensor of the sensor group 26 and stores them in the memory 72 (S12). The controller 70 repeatedly acquires the sensor detection values at a predetermined sampling period and stores them in the memory 72. The controller 70 also acquires the captured image data output by the wide-angle camera 18 (S13).

The controller 70 determines whether the designation information acquired in step S11 indicates the display of a "rear monitoring image" (S14). That is, in step S14, it is determined whether the pilot has designated (selected) the display of a "rear monitoring image" or whether the "rear monitoring image" is an image that is always displayed.

When the controller 70 determines that the designated information indicates the display of a "rear monitoring image" (S14: Yes), it generates rear monitoring image data based on the sensor detection value acquired in step S12, the captured image data acquired in step S13, and the first correction function and first conversion function stored in the memory 72 (S15). In more detail, the controller 70 calls the first correction function, passes the captured image data and the hoisting angle θ detected by the hoisting angle sensor 63 as arguments to the first correction function, and receives the corrected image data as a return value. The controller 70 passes the received corrected image data and the hoisting angle θ as arguments to the first conversion function, and receives the rear monitoring image data as a return value. The image represented by the rear monitoring image data is an image showing a predetermined range around the rear of the cabin 13, and is an overhead image. The process of step S15 corresponds to the "generation process" and "fourth process" described in the claims. The process of acquiring the corrected image data based on the first correction function, the captured image data, and the hoisting angle θ corresponds to the "process of correcting the captured image data based on the fourth correction standard" described in the claims. The process of acquiring the rear monitoring image data based on the corrected image data, the hoisting angle θ, and the first conversion function corresponds to the "conversion process" described in the claims.

If the controller 70 determines that the specified information does not indicate the display of a "rear monitoring image" (S14: No), it skips the processing of step S15.

After executing the process of step S15, or when it is determined that the designation information does not indicate the display of a "rear monitoring image" (S14: No), the controller 70 determines whether the designation information acquired in step S11 indicates the display of a "boom tip monitoring image" (S16). That is, in step S16, it is determined whether the operator has designated (selected) the display of the "boom tip monitoring image", or whether the "boom tip monitoring image" is an image that is always displayed.

When the controller 70 determines that the designation information acquired in step S11 indicates the display of a "boom tip monitoring image" (S16: Yes), it acquires a magnification ratio based on the boom length indicated by the sensor detection value acquired in step S12 (S17). Specifically, the memory 72 pre-stores a table that associates the boom length with the magnification ratio. The controller 70 reads out the magnification ratio associated with the boom length indicated by the sensor detection value from the table.

The controller 70 generates boom tip monitoring image data based on the captured image data acquired in step S13, the magnification factor acquired in step S17, and the second correction function and second conversion function stored in the memory 72 (S18). In more detail, the controller 70 calls the second correction function, passes the captured image data to the second correction function as an argument, and acquires the pre-magnification boom tip monitoring image data as a return value. Next, the controller 70 calls the second conversion function that performs the magnification, passes the pre-magnification boom tip monitoring image data and the magnification factor to the second conversion function as arguments, and acquires the boom tip monitoring image data showing the magnified boom tip monitoring image as a return value. In other words, the boom tip monitoring image showing the periphery of the tip of the boom 22 is magnified by a magnification factor according to the distance from the wide-angle camera 18 to the tip of the boom 22 and displayed on the display 84. The process of step S18 corresponds to the "generation process" and "first process" described in the claims. In step S18, the process of acquiring pre-magnification boom tip monitoring image data based on the second correction function and the captured image data corresponds to the "process of correcting the captured image data based on the first correction standard" described in the claims. In step S18, the process of acquiring boom tip monitoring image data based on the second conversion function, pre-magnification boom tip monitoring image data, and the magnification ratio corresponds to the "fourth magnification process" described in the claims.

If the controller 70 determines that the designation information acquired in step S11 does not indicate the display of a "boom tip monitoring image" (S16: No), it skips the processing of steps S17 and S18.

After determining in step S16 that the designation information does not indicate the display of a "boom tip monitoring image," or after executing the process of step S18, the controller 70 determines whether the designation information acquired in step S11 indicates the display of a "boom top side monitoring image" (S19). That is, in step S19, it is determined whether the operator has designated (selected) the display of a "boom top side monitoring image," or whether the "boom top side monitoring image" is an image that is always displayed.

When the controller 70 determines that the designation information acquired in step S11 indicates the display of a "boom top side monitoring image" (S19: Yes), it generates boom top side monitoring image data based on the captured image data acquired in step S13 and the third correction function stored in memory 72 (S20). To explain in more detail, the controller 70 calls the third correction function, passes the captured image data to the third correction function as an argument, and obtains the boom top side monitoring image data as a return value. The process of step S20 corresponds to the "generation process" and "third process" described in the claims.

Then, the controller 70 determines whether the boom 22 is being raised based on the sensor detection value acquired in step S12 (S21). To explain in more detail, the controller 70 determines whether the hoisting angle θ is increasing over time based on the multiple hoisting angles θ detected by the hoisting angle sensor 63 at a predetermined sampling period.

When the controller 70 determines that the boom 22 is being raised (S21: Yes), it executes a first enlargement process to enlarge the boom top side monitoring image shown by the boom top side monitoring image data generated in step S20 (S22). To explain in more detail, the controller 70 calls a second conversion function that performs the enlargement, and reads out a predetermined enlargement ratio stored in advance from the memory 72. The controller 70 passes the boom top side monitoring image data generated in step S20 and the predetermined enlargement ratio as arguments to the second conversion function, and obtains the boom top side monitoring image data showing the enlarged boom top side monitoring image as a return value. The process of step S22 corresponds to the "first enlargement process" described in the claims.

If the controller 70 determines that the boom 22 is not being raised (S21: No), it skips the processing of step S22. That is, when the boom 22 is not being raised, a boom top side monitoring image of the specified size is displayed on the display 84, and when the boom 22 is being raised, a boom top side monitoring image of a size larger than the specified size is displayed on the display 84. It is while the boom 22 is being raised that there is a possibility that the top surface 45 of the boom 22 may come into contact with an obstacle. That is, the controller 70 enlarges and displays the boom top side monitoring image while the boom 22 is being raised, when there is a possibility that the top surface 45 of the boom 22 may come into contact with an obstacle.

If the controller 70 determines that the designation information acquired in step S11 does not indicate the display of a "boom top side monitoring image" (S19: No), it skips the processing of steps S20, S21, and S22.

After determining in step S19 that the designation information does not indicate the display of the "boom top side monitoring image", or after determining in step S21 that the boom 22 is not being raised, or after executing the processing of step S22, the controller 70 determines whether the designation information acquired in step S11 indicates the display of the "boom left side monitoring image" and the "boom right side monitoring image" (S23). That is, in step S23, it is determined whether the operator has designated (selected) the display of the "boom left side monitoring image" and the "boom right side monitoring image", or whether the "boom left side monitoring image" and the "boom right side monitoring image" are images that are always displayed.

When the controller 70 determines that the designation information acquired in step S11 indicates the display of a "boom left side monitoring image" and a "boom right side monitoring image" (S23: Yes), it determines a first rotation angle and a second rotation angle based on the boom hoisting angle θ of the boom 22 indicated by the sensor detection value acquired in step S12 (S24). For example, it determines the boom hoisting angle θ as the first rotation angle, and -θ as the second rotation angle. The first rotation angle is the angle by which the boom left side monitoring image is rotated, and the second rotation angle is the angle by which the boom right side monitoring image is rotated.

The controller 70 generates boom left side monitoring image data based on the captured image data acquired in step S13, the fourth correction function and the third conversion function stored in the memory 72, and the first rotation angle determined in step S24 (S25). The controller 70 also generates boom right side monitoring image data based on the captured image data acquired in step S13, the fifth correction function and the third conversion function stored in the memory 72, and the second rotation angle determined in step S24 (S25). Note that in the following, the boom left side monitoring image data and the boom right side monitoring image data may be collectively referred to as boom side monitoring image data, and the boom left side monitoring image and the boom right side monitoring image may be collectively referred to as boom side monitoring images.

The boom left side monitoring image data corresponds to the "boom side monitoring image data" described in the claims. The boom right side monitoring image data corresponds to the "boom side monitoring image data" described in the claims. The boom left side monitoring image data and the boom right side monitoring image data correspond to the "boom side monitoring image data" described in the claims. The boom left side monitoring image corresponds to the "boom side monitoring image" described in the claims. The boom right side monitoring image corresponds to the "boom side monitoring image" described in the claims. The boom left side monitoring image and the boom right side monitoring image correspond to the "boom side monitoring image" described in the claims.

The process of step S25 will be described in detail. The controller 70 calls the fourth correction function, passes the captured image data to the fourth correction function as an argument, and obtains the left side monitoring image data of the boom before rotation as a return value. Next, the controller 70 calls the third conversion function that rotates the image, passes the left side monitoring image data of the boom before rotation and the first rotation angle to the third conversion function as arguments, and obtains the left side monitoring image data of the boom showing the rotated left side monitoring image of the boom as a return value. The ground reflected in the rotated left side monitoring image of the boom becomes approximately horizontal on the display 84. In other words, the rotation of the image by the third conversion function is performed to make the ground reflected in the left side monitoring image of the boom horizontal on the display 84 to eliminate the discomfort of the operator. Similarly, the controller 70 calls the fifth correction function, passes the captured image data to the fifth correction function as an argument, and obtains the right side monitoring image data of the boom before rotation as a return value. Next, the controller 70 calls a third transformation function that rotates the image, passes the pre-rotation boom right side monitoring image data and the second rotation angle as arguments to the third transformation function, and obtains boom right side monitoring image data showing the rotated boom right side monitoring image as a return value. The ground reflected in the rotated boom right side monitoring image becomes approximately horizontal on the display 84.

The process of step S25 corresponds to the "generation process" and "second process" described in the claims. In step S25, the process of acquiring pre-rotation boom left side monitoring image data based on the captured image data and the fourth correction function corresponds to the "process of correcting the captured image data based on the second correction standard" described in the claims. In step S25, the process of acquiring pre-rotation boom right side monitoring image data based on the captured image data and the fifth correction function corresponds to the "process of correcting the captured image data based on the second correction standard" described in the claims. In step S25, the process of acquiring pre-rotation boom left side monitoring image data and pre-rotation boom right side monitoring image data based on the captured image data, the fourth correction function, and the fifth correction function corresponds to the "process of correcting the captured image data based on the second correction standard" described in the claims. In step S25, the process of acquiring boom left side monitoring image data based on pre-rotation boom left side monitoring image data, the first rotation angle, and the third conversion function corresponds to the "rotation process" described in the claims. In step S25, the process of acquiring boom right side monitoring image data based on pre-rotation boom right side monitoring image data, the second rotation angle, and the third conversion function corresponds to the "rotation process" described in the claims. In step S25, the process of acquiring boom left side monitoring image data and boom right side monitoring image data based on pre-rotation boom left side monitoring image data, pre-rotation boom right side monitoring image data, the first rotation angle, the second rotation angle, and the third conversion function corresponds to the "rotation process" described in the claims.

Next, the controller 70 determines whether the boom 22 is rotating left based on the sensor detection value acquired in step S12 (S26). To explain in more detail, the controller 70 determines whether the rotation angle α detected by the rotation angle sensor 61 at a predetermined sampling period is increasing over time. If the rotation angle α is increasing, the controller 70 determines that the boom 22 is rotating left.

When the controller 70 determines that the boom 22 is rotating to the left (S26: Yes), it executes a second enlargement process to enlarge the boom left side monitoring image represented by the boom left side monitoring image data generated in step S25 (S27). To explain in more detail, the controller 70 calls a second conversion function that performs the enlargement, and reads out a predetermined enlargement ratio stored in advance from the memory 72. The controller 70 passes the boom left side monitoring image data generated in step S25 and the predetermined enlargement ratio as arguments to the second conversion function, and obtains the boom left side monitoring image data representing the enlarged boom left side monitoring image as a return value. The process of step S27 corresponds to the "second enlargement process" described in the claims.

If the controller 70 determines that the boom 22 is not rotating left (S26: No), it skips the processing of step S27. That is, if the boom 22 is not rotating left, a boom left side monitoring image of the specified size is displayed on the display 84, and if the boom 22 is rotating left, a boom left side monitoring image of a size larger than the specified size is displayed on the display 84. It is during the left rotation of the boom 22 that there is a possibility that the left side of the boom 22 may come into contact with an obstacle. That is, the controller 70 enlarges and displays the boom left side monitoring image during the left rotation of the boom 22 when there is a possibility that the left side of the boom 22 may come into contact with an obstacle.

When the controller 70 determines that the boom 22 is not rotating to the left (S26: No), or after executing the process of step S27, it determines whether or not the boom 22 is rotating to the right based on the sensor detection value acquired in step S12 (S28). To explain in more detail, the controller 70 determines whether or not the rotation angle α detected by the rotation angle sensor 61 at a predetermined sampling period is decreasing over time. If the rotation angle α is decreasing, the controller 70 determines that the boom 22 is rotating to the right.

When the controller 70 determines that the boom 22 is rotating to the right (S28: Yes), it executes a third enlargement process to enlarge the boom right side monitoring image represented by the boom right side monitoring image data generated in step S25 (S29). To explain in more detail, the controller 70 calls a second conversion function that performs the enlargement, and reads out a predetermined enlargement ratio stored in advance from the memory 72. The controller 70 passes the boom right side monitoring image data generated in step S25 and the predetermined enlargement ratio as arguments to the second conversion function, and obtains the boom right side monitoring image data representing the enlarged boom right side monitoring image as a return value. The process of step S29 corresponds to the "third enlargement process" described in the claims.

If the controller 70 determines that the boom 22 is not rotating to the right (S28: No), it skips the processing of step S29. In other words, if the boom 22 is not rotating to the right, a boom right side monitoring image of the specified size is displayed on the display 84, and if the boom 22 is rotating to the right, a boom right side monitoring image of a size larger than the specified size is displayed on the display 84. It is during the right rotation of the boom 22 that there is a possibility that the right side of the boom 22 may come into contact with an obstacle. In other words, the controller 70 enlarges and displays the boom right side monitoring image during the right rotation of the boom 22 when there is a possibility that the right side of the boom 22 may come into contact with an obstacle.

If the controller 70 determines that the designation information acquired in step S11 does not indicate the display of a "boom side monitoring image" (S23: No), it skips the processing from steps S24 to S29.

After determining that the designation information does not indicate the display of a "boom side monitoring image" (S23: No), or after determining that a right turn is not being made (S28: No), or after executing the processing of step S29, the controller 70 inputs the generated image data to the display 84 and causes the monitoring image to be displayed on the display 84 (S30). Specifically, the controller 70 inputs the generated monitoring image data to the display 84. The monitoring image displayed on the display 84 is an image determined to be displayed according to the specifications (first designation information) and an image designated (selected) by the operator. In addition, the monitoring image displayed on the display 84 is an image that is appropriately enlarged in accordance with the extension, contraction, elevation, and rotation of the boom 22. The processing of step S30 corresponds to the "display processing" described in the claims.

Each monitoring image will now be described in detail. The wide-angle camera 18 is attached to the boom 22 and rotates integrally with the boom 22. Therefore, the rear monitoring image shown in Figure 6 (B) always captures the rear of the cabin 13 (see Figure 2), rather than the rear of the crane truck 10. In other words, the rear monitoring image always captures the rear of the cabin 13, which is the blind spot for the operator. Furthermore, the rear monitoring image is always a bird's-eye view image looking down at the rear of the cabin 13 from the same viewpoint, regardless of whether the boom 22 is rising or falling. Furthermore, the rear monitoring image is an image that always captures a fixed range, regardless of whether the boom 22 is rising or falling.

The boom left side monitoring image shown in FIG. 6(C) is an image looking left from the boom 22, and the boom right side monitoring image shown in FIG. 6(D) is an image looking right from the boom 22. The ground shown in the boom left side monitoring image and the ground shown in the boom right side monitoring image are roughly horizontal on the display 84, and obstacles standing vertically from the ground in the boom left side monitoring image and the boom right side monitoring image are roughly aligned in the up-down direction on the display 84.

The boom tip monitoring image shown in FIG. 6 (E) shows the area around the tip of the boom 22. The range shown in the boom tip monitoring image is, for example, the range in which an operator who notices an obstacle in the boom tip monitoring image can stop the crane apparatus 12 before the tip of the boom 22 comes into contact with the obstacle. This range becomes constant regardless of the extended length of the boom 22 by enlarging the boom tip monitoring image (S17, S18).

The boom top side monitoring image shown in FIG. 6 (F) consists of an image of the tip and upward view of the boom 22 from the wide-angle camera 18, and an image of the base and upward view of the boom 22 from the wide-angle camera 18. In other words, the boom top side monitoring image is an image that shows the area facing the top surface 45 of the boom 22. The two images may be displayed side by side on the display 84, or side by side above and below. Alternatively, the two images may be combined and displayed as a single image on the display 84.

The five surveillance images shown in Figures 6(B) through 6(F) may be arranged and displayed in any manner on the display 84.

Furthermore, the enlarged boom side monitoring image and boom top side monitoring image may be displayed on the display 84 separately from the boom side monitoring image and boom top side monitoring image before enlargement. For example, when the boom 22 is not rotated, the boom side monitoring image before enlargement is displayed on the display 84, and when the boom 22 is rotated, the enlarged boom side monitoring image is displayed on the display 84 together with the boom side monitoring image before enlargement.

As shown in FIG. 5, after executing the process of step S30, the controller 70 determines whether or not to end the surveillance image display process (S31). The controller 70 determines to end the surveillance image display process based on, for example, the power to the control device 29 being turned off, or based on an end instruction being input through the control monitor device 80 (S31: Yes, end). If the controller 70 determines not to end the surveillance image display process (S31: No), it executes the processes from step S11 onwards again. In other words, the controller 70 repeatedly executes the surveillance image display process. The period (sampling period) at which the controller 70 repeats the surveillance image display process is set to, for example, between several hundred milliseconds and several minutes.

[Effects of the embodiment]

The wide-angle camera 18 is attached to the top surface 45 of the boom 22, and can simultaneously capture images of the area around the tip of the boom 22, the left and right side areas of the boom 22, the area facing the top surface 45 of the boom 22, and the rear area around the base end of the boom 22. The controller 70 generates monitoring image data from the captured image data for the area indicated by the first designation information prestored in the memory 72 or the area designated (selected) by the operator, and displays the monitoring image indicated by the generated monitoring image data on the display 84. Therefore, the safety device 17 can display a monitoring image of a blind spot area or a monitoring area desired by the operator of the crane vehicle 10 on the display 84, even if the area is an area around the tip of the boom 22, a side area of the boom 22, an area facing the top surface 45 of the boom 22, or a rear area around the base end of the boom 22 (rear area of the cabin 13). Therefore, the crane device 12 can eliminate blind spots regardless of the type or specifications, or can eliminate blind spots according to the operator's request.

The safety device 17, which is made up of the wide-angle camera 18, the display 84, and the controller 70, can change the type of image displayed on the display 84 based on the first designation information stored in the memory 72. Therefore, the safety device 17 can be used in common with various crane devices 12. As a result, it is possible to standardize parts.

The controller 70 reads out second designation information indicating the operator's image designation from the memory 72, and causes the type of surveillance image designated by the second designation information to be displayed on the display 84. Therefore, the surveillance image desired by the operator can be displayed on the display 84.

The position and height of the wide-angle camera 18 relative to the swivel base 21 changes as the boom 22 is raised and lowered. A viewpoint conversion process is performed to obtain an overhead image based on the angle of the boom 22, which is a parameter that determines the position and height of the wide-angle camera 18. Therefore, the crane apparatus 12 can always display on the display 84 a rear monitoring image that overlooks the rear of the cabin 13 from a fixed position, regardless of the angle of the boom 22.

The boom side monitoring image tilts according to the elevation of the boom 22. The controller 70 rotates the boom side monitoring image according to the elevation angle of the boom 22, correcting the tilt of the boom side monitoring image. Therefore, the crane apparatus 12 makes it easier for the operator to recognize obstacles compared to a case in which the tilt of the boom side monitoring image is not corrected.

When the boom 22 is being raised, there is a risk of contact between the top surface 45 of the boom 22 and an obstacle. When the boom 22 is being raised, the controller 70 enlarges and displays the boom top side monitoring image. Therefore, the crane apparatus 12 makes it easier for the operator to recognize obstacles compared to when the boom top side monitoring image is always displayed at a constant size on the display 84.

When the boom 22 is rotating, there is a risk of contact between the side of the boom 22 and an obstacle. The controller 70 enlarges and displays the boom left side monitoring image when the boom 22 is rotating left, and enlarges and displays the boom right side monitoring image when the boom 22 is rotating right. Therefore, the crane apparatus 12 makes it easier for the operator to recognize obstacles compared to when the boom side monitoring image is always displayed at a constant size on the display 84.

The wide-angle camera 18 is mounted on the base boom 42, whose position does not change with the extension and retraction of the boom 22. This makes it easier to establish a wired connection between the wide-angle camera 18 and the controller 70 compared to when the wide-angle camera 18 is mounted on the intermediate boom 43 or top boom 44, whose position changes with the extension and retraction of the boom 22.

When the wide-angle camera 18 is mounted on the base boom 42, the tip of the boom 22 moves away from the wide-angle camera 18 as the boom 22 extends. The boom tip monitoring image is enlarged at a magnification that corresponds to the extension length of the boom 22, so the crane apparatus 12 can make it easier for the operator to recognize obstacles approaching the tip of the boom 22.

[Variations]

In the above embodiment, an example was described in which the angle of view of the wide-angle camera 18 is 220 degrees. However, the angle of view of the wide-angle camera 18 may be other angles between 180 degrees and 220 degrees, such as 180 degrees or 200 degrees.

In the above embodiment, an example was described in which the crane apparatus 12 was mobile (crane vehicle 10). However, the crane apparatus 12 may also be a fixed type, such as a tower crane.

In the above embodiment, an example has been described in which the first designation information is pre-stored in memory 72, and the second designation value input by the operator is stored in memory 72. In other words, an example has been described in which the monitoring image to be displayed is determined in advance according to the format and specifications, and the operator can select the monitoring image to be displayed. However, the first designation information does not have to be pre-stored in memory 72, or the function for the operator to input the second designation value does not have to be provided. In other words, the operator may only be able to select the monitoring image to be displayed, or the monitoring image to be displayed may be determined in advance according to the format and specifications, and the operator may not be able to select the monitoring image.

In the above embodiment, an example has been described in which the boom left side monitoring image is enlarged when the boom 22 is rotated left, and the boom right side monitoring image is enlarged when the boom 22 is rotated right. However, when the boom 22 is not rotated, the boom left side monitoring image and the boom right side monitoring image are not displayed on the display 84, and when the boom 22 is rotated left, the boom left side monitoring image is displayed normally or enlarged, and when the boom 22 is rotated right, the boom right side monitoring image is displayed normally or enlarged.

In the above embodiment, an example has been described in which the boom top side monitoring image is enlarged when the boom 22 is raised. However, when the boom 22 is not raised, the boom top side monitoring image is not displayed on the display 84, and when the boom 22 is raised, the boom top side monitoring image may be displayed normally or enlarged.

In the above embodiment, an example was described in which the first correction function that corrects the captured image and the first conversion function that changes the viewpoint are separate functions. However, the first correction function and the first conversion function may be one function. Similarly, the second correction function and the second conversion function may be one function, the fourth correction function and the third conversion function may be one function, and the fifth correction function and the third conversion function may be one function.

In the above embodiment, an example of correcting and converting image data using functions has been described. However, as long as image data can be corrected and converted, a class that generates monitoring image data as an instance (object), an arithmetic expression, or the like may also be used.

10: Crane truck 11: Running body 12: Crane device 13: Cabin 17: Safety device 18: Wide-angle camera 19: Cable 21: Swivel base 22: Boom 23: Winch 26: Sensor group 29: Control device 41: Wire rope 42: Base boom (cylindrical body)
43... Intermediate boom (cylinder)
44...Top boom (cylinder)
45: Upper surface 51: Swing motor 52: Boom hoisting cylinder 53: Telescopic cylinder 54: Hydraulic motor 61: Swing angle sensor 62: Boom length sensor 63: Boom hoisting angle sensor 70: Controller 71: CPU
72: Memory 75: Control program 80: Control monitor device 81: Display 82: Touch sensor 84: Display

Claims (9)

A swivel base,
A cabin mounted on the rotating base and having a control device installed therein;
A boom that is provided on the swivel base and is extendable and retractable;
a safety device having a wide-angle camera, a controller, and a display attached to an upper surface of the boom in the lowered position;
The above controller is
A generation process of generating monitoring image data based on the captured image data generated by the wide-angle camera;
A display process of displaying a monitoring image represented by the monitoring image data on the display. The above controller is
A specific information acquisition process is further performed to acquire specific information;
The above generation process is
a first process for generating boom-tip monitoring image data by correcting the captured image data based on a first correction standard corresponding to the tip of the boom shown in a captured image represented by the captured image data;
a second process of correcting the captured image data based on a predetermined second correction standard in an area shown in an image represented by the captured image data and located to the side of the boom, to generate boom side monitoring image data;
a third process for generating boom top side monitoring image data by correcting the captured image data based on a third predetermined correction standard in an area shown in the captured image represented by the captured image data and facing the top surface of the boom;
a fourth process of correcting the captured image data based on a fourth correction standard corresponding to the base end of the boom shown in the captured image represented by the captured image data to generate rear monitoring image data, the fourth process being one or more processes indicated by the specified information. a boom hoisting angle sensor that outputs a first detection value indicating a boom hoisting angle,
The fourth process is as follows:
a process of correcting the captured image data based on the fourth correction standard;
A conversion process of converting a viewpoint based on the elevation angle indicated by the first detection value,
The crane apparatus according to claim 2 , wherein the rear monitoring image represented by the rear monitoring image data is an overhead image showing a rear of the cabin. a boom hoisting angle sensor that outputs a first detection value indicating a boom hoisting angle,
The second process is
a process of correcting the captured image data based on the second correction standard;
3. The crane apparatus according to claim 2, further comprising a rotation process of rotating a boom side monitoring image, which is a monitoring image represented by the boom side monitoring image data, at a rotation angle corresponding to the hoisting angle indicated by the first detection value. a boom hoisting angle sensor that outputs a first detection value indicating a boom hoisting angle,
The controller executes the third process based on the designation information indicating execution of the third process;
3. The crane apparatus according to claim 2, further comprising: a first enlargement process for enlarging a boom top side monitoring image indicated by the boom top side monitoring image data generated in the third process, before executing the display process, based on a determination that the boom is being raised based on a change over time in the hoisting angle indicated by the first detection value. A rotation angle sensor is further provided for outputting a second detection value indicating a rotation angle of the rotation table.
The boom side monitoring image data includes boom left side monitoring image data showing the left side of the boom and boom right side monitoring image data showing the right side of the boom,
The above controller is
executing the second process based on the designation information indicating execution of the second process;
a second enlargement process is executed to enlarge a boom left side monitoring image indicated by the boom left side monitoring image data generated in the second process before the execution of the display process, based on the determination that a left turn is being made based on the second detection value;
3. The crane apparatus according to claim 2, further comprising: a third enlargement process for enlarging the boom right side monitoring image indicated by the boom right side monitoring image data generated in the second process, before executing the display process, based on a determination that a right turn is being made based on the second detection value. The boom has a plurality of cylinders including a base boom,
The crane apparatus according to claim 1 , wherein the wide-angle camera is provided on the base boom. a boom length sensor that outputs a third detection value indicating an extended length of the boom;
The boom has a plurality of cylinders including a base boom,
The wide-angle camera is provided on the base boom,
The first process is
a process of correcting the captured image data based on the first correction standard;
The crane apparatus according to claim 2 , further comprising a fourth enlargement process for enlarging the image by an enlargement factor corresponding to the third detection value. The crane apparatus according to claim 1, wherein the angle of view of the wide-angle camera is 180 degrees or more.

Images