JP7146767B2 - road machinery - Google Patents

Description

The present invention relates to a road machine with a paving material axial feeding screw.

2. Description of the Related Art Conventionally, there has been known an image generation device that generates an image showing a view of an asphalt finisher and its surroundings viewed from above and presents it to the driver of the asphalt finisher (see Patent Document 1). This image generation device generates and displays an image showing the entire surroundings of the asphalt finisher, thereby allowing the driver to intuitively recognize the positional relationship between the asphalt finisher and surrounding objects.

Japanese Patent No. 6029941

However, in the above configuration, the space around the front end of the hopper wing is a blind spot for the hopper wing when viewed from the front camera, left camera, and right camera. Also, the above-described arrangements are not suitable for applications that require the driver to check the condition of a given local area, such as the area in front of the screed, where pavement material normally accumulates during construction. Therefore, the operator needs to directly visually confirm the presence or absence of an object in the space to be visually recognized, the amount of pavement material in a predetermined local area, and the like.

In view of the above, it would be desirable to provide a road machine that further reduces the range that is difficult to see in the image.

A road machine according to an embodiment of the present invention includes a tractor and a screed arranged behind the tractor, the screed comprising a front screed, a right rear screed and a left rear screed. wherein the road machine includes a working device that supplies paving material to an area in front of the right rear screed and an area in front of the left rear screed, and an image showing the surroundings of the road machine. A local image, which is an image displaying a certain surrounding image and showing at least one of an area in front of the right rear screed and an area in front of the left rear screed, is obtained from the surrounding image. and a display device for highlighting also.

By means of the above-described measures, a road machine is provided which further reduces the area that is difficult to see in the image.

1 is a side view of an asphalt finisher according to an embodiment of the present invention; FIG. 1 is a top view of an asphalt finisher according to an embodiment of the present invention; FIG. 1 is a rear view of an asphalt finisher according to an embodiment of the present invention; FIG. 1B is a diagram showing a configuration example of an image generation system mounted on the asphalt finisher of FIG. 1A; FIG. It is a display example of a first output image. It is a display example of a second output image. It is a display example of a second output image. It is a display example of a second output image. 9 is a flowchart of output image generation processing; 9 is a flowchart of output image switching processing; It is another display example of the second output image. It is still another display example of the second output image. It is still another display example of the second output image. 1 is a side view of an asphalt finisher according to an embodiment of the present invention; FIG. FIG. 11 is a top view of the asphalt finisher of FIG. 10; 11 is a block diagram showing a configuration example of a display system mounted on the asphalt finisher of FIG. 10; FIG. 13 is a diagram showing an example of an image generated by the display system of FIG. 12; FIG. FIG. 13B illustrates the partitioning of the input image used to generate the image of FIG. 13A; 13 is a diagram showing another example of an image generated by the display system of FIG. 12; FIG. 13 is a diagram showing yet another example of an image generated by the display system of FIG. 12; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

1A to 1C show a configuration example of an asphalt finisher 100 as a road machine according to an embodiment of the present invention, with FIG. 1A showing a side view, FIG. 1B showing a top view, and FIG. 1C showing a rear view. .

The asphalt finisher 100 is mainly composed of a tractor 1, a hopper 2 and a screed 3.

The tractor 1 is a device for running the asphalt finisher 100 and pulls the screed 3 . In this embodiment, the tractor 1 uses a traveling hydraulic motor to rotate two or four wheels to move the asphalt finisher 100 . The traveling hydraulic motor rotates by being supplied with hydraulic oil from a hydraulic pump driven by a motor such as a diesel engine. A driver's seat 1S and an operation panel 65 are arranged above the tractor 1 .

Imaging devices 51 (right camera 51R, left camera 51L, front camera 51F, right auxiliary camera 51V, left auxiliary camera 51U) are attached to the right side, left side, and front of the tractor 1 . A display device 52 is installed at a position where the driver sitting in the driver's seat 1S can easily visually recognize it. In this embodiment, the direction of the hopper 2 viewed from the tractor 1 is the front (+X direction), and the direction of the screed 3 viewed from the tractor 1 is the rear (-X direction). The +Y direction corresponds to the left direction and the -Y direction corresponds to the right direction.

A hopper 2, which is an example of a working device, is a mechanism for receiving pavement material (for example, an asphalt mixture). The working device is a device that feeds paving material in front of the screed 3 . In this embodiment, it is configured to be openable and closable in the vehicle width direction by means of a hydraulic cylinder. The asphalt finisher 100 normally opens the hopper 2 to receive pavement material from the dump truck bed. When the pavement material in the hopper 2 decreases, the hopper 2 is closed and the pavement material near the inner wall of the hopper 2 is collected in the center of the hopper 2, so that the conveyor CV, which is an example of a working device, screeds the pavement material. 3 so that it can be fed.

The screed 3 is a mechanism for spreading the paving material evenly. In this embodiment, the hydraulic cylinder is configured to be movable up and down in the vertical direction and to extend and contract in the width direction of the vehicle. The width of the screed 3 is greater than the width of the tractor 1 when extended in the vehicle width direction. In this embodiment, the screeds 3 include a front screed 30, a left rear screed 31L and a right rear screed 31R. The left rear side screed 31L and the right rear side screed 31R are configured to be able to expand and contract in the vehicle width direction (Y-axis direction). The left rear screed 31L and the right rear screed 31R, which are extendable in the vehicle width direction, are offset from each other in the traveling direction (X-axis direction). Therefore, it can have a longer width (length in the vehicle width direction) than when it is not offset, and can be stretched longer in the vehicle width direction, making it possible to construct a wider new pavement.

FIG. 2 schematically shows a configuration example of an image generation system SYS mounted on the asphalt finisher 100 of FIG. 1A. The image generation system SYS generates an output image based on an input image captured by the imaging device 51 mounted on the asphalt finisher 100, for example. In this embodiment, the image generation system SYS is mainly composed of a controller 50 , an imaging device 51 , a display device 52 , a storage device 54 and an input device 55 .

The controller 50 is, for example, a computer equipped with a CPU, volatile memory, non-volatile memory, and the like. The controller 50, for example, causes the CPU to execute programs respectively corresponding to the output image generating section 50A and the highlighting section 50B, and implements functions corresponding to the output image generating section 50A and the highlighting section 50B.

The imaging device 51 is a device that acquires an input image for generating an output image. In this embodiment, the camera is equipped with an imaging device such as CCD or CMOS. The imaging device 51 is attached to the tractor 1 so as to capture an image of the blind spot of the driver sitting in the driver's seat 1S, for example. The blind spots include, for example, the inner space of the hopper 2 (especially the portion near the tractor 1), the space outside the front end of the hopper 2, the space near the side of the asphalt finisher 100 near the road surface, and the like.

The imaging device 51 may be attached to a position other than the right side, the left side, and the front of the tractor 1 (for example, the rear). The imaging device 51 may be equipped with a wide-angle lens or a fisheye lens. The imaging device 51 may be attached to the hopper 2 or may be attached to the screed 3 .

In this embodiment, the imaging device 51 includes a front camera 51F, a left camera 51L, a right camera 51R, a left auxiliary camera 51U, and a right auxiliary camera 51V. As shown in FIGS. 1A and 1B, the front camera 51F is attached to the front upper end of the tractor 1, its optical axis 51FX extends forward in the traveling direction, and forms an angle α with the road surface in a side view. installed to The left camera 51L, as shown in FIGS. 1A to 1C, is attached to the left upper end of the tractor 1, and its optical axis 51LX forms an angle β with the left side of the tractor 1 in top view, and , and the road surface so as to form an angle γ in a rear view. The right camera 51R is mounted in the same manner as the left camera 51L, with left and right reversed. The left auxiliary camera 51U, as shown in FIGS. 1A to 1C, is attached to the left upper end of the tractor 1, and its optical axis 51UX forms an angle δ with the left side of the tractor 1 in top view, In addition, it is attached so as to form an angle ε with the road surface when viewed from the rear. The right auxiliary camera 51V is mounted in the same manner as the left auxiliary camera 51U, with left and right reversed. An area 51FA surrounded by a dashed line in FIG. 1B indicates the imaging range of the front camera 51F, an area 51LA surrounded by the dashed line indicates the imaging range of the left camera 51L, and an area 51RA surrounded by the dashed line indicates the right camera 51R. shows the imaging range of An area 51UA surrounded by a two-dot chain line indicates the imaging range of the left auxiliary camera 51U, and an area 51VA surrounded by the two-dot chain line indicates the imaging range of the right auxiliary camera 51V.

The left camera 51L and the left auxiliary camera 51U are attached to the tractor 1 so that the area 51UA indicating the imaging range of the left auxiliary camera 51U is completely included in the area 51LA indicating the imaging range of the left camera 51L. However, it may be attached to the tractor 1 so that the area 51LA and the area 51UA partially overlap, that is, the area 51UA protrudes from the area 51LA. Similarly, the right camera 51R and the right auxiliary camera 51V are attached to the tractor 1 so that the area 51VA indicating the imaging range of the right auxiliary camera 51V is completely included in the area 51RA indicating the imaging range of the right camera 51R. . However, it may be attached to the tractor 1 so that the area 51RA and the area 51VA partially overlap, that is, the area 51VA protrudes from the area 51RA. Also, the left auxiliary camera 51U and the right auxiliary camera 51V may be omitted.

The imaging device 51 is attached to the asphalt finisher 100 via brackets, stays, bars, or the like, for example. In this embodiment, the imaging device 51 is attached to the tractor 1 via an attachment stay. However, the imaging device 51 may be directly attached to the tractor 1 without the attachment stay, or may be embedded in the tractor 1 .

In this embodiment, the imaging device 51 outputs the acquired input image to the controller 50 . When the imaging device 51 acquires an input image using a fisheye lens or a wide-angle lens, the image pickup device 51 outputs to the controller 50 a corrected input image in which apparent distortion and tilt caused by using the lens are corrected. may Alternatively, the input image whose apparent distortion and tilt are not corrected may be output to the controller 50 as it is. In this case, the apparent distortion and tilt are corrected by the controller 50 .

In this way, the imaging device 51 is arranged so that its imaging range includes a plurality of blind spots on the left and right sides of the asphalt finisher 100 and inside and outside the hopper 2 .

The input device 55 is a device that enables the driver to input various information to the image generation system SYS, and includes, for example, a touch panel, buttons, switches, and the like. In this embodiment, the input device 55 includes a display changeover switch and a screw dial.

The display changeover switch is a switch for switching the configuration of the output image displayed on the display device 52 . A screw dial is a dial for adjusting the rotational speed of a screw SC, which is an example of a working device.

The storage device 54 is a device for storing various information. In this embodiment, memory device 54 is a non-volatile memory device and is integrated into controller 50 . However, the storage device 54 may be arranged outside the controller 50 as a structure separate from the controller 50 .

The display device 52 is a device for displaying various information. In this embodiment, a liquid crystal display is installed on the operation panel 65 and displays various images output by the controller 50 .

The output image generator 50A is a functional element for generating an output image, and is configured by, for example, software, hardware, or a combination thereof. In this embodiment, the output image generation unit 50A refers to the input image/output image correspondence map 54a stored in the storage device 54, and coordinates on the input image plane where the input image captured by the imaging device 51 is located, The coordinates on the output image plane where the output image is located are associated with each other. Then, the output image generation unit 50A generates an output image by associating the value of each pixel in the output image (for example, luminance value, hue value, saturation value, etc.) with the value of each pixel in the input image. .

The input image/output image correspondence map 54a stores the correspondence relationship between the coordinates on the input image plane and the coordinates on the output image plane in a referable manner. The correspondence relationship is preset based on various parameters such as the optical center of the imaging device 51, the focal length, the CCD size, the optical axis direction vector, the camera horizontal direction vector, and the projection method. The correspondence relationship may be set such that when the input image contains apparent distortion or tilt, the apparent distortion or tilt does not appear in the output image. In this case, the coordinate group forming the non-rectangular area on the input image plane is associated with the coordinate group forming the rectangular area on the output image plane. As for the correspondence relationship, if the apparent distortion or tilt in the input image has already been corrected when the input image is acquired, the coordinate group that forms the rectangular area on the input image plane forms the rectangular area on the output image plane as it is. It may be set so as to correspond to the coordinate group that

The highlighting portion 50B is a functional element for switching the content of the output image displayed on the display device 52, and is configured by, for example, software, hardware, or a combination thereof. In this embodiment, the highlighting section 50B switches the output image displayed on the display device 52 between the first output image and the second output image when the display switching switch as the highlighting switch is pressed. When the screw dial is operated, the first output image is switched to the second output image, and after that, when the screw dial is not operated (non-operating time) reaches a predetermined time, the second output image is changed to the first output image. You can switch to Similarly, when the highlighting switch is operated, the first output image is switched to the second output image, and after that, when the time during which the highlighting switch is not operated (non-operating time) reaches a predetermined time, the second output image is displayed. may be switched to the first output image. The non-operation time is counted using the timer function of the controller 50, for example.

The first output image contains the ambient image and does not contain the local image. The second output image includes an ambient image and a local image. The surrounding image is an image showing the surroundings of the asphalt finisher 100 . The local image is an image showing a predetermined local area related to the asphalt finisher 100, for example, an image showing an area where pavement material is supplied (spreading out) by the screw SC. The area to which the paving material is supplied by the screw SC is, for example, the area in front of the screed 3 and includes a retaining plate 70 (see FIG. 1B), a side plate 71 (see FIG. 1B) and a mold. It is an area surrounded by a board 72 (see FIG. 1B). By viewing this local image, the driver can view the local area while seated on the driver's seat 1S without moving around on the tractor 1 or twisting in the driver's seat 1S to look into the local area. It is possible to check the amount of paving material held. In addition, the surrounding conditions and the amount of paving material held in a local area can be confirmed almost simultaneously without moving the line of sight significantly. In this way, the asphalt finisher 100 equipped with the image generation system SYS can reduce the driver's fatigue due to the work of confirming the amount of holding. As a result, the safety of the asphalt finisher 100 can be improved. The local image may be an image showing the area inside the hopper 2 .

For example, when displaying the second output image on the display device 52, the highlighting unit 50B highlights the local image on the display device 52 so that the driver can distinguish between the surrounding image and the local image. For example, the local image is displayed in a display frame different from the display frame surrounding the surrounding image. In this case, the local image may be displayed on the surrounding image so as to overlap a part of the surrounding image, or may be displayed at a position different from the surrounding image so as not to overlap the surrounding image. Alternatively, an image portion corresponding to the local area in the surrounding image may be enlarged and displayed. In this case, at least a portion of the other image portion of the surrounding image may be reduced and displayed, or the display thereof may be omitted. Further, when the image portion corresponding to the local area in the surrounding image is enlarged and displayed, the display of the local image in another display frame may be omitted.

Next, with reference to FIG. 3, the first output image generated using the input images of the left camera 51L, right camera 51R, and front camera 51F will be described. FIG. 3 is a display example of the first output image displayed on the display device 52. As shown in FIG.

The first output image mainly includes a hopper image HG, a left surrounding image LG, a right surrounding image RG, and an illustration image 1CG. The hopper image HG, the left surrounding image LG and the right surrounding image RG constitute the surrounding image. The image generation system SYS generates a hopper image HG, a left surrounding image LG, a right surrounding image RG, and an illustration image so that the driver can recognize that the front of the asphalt finisher 100 matches the top of the screen of the display device 52. 1CG is displayed in a predetermined size at a predetermined position on the first output image. The positional relationship between the asphalt finisher 100 and the surrounding objects is presented to the driver by presenting the driver with the first output image as a bird's-eye view image showing the scene seen when looking down at the asphalt finisher 100 and its surroundings from above. This is to allow the driver to intuitively recognize.

The hopper image HG is generated based on the input image from the front camera 51F. In this embodiment, the hopper image HG is an image showing the inside of the hopper 2 when looking down on the hopper 2 from the tractor 1, and is generated by cutting out a part of the input image of the front camera 51F. , is placed at the top center of the first output image.

The left surrounding image LG is generated based on the input image of the left camera 51L. In this embodiment, the left surrounding image LG is an image showing the appearance of the left surrounding area seen when looking down from the tractor 1 on the left surrounding area on the left side of the asphalt finisher 100 in the traveling direction. Specifically, the left peripheral image LG is generated by cutting out a portion of the input image of the left camera 51L, performing distortion correction, and further performing image rotation processing, and is arranged at the left edge of the first output image. be. The left surrounding image LG includes an image of the left edge of the screed 3 and an image of the left edge of the hopper 2 .

The right surrounding image RG is generated based on the input image of the right camera 51R. In the present embodiment, the right surrounding image RG is an image showing the appearance of the right surrounding area seen when looking down from the tractor 1 on the right surrounding area on the right side of the asphalt finisher 100 in the traveling direction. Specifically, the right surrounding image RG is generated by cutting out a portion of the input image of the right camera 51R, performing distortion correction, and further performing image rotation processing, and arranged at the right end of the first output image. be. Also, the right surrounding image RG includes an image of the right edge of the screed 3 and an image of the right edge of the hopper 2 .

Distortion correction is image processing for correcting apparent distortion and tilt caused by using a wide-angle lens or the like. The image rotation processing is image processing for matching the directions of the forward direction of travel of the asphalt finisher 100 (upper side of the screen of the display device 52) with the directions of the left surrounding image LG and the right surrounding image RG. In this embodiment, the correspondence relationship between the coordinates on the input image plane and the coordinates on the output image plane for the input images of the left camera 51L and the right camera 51R incorporates the effects of distortion correction and image rotation processing in advance. are stored in the input image/output image correspondence map 54a. The distortion correction and image rotation processing may be performed on the hopper image HG.

The illustration image 1 CG is computer graphics of the tractor 1 and is displayed so that the driver can recognize the position of the tractor 1 . In this embodiment, the illustration image 1CG is placed in the lower center of the first output image.

In this way, the display device 52 can display the first output image showing the scene seen when looking down on the asphalt finisher 100 and its vicinity from the sky.

In the above-described embodiment, the hopper image HG, the left surrounding image LG, and the right surrounding image RG are arranged adjacently as independent images. However, the three images may be combined into one continuous image. In this case, image processing may be performed to prevent the image of the object from disappearing in the range where the imaging range of the front camera 51F and the imaging range of the left camera 51L or the right camera 51R overlap.

Further, in the above-described embodiment, each of the hopper image HG, the left surrounding image LG, and the right surrounding image RG is generated based on the input image captured by one corresponding camera. However, each of the hopper image HG, the left surrounding image LG, and the right surrounding image RG may be generated based on input images captured by two or more cameras. For example, the left surrounding image LG may be generated based on input images captured by the left camera 51L and the left auxiliary camera 51U. Also, the right surrounding image RG may be generated based on the input images captured by the right camera 51R and the right auxiliary camera 51V.

Next, with reference to FIGS. 4A to 4C, the second output image generated using the input images of the left camera 51L, right camera 51R, front camera 51F, left auxiliary camera 51U, and right auxiliary camera 51V will be described. do. 4A to 4C are display examples of the second output image displayed on the display device 52. FIG.

The second output image mainly includes a hopper image HG, a left surrounding image LG, a right surrounding image RG, an illustration image 1CG, and a local image SG. In this embodiment, the local image SG is superimposed on the illustration image 1CG. FIG. 4A shows the area surrounded by the retaining plate 70 (see FIG. 1B), the side plate 71 (see FIG. 1B), and the mold board 72 (see FIG. 1B) on the right side of the tractor 1. Fig. 2 shows a second output image comprising a right local image SGR showing local regions; FIG. 4B shows the area on the left side of the tractor 1 surrounded by a retaining plate 70 (see FIG. 1B), a side plate 71 (see FIG. 1B) and a mold board 72 (see FIG. 1B). Fig. 2 shows a second output image containing a left local image SGL showing a local area; FIG. 4C shows a second output image comprising a right local image SGR and a left local image SGL.

The right local image SGR is generated based on the input image of the right auxiliary camera 51V. In this embodiment, the right local image SGR is an image showing the appearance of the right local area seen when looking down on the right local area from the tractor 1 . Specifically, the right local image SGR is generated by cutting out a portion of the input image of the right auxiliary camera 51V, performing distortion correction, and further performing image rotation processing. It is arranged along the right edge of 1CG.

Left local image SGL is generated based on the input image of left auxiliary camera 51U. In this embodiment, the left local image SGL is an image showing the state of the left local area seen when looking down on the left local area from the tractor 1 . Specifically, the left local image SGL is generated by cutting out a portion of the input image of the left auxiliary camera 51U, performing distortion correction, and further performing image rotation processing. It is arranged along the left edge of 1CG.

The display changeover switch selects a first switch for displaying a second output image (see FIG. 4A) including the right local image SGR and a second output image (see FIG. 4B) including the left local image SGL. and a second switch for displaying. Alternatively, it may consist only of a switch for displaying the second output image (see FIG. 4C) including the right local image SGR and the left local image SGL. Alternatively, it may be configured to include those three switches.

The screw dial may be configured to include a right dial for adjusting the speed of rotation of the right screw and a left dial for adjusting the speed of rotation of the left screw so that the speed of rotation of the left and right screws can be adjusted simultaneously. It may consist only of a common dial for adjustment. Alternatively, it may be configured to include those three dials. For example, the highlighting unit 50B displays a second output image (see FIG. 4A) including the right local image SGR when the right dial is operated, and displays the left local image SGL when the left dial is operated. A second output image (see FIG. 4B) containing Alternatively, the second output image (see FIG. 4C) including the right local image SGR and the left local image SGL may be displayed when the common dial is operated.

The display frame surrounding the local image SG may be displayed differently from the display frame surrounding each of the hopper image HG, the left surrounding image LG, the right surrounding image RG, and the illustration image 1CG. For example, they may be displayed in different colors, line types, thicknesses, etc., or may be blinked.

Next, referring to FIG. 5, the process of generating an output image by the image generating system SYS (hereinafter referred to as "output image generating process") will be described. FIG. 5 is a flowchart of output image generation processing. The output images include a first output image and a second output image. The image generation system SYS repeatedly executes this output image generation process at a predetermined control cycle to alternatively generate one of the first output image and the second output image. However, the image generation system SYS may generate both the first output image and the second output image.

First, the output image generator 50A of the controller 50 associates the coordinate values on the output image plane with the coordinate values on the input image plane (step S1). In this embodiment, the input image/output image correspondence map 54a is referenced, and the values of the coordinates on the input image plane corresponding to the coordinates on the output image plane (for example, the luminance value, hue value, saturation value, etc.) ) is acquired, and the acquired value is set as the value of each coordinate on the corresponding output image plane.

After that, the controller 50 determines whether or not all coordinate values on the output image plane have been associated with coordinate values on the input image plane (step S2).

Then, when it is determined that all the coordinate values are not associated yet (NO in step S2), the output image generation unit 50A repeats the processes of steps S1 and S2.

If it is determined that all coordinate values have been associated (YES in step S2), the output image generation unit 50A terminates the current output image generation process.

Next, referring to FIG. 6, the image generation system SYS performs processing for switching the output image displayed on the display device 52 between the first output image and the second output image (hereinafter referred to as "output image switching processing"). ) will be explained. FIG. 6 is a flowchart of output image switching processing. The image generation system SYS repeatedly executes this output image switching process at a predetermined control cycle.

First, the highlighting unit 50B of the controller 50 determines whether or not the highlighting has been turned on (step S11). The highlighting unit 50B determines that the highlighting has been turned on when, for example, the display changeover switch as the highlighting switch is pressed while the first output image is being displayed on the display device 52 . It may be determined that the highlighting is turned on when the screw dial is operated.

When it is determined that the highlighting has been turned on (YES in step S11), the highlighting unit 50B highlights the local image (step S12). For example, the highlighting unit 50B switches the first output image (see FIG. 3) displayed on the display device 52 to the second output image (see FIGS. 4A to 4C) to display the left local image SGL and the right local image SGL. At least one of the local images SGR is displayed together with the ambient image.

When it is determined that the highlight display has not been turned on (NO in step S11), the highlight display unit 50B does not switch the first output image displayed on the display device 52 to the second output image, and displays the first output image. Continue displaying the image.

After that, the highlighting unit 50B determines whether or not the highlighting has been turned off (step S13). The highlighting unit 50B determines that highlighting has been turned off when, for example, the display changeover switch as the highlighting switch is pressed while the second output image is being displayed on the display device 52 . It may be determined that the highlighted display has been turned off when a predetermined period of time has elapsed since the operation of the screw dial was completed. Alternatively, it may be determined that the highlighted display has been turned off when a predetermined time has elapsed since the display changeover switch was pressed.

When it is determined that the highlighting has been turned off (YES in step S13), the highlighting unit 50B stops highlighting the local image (step S14). The highlighting unit 50B, for example, switches the second output image displayed on the display device 52 to the first output image and stops highlighting the local image.

When it is determined that the highlighting has not been turned off (NO in step S13), the highlighting unit 50B does not switch the second output image displayed on the display device 52 to the first output image, and displays the second output image. Continue displaying the image.

With this configuration, the image generation system SYS can display a local image on the display device 52 in response to a driver's request. By viewing this local image, the driver can view the local area while seated on the driver's seat 1S without moving around on the tractor 1 or twisting in the driver's seat 1S to look into the local area. It is possible to check the amount of paving material held. In addition, the surrounding conditions and the amount of paving material held in a local area can be confirmed almost simultaneously without moving the line of sight significantly. In this way, the asphalt finisher 100 equipped with the image generation system SYS can reduce the driver's fatigue due to the work of confirming the amount of holding. As a result, the safety of the asphalt finisher 100 can be improved.

Next, another display example of the second output image will be described with reference to FIG. FIG. 7 is another display example of the second output image displayed on the display device 52 . The second output image in FIG. 7 is different from the second output image in FIG. 4A in that the right local image SGR is superimposed on the right surrounding image RG instead of the illustration image 1CG, but in other respects they are common. do. Therefore, the description of the common parts is omitted, and the different parts are explained in detail. The following description relates to the right local image SGR, but applies equally to the left local image SGL.

In the example of FIG. 7, the right local image SGR is generated based on the input image of the right camera 51R, like the right surrounding image RG. Therefore, the right auxiliary camera 51V may be omitted. However, the right local image SGR may be an image generated based on the input image of the right auxiliary camera 51V.

The right local image SGR corresponds to part of the right surrounding image RG. For example, when the right surrounding image RG is composed of the first image portion RG1 to the eleventh image portion RG11, the right local image SGR in FIG. 7 corresponds to the ninth image portion RG9. Specifically, the image generation system SYS displays the right local image SGR, which is an image obtained by vertically enlarging the ninth image portion RG9, where the seventh image portion RG7 to the eleventh image portion RG11 have been displayed. are superimposed. That is, the first image portion RG1 to sixth image portion RG6 are displayed as they are, and the seventh image portion RG7 to eleventh image portion RG11 are blocked by the right local image SGR and become invisible.

In this way, the image generation system SYS displays the right local image SGR superimposed on the image portion of the right surrounding image RG in which the right local area was shown. Therefore, the driver can intuitively recognize that the right local area is shown in the right local image SGR. Further, by displaying the right local image SGR, the right local area can be enlarged and displayed more than when the right surrounding image RG is displayed. Therefore, the state of the right local area to which the pavement material is supplied by the right screw can be presented to the driver in an easy-to-understand manner.

Next, still another display example of the second output image will be described with reference to FIG. FIG. 8 is still another display example of the second output image displayed on the display device 52 . The second output image of FIG. 8 differs from the second output image of FIG. 7 in that the right local image SGR is displayed so as to cover the entire right surrounding image RG rather than part of it. Common. Therefore, the description of the common parts is omitted, and the different parts are explained in detail. The following description relates to the right local image SGR, but applies equally to the left local image SGL.

The right local image SGR corresponds to part of the right surrounding image RG, as in FIG. For example, when the right surrounding image RG is composed of the first image portion RG1 to the eleventh image portion RG11, the right local image SGR in FIG. 8 corresponds to the ninth image portion RG9. Specifically, the image generation system SYS displays the right local image SGR, which is an image obtained by vertically enlarging the ninth image portion RG9, where the first image portion RG1 to the eleventh image portion RG11 have been displayed. are superimposed. That is, the first image portion RG1 to the eleventh image portion RG11 are blocked by the right local image SGR and become invisible.

In this manner, the image generation system SYS superimposes and displays the right local image SGR over the entire right surrounding image RG including the image portion in which the right local area was shown. Therefore, the driver can intuitively recognize that the right local area is shown in the right local image SGR. In addition, by displaying the right local image SGR having a size that covers the entire vertical length of the display device 52, the right local area can be displayed even larger than in the case of the second output image in FIG. Therefore, the state of the right local area to which the pavement material is supplied by the right screw can be presented to the driver in a more comprehensible manner.

Next, still another display example of the second output image will be described with reference to FIG. FIG. 9 is still another display example of the second output image displayed on the display device 52 . The second output image of FIG. 9 differs from the second output image of FIG. 8 in that the right local image SGR is generated using the entire right surrounding image RG rather than part of it, and in that the indicator BG is displayed. Although different from the output image, they are common in other respects. Therefore, the description of the common parts is omitted, and the different parts are explained in detail. The following description relates to the right local image SGR, but applies equally to the left local image SGL.

The right local image SGR corresponds to the entire right surrounding image RG, unlike the case of FIG. For example, when the right surrounding image RG is composed of the first image portion RG1 to the eleventh image portion RG11, the right local image SGR in FIG. It is generated by enlarging or shrinking to . Specifically, an image obtained by vertically reducing each of the first image portion RG1 to sixth image portion RG6 and an eleventh image portion RG11, and each of the seventh image portion RG7 to tenth image portion RG10 are vertically reduced. It consists of an enlarged image. 7 and 8, the scene shown in the right surrounding image RG can be continuously viewed even when the right local image SGR is displayed.

The indicator BG is a graphical image that represents the scaling status of the image parts that make up the local image SG relative to the corresponding image parts that make up the surrounding image. In the example of FIG. 9, the right indicator BGR represents the scaling state of the image portion making up the right local image SGR relative to the corresponding image portion making up the right surrounding image RG. Specifically, the right indicator BGR is a bar-shaped indicator extending in the vertical direction, which is composed of eleven rectangular segments corresponding to the first image portion RG1 to the eleventh image portion RG11, respectively, and is displayed at the right end of the screen. ing. When the left indicator is displayed, the left indicator may be displayed on the left edge of the screen. The left indicator represents the scaling state of the image portion making up the left local image SGL relative to the corresponding image portion making up the left surrounding image LG. In either case, the bar indicators indicate that the longer the rectangular segment, the greater the magnification, and the shorter the rectangular segment, the greater the reduction. However, display of the indicator BG may be omitted.

In this manner, the image generation system SYS displays the right local image SGR superimposed on the entire right surrounding image RG including the image portion in which the right local area was shown, as in the case of FIG. Therefore, the driver can intuitively recognize that the right local area is shown in the right local image SGR. Also, as in the case of FIG. 8, by displaying the right local image SGR over the entire vertical length of the display device 52, the right local area can be displayed larger than in the case of the second output image of FIG. Therefore, the state of the right local area to which the pavement material is supplied by the right screw can be presented to the driver in an easy-to-understand manner. Also, unlike the case of FIG. 8, the driver can continuously view the scene shown in the right surrounding image RG while the right local area is displayed in a large size. Therefore, the driver can check the state of the right local area while monitoring the worker on the right side of the asphalt finisher 100, for example.

With the above configuration, the image generation system SYS can make the driver intuitively recognize the positional relationship between the asphalt finisher 100 and workers working around it based on the input images acquired by the multiple cameras. output image.

In addition, the image generation system SYS displays the hopper image HG, the left surrounding image LG, and the right surrounding image RG so that the driver can be presented with an image showing the scene seen when looking down at the asphalt finisher 100 and its vicinity from the sky. , and an illustration image 1CG are displayed. Accordingly, the driver can visually recognize blind spots around the asphalt finisher 100 without leaving the driver's seat 1S. As a result, the image generation system SYS can improve the safety and operability of the asphalt finisher 100 . Specifically, the image generation system SYS can present the remaining amount of pavement material in the hopper 2, the position of a feature (for example, a manhole) on the road surface to be paved, and the like to the driver. In addition, the image generation system SYS can present the positions of workers working around the asphalt finisher 100 to the driver. Therefore, the driver can perform various operations such as opening and closing the hopper 2, extending and retracting the screed 3, and raising and lowering the screed 3 after confirming the positions of the workers, etc., by looking at the display device 52. FIG. In addition, when the driver perceives danger due to the positional relationship between the worker and the hopper, screed, or dump truck, he can stop various operations, stop the asphalt finisher, and the like.

In addition, the image generation system SYS displays a surrounding image, which is an image showing the surroundings of the asphalt finisher 100, and emphasizes a local image, which is an image showing an area where pavement material is supplied by the screws SC. indicate. The highlighting of the local image includes displaying the local image in another display frame, enlarging and displaying the local image, changing the display mode of the display frame of the local image, and the like. With this processing, the image generation system SYS can present the situation of the predetermined local area in addition to the situation around the asphalt finisher 100 in an easy-to-understand manner to the driver.

The image generation system SYS includes a right camera 51R as a first camera that captures an image of an area to the right of the asphalt finisher 100, and a right auxiliary camera 51V as a second camera that captures an image of an area where pavement material is supplied by the right screw. and may be provided. In this case, the right surrounding image RG forming the surrounding image is generated based on the image captured by the right camera 51R, and the right local image SGR forming the local image is based on the image captured by the right auxiliary camera 51V. may be generated by

The image generation system SYS also includes a left camera 51L as a first camera that captures an area to the left of the asphalt finisher 100, and a left auxiliary camera 51L as a second camera that captures an area where pavement material is supplied by the left screw. A camera 51U may be provided. In this case, the left surrounding image LG forming the surrounding image is generated based on the image captured by the left camera 51L, and the left local image SGL forming the local image is generated based on the image captured by the left auxiliary camera 51U. may be generated by

The image generation system SYS may include a right camera 51R that captures an area to the right of the asphalt finisher 100 and a right local area to which paving material is fed by the right screw. In this case, the right surrounding image RG and the right local image SGR may be generated based on the images captured by the right camera 51R. In this case, the right auxiliary camera 51V may be omitted.

The image generation system SYS may also include a left camera 51L that captures an area to the left of the asphalt finisher 100 and a left local area to which pavement material is supplied by the left screw. In this case, the left surrounding image LG and the left local image SGL may be generated based on the images captured by the left camera 51L. In this case, the left auxiliary camera 51U may be omitted.

The local image may be displayed so as not to overlap the surrounding image, for example as shown in FIGS. 4A-4C, or may be displayed so as to overlap the surrounding image as shown in FIGS. 6-8, for example. .

The display device 52 may display an indicator BG representing the scaling state of the local image SG. By looking at the indicator BG, the driver can instantly grasp the enlargement/reduction state of each of the image portions forming the local image SG.

2. Description of the Related Art Conventionally, an asphalt finisher is known in which images captured by a front camera, a left camera, and a right camera are displayed side by side around computer graphics of a tractor (see Patent Document 1). A front camera is mounted at the front top of the tractor for imaging the interior of the hopper in front of the tractor. A left camera is mounted on the top left side of the tractor to image the space to the left of the asphalt finisher. A right camera is mounted on the top of the right side of the tractor to image the space to the right of the asphalt finisher.

However, in the above configuration, the space around the front end of the hopper wing is a blind spot for the hopper wing when viewed from the front camera, left camera, and right camera. Therefore, the operator of the asphalt finisher cannot grasp the state of the space around the front end of the hopper wing even by looking at the displayed image. For a space that cannot be visually recognized in the image, the operator needs to confirm safety by direct visual inspection.

In view of the above, it would be desirable to provide a road machine that further reduces the range invisible in the image.

FIG. 10 is a side view of an asphalt finisher 100, which is an example of a road machine according to an embodiment of the invention. FIG. 11 is a top view of the asphalt finisher 100. FIG. The asphalt finisher 100 is mainly composed of a tractor 1, a hopper 2 and a screed 3. Hereinafter, the direction of the hopper 2 (+X direction) viewed from the tractor 1 is defined as the front, and the direction of the screed 3 (-X direction) viewed from the tractor 1 is defined as the rear.

The tractor 1 is a vehicle for running the asphalt finisher 100 . In this embodiment, the tractor 1 moves the asphalt finisher 100 by rotating the rear wheels 5 using the rear wheel travel hydraulic motor and rotating the front wheels 6 using the front wheel travel hydraulic motor. The rear-wheel traveling hydraulic motor and the front-wheel traveling hydraulic motor are supplied with hydraulic oil from the hydraulic pump to rotate. The rear wheels 5 and front wheels 6 may be replaced by crawlers. The tractor 1 also includes a canopy 1C. The canopy 1C is attached to the top of the tractor 1.

The controller 50 is a control device that controls the asphalt finisher 100 . In this embodiment, the controller 50 comprises a microcomputer including a CPU, a volatile memory device, a non-volatile memory device, etc., and is mounted on the tractor 1 . Various functions of the controller 50 are implemented by the CPU executing programs stored in the nonvolatile storage device.

Hopper 2 is a mechanism for receiving paving material and mainly includes hopper wings 20 and hopper cylinders 24 . In this embodiment the hopper 2 is mounted on the front side of the tractor 1 and receives pavement material within the hopper wings 20 . The hopper wing 20 includes a left hopper wing 20L that can be opened and closed in the Y-axis direction (vehicle width direction) by a left hopper cylinder 24L, and a right hopper wing 20R that can be opened and closed in the Y-axis direction (vehicle width direction) by a right hopper cylinder 24R. including. The asphalt finisher 100 normally receives paving material (eg, an asphalt mixture) from the dump truck bed with the hopper wings 20 fully open. FIG. 11 shows the hopper wings 20 fully open. When the pavement material in the hopper 2 is reduced, the hopper wing 20 is closed, and the pavement material near the inner wall of the hopper 2 is collected in the central portion of the hopper 2.例文帳に追加This is so that the conveyor CV in the central part of the hopper 2 can continuously feed the paving material behind the tractor 1 . That is, this is to maintain the state in which the pavement material is accumulated on the conveyor CV. After that, the pavement material fed to the rear side of the tractor 1 is spread in the vehicle width direction on the rear side of the tractor 1 and the front side of the screed 3 by the screw SC. In this embodiment, the screw SC is in a state in which the extension screws are connected to the left and right.

The screed 3 is a mechanism for spreading the paving material evenly. In this example the screed 3 comprises a front screed 30 and a rear screed 31 . Screed 3 is a floating screed towed by tractor 1 and is connected to tractor 1 via leveling arms 3A. The rear screeds 31 include a left rear screed 31L and a right rear screed 31R. The left rear side screed 31L is expanded and contracted in the vehicle width direction using the left screed telescopic cylinder 26L, and the right rear side screed 31R is expanded and contracted in the vehicle width direction using the right screed telescopic cylinder 26R.

The imaging device 51 is a device that acquires an image. In this embodiment, the imaging device 51 is a monocular camera and is connected to the controller 50 wirelessly or by wire. For example, the controller 50 can generate a bird's-eye view image by performing viewpoint conversion processing on the image captured by the imaging device 51 . The overhead image is, for example, an image of the space around the asphalt finisher 100 virtually viewed from directly above. The imaging device 51 may be a stereo camera. In this embodiment, the imaging device 51 includes a front camera 51F, a left camera 51L, a right camera 51R, and a rear camera 51B. The rear camera 51B may be omitted.

The front camera 51F images the space in front of the asphalt finisher 100 . In this embodiment, the front portion of the tractor 1 is configured so that an image of the inside of the hopper 2, which is a blind spot, can be captured from the viewpoint of the driver sitting in the driver's seat 1S (hereinafter referred to as "driver's seat viewpoint"). attached to the bonnet. It may be attached to the front edge of the top plate of the canopy 1C. A gray area Z1 in FIG. 11 indicates the imaging range of the front camera 51F.

The left camera 51L images the space to the left of the asphalt finisher 100 . In this embodiment, a rod member extending in the +Y direction (leftward) from the left edge of the top plate of the canopy 1C is used so that an image of the space outside the left hopper wing 20L in the vehicle width direction, which is a blind spot from the viewpoint of the driver's seat, can be captured. Attached to the tip of BL. It may be attached to the tip of a rod member BL extending in the +Y direction (leftward) from the right side of the tractor 1 . The left camera 51L is mounted, for example, so as to protrude outward (to the left) in the vehicle width direction from the left end of the fully opened left hopper wing 20L. A gray area Z2 in FIG. 11 indicates the imaging range of the left camera 51L.

The right camera 51R images the space to the right of the asphalt finisher 100. FIG. In this embodiment, a rod extending in the -Y direction (rightward) from the right edge of the top plate of the canopy 1C is used so that an image of the space outside the right hopper wing 20R in the vehicle width direction, which is a blind spot from the driver's seat viewpoint, can be captured. It is attached to the tip of the member BR. It may be attached to the tip of a rod member BR extending from the right side of the tractor 1 in the -Y direction (rightward). The right camera 51R is mounted, for example, so as to protrude outward (to the right) in the vehicle width direction from the right end of the fully opened right hopper wing 20R. A gray area Z3 in FIG. 11 indicates the imaging range of the right camera 51R.

The bar members BL, BR are desirably configured to be removable. It may be configured to be stretchable. This is for coping with the case where the asphalt finisher 100 is transported by a trailer or the like.

The rear camera 51B captures an image of the space behind the asphalt finisher 100 . In this embodiment, it is attached to the rear edge of the top plate of the canopy 1C so that the space behind the screed 3, which is a blind spot from the viewpoint of the driver's seat, can be imaged. A gray area Z4 in FIG. 11 indicates the imaging range of the rear camera 51B.

The imaging range of the front camera 1F and the imaging range of the left camera 1L may overlap. Also, the imaging range of the rear camera 1B and the imaging range of the left camera 1L do not have to overlap. The same applies to the imaging range of the right camera 1R.

The controller 50 converts the viewpoint of the images captured by the front camera 51F, the left camera 51L, the right camera 51R, and the rear camera 51B and synthesizes them to generate a bird's-eye view image. The bird's-eye view image is an image in which the space inside the hopper 2, the space to the left of the left hopper wing 20L, the space to the right of the right hopper wing 20R, and the space behind the screed 3 are virtually viewed from directly above. and a computer graphics image of the asphalt finisher 100 (hereinafter referred to as "model image"). The controller 50 may generate a bird's-eye view image by performing viewpoint conversion and synthesizing images captured by each of the three cameras, the front camera 51F, the left camera 51L, and the right camera 51R. That is, the overhead image may be generated without using the rear camera 51B.

The display device 52 is a device that displays various images. In this embodiment, the display device 52 is a liquid crystal display and is connected to the controller 50 wirelessly or by wire. The display device 52 can display an image captured by each of the plurality of imaging devices 51, and is arranged at a position where the driver sitting in the driver's seat 1S can easily see the image. It may be located at the rear controller. The controller 50 displays, on the display device 52, an image generated by, for example, subjecting the image captured by the imaging device 51 to viewpoint conversion processing.

Next, the display system GS mounted on the asphalt finisher 100 will be described with reference to FIG. FIG. 12 is a block diagram showing a configuration example of the display system GS. The display system GS mainly includes a controller 50, an imaging device 51, a display device 52, an information acquisition device 53, a storage device 54, and the like. The display system GS, for example, generates an image for display (hereinafter referred to as an "output image") based on an image captured by the imaging device 51 (hereinafter referred to as an "input image"), and outputs the output image. It is displayed on the display device 52 .

The information acquisition device 53 acquires information and outputs the acquired information to the controller 50 . The information acquisition device 53 includes, for example, at least one of a hopper cylinder stroke sensor, a screed telescopic cylinder stroke sensor, a steering angle sensor, a travel speed sensor, a positioning sensor, and the like. A hopper cylinder stroke sensor detects the stroke amount of the hopper cylinder 24 . The screed telescopic cylinder stroke sensor detects the stroke amount of the screed telescopic cylinder 26 . A steering angle sensor detects the steering angle of the front wheels 6 . The running speed sensor detects the running speed of the asphalt finisher 100 . The positioning sensor is, for example, a GNSS compass, and detects the position (latitude, longitude, altitude) and orientation of the asphalt finisher 100 .

The storage device 54 is a device for storing various information. In this embodiment, the storage device 54 is a nonvolatile storage device that stores an input image/output image correspondence map 54a in a referable manner.

The input image/output image correspondence map 54a stores the correspondence relationship between the coordinates on the input image plane and the coordinates on the output image plane. The corresponding relationship is set in advance based on various parameters such as the optical center of the imaging device 51, the focal length, the CCD size, the optical axis direction vector, the camera horizontal direction vector, and the projection method so as to realize desired viewpoint conversion. there is The correspondence relationship is set so that apparent distortion and tilt do not appear in the output image.

The controller 50 includes a viewpoint conversion unit 50a and an auxiliary line creation unit 50b. The viewpoint conversion unit 50a and the auxiliary line creation unit 50b are configured by software, hardware, or firmware.

The viewpoint conversion unit 50a is a functional element that generates an output image. In the present embodiment, the input image/output image correspondence map 54a stored in the storage device 54 is referred to, and the coordinates on the input image plane where the input image captured by the imaging device 51 is located and the bird's-eye view image as the output image are obtained. Corresponds with the coordinates on the output image plane where it is located. Specifically, the viewpoint conversion unit 50a associates the value of each pixel in the input image (for example, luminance value, hue value, saturation value, etc.) with the value of each pixel in the output image, and converts the output image. Generate.

The auxiliary line creation unit 50b is a functional element that creates auxiliary lines superimposed on the output image. In this embodiment, the auxiliary line creation unit 50b creates auxiliary lines so as to match the bird's-eye view image generated by the viewpoint conversion unit 50a. The auxiliary lines include, for example, an auxiliary line indicating an expected paved trajectory that is an expected trajectory of the end of the screed 3, an auxiliary line indicating an expected traveling trajectory that is an expected trajectory of the wheels, and the like.

In this embodiment, the controller 50 refers to the input image/output image correspondence map 54a by the viewpoint conversion unit 50a. Then, the values of the coordinates on the input image plane corresponding to the coordinates on the output image plane (for example, brightness, hue, saturation, etc.) are acquired, and the acquired values are converted to the corresponding coordinates. It is adopted as the value of each coordinate on the output image plane.

Controller 50 then determines whether it has associated the values of all the coordinates on the output image plane with the values of the coordinates on the input image plane. If it is determined that all coordinate values have not been associated yet, the above processing is repeated.

On the other hand, when it is determined that all the coordinate values are associated, the controller 50 superimposes the auxiliary line indicating the predicted pavement path, the auxiliary line indicating the predicted traveling path, etc. on the output image by the auxiliary line creating unit 50b. do. In this embodiment, the position on the output image where the auxiliary line is superimposed is set in advance, but may be dynamically derived. Further, after displaying the auxiliary lines, the controller 50 may associate the coordinates on the input image plane with the coordinates on the output image plane.

Next, referring to FIGS. 13A and 13B, input images captured by the four imaging devices 51 (front camera 51F, left camera 51L, right camera 51R, and rear camera 51B) mounted on the asphalt finisher 100. A bird's-eye view image generated using is described. 13A and 13B are diagrams showing examples of overhead images. Specifically, FIG. 13A shows an example of a bird's-eye view image displayed on the display device 52. FIG. FIG. 13B shows the segmentation of the input image used to generate the overhead image of FIG. 13A.

The bird's-eye view image of FIG. 13A is an image of the space around the asphalt finisher 100 virtually viewed from directly above. The bird's-eye view image of FIG. 13A mainly includes an image G1 (see hatched area) generated by the viewpoint conversion unit 50a and a model image CG1.

The model image CG1 is an image representing the asphalt finisher 100, and includes a model image CGa of the hopper 2, a model image CGb of the tractor 1, and a model image CGc of the screed 3.

The model image CGa of the hopper 2 includes a model image WL of the left hopper wing 20L and a model image WR of the right hopper wing 20R. The model images WL and WR change in shape according to the output of the hopper cylinder stroke sensor. FIG. 13A shows the model image CGa when each of the left hopper wing 20L and the right hopper wing 20R is fully open. A portion of the bird's-eye view image (an image of the inside of the hopper 2 virtually viewed from directly above) is arranged in the hatched area between the model image WL and the model image WR. The model image CGa of the hopper 2 may be omitted. In this case, part of the overhead image based on the image captured by the front camera 1F is arranged.

The model image CGc of the screed 3 includes a model image SL of the left rear screed 31L and a model image SR of the right rear screed 31R. The model images SL and SR change in shape according to the output of the screed telescopic cylinder stroke sensor. FIG. 13A shows the model image CGc when each of the left rear screed 31L and the right rear screed 31R is stretched to the maximum. The model image CGc of the screed 3 may be omitted. In this case, part of the overhead image based on the image captured by the rear camera 1B is arranged.

The image G1 is an image generated using input images captured by each of the four imaging devices 51 . In this embodiment, the image G1 includes an image Ga of a worker present in front left of the asphalt finisher 100 and an image Gb of a manhole cover present in front right of the asphalt finisher 100 . The controller 50 synthesizes the front image R1, the left image R2, the right image R3, and the rear image R4 to generate the image G1, as shown in FIG. 13B. The image Ga of the worker is included in the left image R2, and the image Gb of the manhole cover is included in the right image R3.

The previous image R1 is an image generated based on the input image captured by the front camera 51F. In this embodiment, the front image R1 includes an image showing the inside of the hopper 2 as viewed from the tractor 1 side. The controller 50 cuts out a part of the input image captured by the front camera 51F and performs viewpoint conversion processing to generate the front image R1. The previous image R1 is arranged between and above the model image WL and the model image WR. The shape of the previous image R1 may change according to changes in the shapes of the model images WL and WR.

The left image R2 is an image generated based on the input image captured by the left camera 51L. In this embodiment, the left image R2 includes an image of the space outside (on the left side of) the left hopper wing 20L in the vehicle width direction. The controller 50 cuts out a part of the input image captured by the left camera 51L and performs viewpoint conversion processing by the viewpoint conversion unit 50a to generate the left image R2. The left image R2 is arranged on the left side of the model image CG1. The shape of the left image R2 may change according to changes in the shapes of the model images WL and SL.

The right image R3 is an image generated based on the input image captured by the right camera 51R. In this embodiment, the right image R3 includes an image of the space outside (on the right side of) the right hopper wing 20R in the vehicle width direction. The controller 50 cuts out a part of the input image captured by the right camera 51R and performs viewpoint conversion processing by the viewpoint conversion unit 50a to generate the right image R3. The right image R3 is arranged on the right side of the model image CG1. The shape of the right image R3 may change according to changes in the shapes of the model images WR and SR.

The rear image R4 is an image generated based on the input image captured by the rear camera 51B. In this embodiment, the rear image R4 includes an image showing the screed 3 viewed from the tractor 1 side. The controller 50 cuts out a part of the input image captured by the rear camera 51B and performs viewpoint conversion processing to generate a rear image R4. The rear image R4 is arranged below the model image CGc of the screed 3 . The shape of the post-image R4 may change according to changes in the shapes of the model images SL and SR.

In this embodiment, the correspondence relationship between the coordinates on the input image plane and the coordinates on the output image plane of each of the four cameras is obtained in the input image/output image correspondence map 54a in a state in which the effect of viewpoint conversion processing is incorporated in advance. stored in Therefore, the controller 50 can associate coordinates on a plurality of input image planes with coordinates on the output image plane simply by referring to the input image/output image correspondence map 54a. As a result, the controller 50 can generate and display output images with a relatively low computational load.

Also, in this embodiment, each of the front image R1, the left image R2, the right image R3, and the rear image R4 is generated based on an input image captured by one corresponding camera. However, the invention is not limited to this configuration. For example, each of the front image R1, the left image R2, the right image R3, and the rear image R4 may be generated based on input images captured by two or more cameras.

As described above, the asphalt finisher 100 includes the plurality of imaging devices 51 attached to the tractor 1, the controller 50 that converts the viewpoint of the images captured by the plurality of imaging devices 51 and synthesizes them to generate a bird's-eye view image, and a display device 52 for displaying a bird's-eye view image on one screen. The bird's-eye view image is configured to include an image of the space around the asphalt finisher 100 virtually viewed from directly above. Therefore, it is possible to further reduce the blind spot, which is a range that cannot be visually recognized in the output image. As a result, for example, the state of the space around the front end of the hopper wing 20 can be presented to the operator of the asphalt finisher 100 . Alternatively, the state of the pavement inside the hopper 2, the state of the space behind the screed 3, and the like can be presented to the operator.

In addition, the asphalt finisher 100 can improve visibility, safety, operability, and workability by presenting a bird's-eye view image. Specifically, the asphalt finisher 100 allows the operator to intuitively recognize the remaining amount of pavement material in the hopper 2, the position of a feature (for example, a manhole) on the road surface to be paved, and the like. . Also, the operator can intuitively recognize the position of the worker working around the hopper 2 . Therefore, the operator can perform various operations such as opening and closing the hopper wing 20 after confirming the location of the feature and the operator by looking at the bird's-eye view image.

Next, another example of the output image will be described with reference to FIGS. 14A and 14B. 14A and 14B show examples in which auxiliary lines are superimposed on the image G1 generated by the viewpoint conversion unit 50a. Specifically, FIG. 14A shows a bird's-eye view image of the asphalt finisher 100 about to go straight. FIG. 14B shows an overhead image of the asphalt finisher 100 about to turn right.

In the example of FIGS. 14A and 14B, the auxiliary line creation unit 50b creates auxiliary lines L1 and L2 based on the respective outputs of the steering angle sensor and the running speed sensor. Auxiliary line L1 is the estimated running locus of left rear wheel 5L, and auxiliary line L2 is the estimated running locus of right rear wheel 5R. In this embodiment, the auxiliary lines L1 and L2 are derived based on the current steering angle and running speed. It may be derived based on the steering angle only. Auxiliary lines L1 and L2 represent, for example, the travel trajectory of the period from the current time until a predetermined time (for example, several tens of seconds) elapses.

Further, the auxiliary line creation unit 50b additionally refers to the output of the screed telescopic cylinder stroke sensor to create the auxiliary lines L3 and L4. The auxiliary line L3 is the predicted trajectory of the left end of the left rear screed 31L, and the auxiliary line L4 is the predicted trajectory of the right end of the right rear screed 31R. In this embodiment, the auxiliary lines L3 and L4 are derived based on the current steering angle, travel speed, and stroke amount of the screed telescopic cylinder 26 . Auxiliary lines L3 and L4 represent, for example, trajectories of a period from the present time until a predetermined time (for example, several tens of seconds) has passed.

Furthermore, the auxiliary line creation unit 50b creates auxiliary lines L5 and L6 based on the road design data and the output of the positioning sensor. Road design data is data relating to roads to be constructed, and is stored in advance in, for example, a non-volatile storage device. The road design data includes, for example, data regarding the positions of features on the road surface to be paved. The auxiliary line L5 represents the left edge of the construction target road, and the auxiliary line L6 represents the right edge of the construction target road.

Furthermore, the controller 50 superimposes an image of a feature such as a manhole on the bird's-eye view image based on the road design data and the output of the positioning sensor. In the example of FIGS. 14A and 14B, the controller 50 superimposes the model image CG2 of the manhole cover on the overhead image.

The operator of the asphalt finisher 100 can determine whether the current steering angle, traveling speed, expansion/contraction amount of the rear screed 31, etc. of the asphalt finisher 100 are appropriate by viewing the bird's-eye view image on which auxiliary lines and the like are superimposed as described above. can determine whether there is For example, by looking at the bird's-eye view image of FIG. 14A, the operator can recognize that if the asphalt finisher 100 is allowed to move straight ahead, the right rear wheel 5R will run over the manhole cover and the road will not be paved as designed. . On the other hand, by looking at the bird's-eye view image of FIG. 14B, the operator can avoid the right rear wheel 5R from running over the manhole cover if the current steering state (the state in which the steering wheel is turned to the right) and the running speed are maintained. And, it can be recognized that the road is paved as designed.

In the example of FIGS. 14A and 14B , the controller 50 displays the expected running trajectory of the rear wheels 5, but the expected running trajectory of the front wheels 6 may be displayed. A running locus may be displayed.

With the above-described configuration, the controller 50 can present to the operator in advance the movement of the asphalt finisher 100 during the period from the current time to the elapse of a predetermined time, in addition to the effect of the bird's-eye view image described with reference to FIGS. 13A and 13B. additional effect can be achieved.

A preferred embodiment of the present invention has been described above. However, the invention is not limited to the embodiments described above. Various modifications, substitutions, etc., may be applied to the above-described embodiments without departing from the scope of the present invention. Also, each of the features described with reference to the above embodiments may be combined as appropriate as long as they are not technically inconsistent.

For example, in the above-described embodiment, the image portion forming the local image SG is scaled only in the vertical direction, but it may be scaled only in the horizontal direction. good too. If the image portion is scaled only laterally, the indicator BG may be a bar-like indicator extending laterally. If the image portion is scaled vertically and horizontally, the indicator BG may be a matrix-like indicator extending vertically and horizontally.

Alternatively, the asphalt finisher 100 may be a goose asphalt finisher using a goose asphalt mixture. The image generation system SYS may be mounted on a goose asphalt finisher that uses goose asphalt mix.

This application claims priority based on Japanese Patent Application No. 2017-153668 filed on August 8, 2017 and priority based on Japanese Patent Application No. 2017-164687 filed on August 29, 2017 The entire contents of these Japanese patent applications are incorporated herein by reference.

DESCRIPTION OF SYMBOLS 1... Tractor 1S... Driver's seat 2... Hopper 3... Screed 3A... Leveling arm 5... Rear wheel 6... Front wheel 20... Hopper wing 20L... Left hopper Wing 20R Right hopper wing 24 Hopper cylinder 24L Left hopper cylinder 24R Right hopper cylinder 26 Screed telescopic cylinder 26L Left screed telescopic cylinder 26R Right screed telescopic Cylinder 30... Front side screed 31... Rear side screed 31L... Left rear side screed 31R... Right rear side screed 50... Controller 50A... Output image generation section 50B... Emphasis display section 50a... View point conversion unit 50b... Auxiliary line creation unit 51... Imaging device 51B... Rear camera 51F... Front camera 51L... Left camera 51R... Right camera 51U... Left Auxiliary camera 51V... Right auxiliary camera 52... Display device 53... Information acquisition device 54... Storage device 54a... Input image/output image correspondence map 55... Input device 65... Operation Panel 70... Retaining plate 71... Side plate 72... Mold board 100... Asphalt finisher BL, BR... Bar member CV... Conveyor SC... Screw SYS... Image generation system

Claims (7)

A road machine comprising a tractor and a screed arranged behind the tractor,
the screeds include a front screed, a right rear screed, and a left rear screed;
The road machine is
a working device that supplies paving material to a region in front of the right rear screed and a region in front of the left rear screed ;
A surrounding image that is an image showing the surroundings of the road machine is displayed, and at least one of an area in front of the right rear screed and an area in front of the left rear screed is shown. a display device that highlights a local image, which is an image, relative to the surrounding image;
road machinery. a first camera that captures an area to the side of the road machine;
A right second camera that captures an area in front of the right rear screed , and a left second camera that captures an area in front of the left rear screed ,
The surrounding image is generated based on an image captured by the first camera,
The local image is generated based on at least one of an image captured by the second right camera and an image captured by the second left camera.
Road machine according to claim 1. a camera for imaging an area to the side of the road machine and an area to which paving material is supplied by the working device;
wherein the ambient image and the local image are generated based on images captured by the camera;
Road machine according to claim 1. the local image is displayed so as not to overlap the surrounding image;
Road machine according to claim 1. the local image is displayed so as to overlap the surrounding image;
Road machine according to claim 1. the display device displays an indicator representing a scaling state of the local image;
Road machine according to claim 1. The left rear screed and the right rear screed are offset from each other in the direction of travel,
Road machine according to claim 1.

Images