JP2023113486A - Imaging system, tool system, setting method and program - Google Patents

Abstract

An object of the present invention is to satisfactorily perform image correction of a captured image. An imaging system (2) includes an imaging section (32), a correction section (331), and a setting section (332). The imaging unit 32 is detachable from the tool 4 and generates a captured image. The correction unit 331 performs image correction on the captured image. The setting unit 332 sets correction values for image correction by the correction unit 331 . [Selection drawing] Fig. 1

Description

TECHNICAL FIELD The present disclosure relates generally to imaging systems, tool systems, setting methods and programs, and more particularly to imaging systems, tool systems, setting methods and programs that generate captured images.

A tool system is disclosed in US Pat. The tool system includes a portable tool and an identifier. The tool has an imaging section that generates a captured image, and the identifying section identifies the work target on which the tool is set from the captured image in a non-contact manner.

JP 2018-108633 A

The tool described in Patent Literature 1 has an imaging section. Since the life of a tool is generally shorter than the life of an imaging unit, there is a problem that when the tool reaches the end of its life and becomes unusable, the imaging unit is not effectively used even though it has not yet reached the end of its life.

For example, when the image capturing unit is detachable from a tool and the image capturing unit is used effectively, the attachment position of the image capturing unit, the tool to which the image capturing unit is attached, and the like may change. When the mounting position of the image capturing unit, the tool to which the image capturing unit is attached, and the like change, the image capturing range, the image capturing angle, and the like change before and after the image capturing unit is attached.

The present disclosure has been made in view of the above reasons, and aims to provide an imaging system, a tool system, a setting method, and a program capable of satisfactorily correcting a captured image.

An imaging system according to an aspect of the present disclosure includes an imaging section, a correction section, and a setting section. The imaging unit is attachable to and detachable from the tool, and generates a captured image. The correction unit performs image correction on the captured image. The setting unit sets a correction value for the image correction by the correction unit.

A tool system according to one aspect of the present disclosure includes the imaging system and a tool.

A setting method according to an aspect of the present disclosure is a setting method used in an imaging system that is detachable from a tool and includes an imaging unit that generates a captured image. The setting method includes a correction step and a setting step. In the correction step, image correction is performed on the captured image. The setting step sets a correction value for the image correction in the correction step.

A program according to an aspect of the present disclosure is a program for causing one or more processors to execute the setting method.

According to the imaging system, the tool system, the setting method, and the program according to the above aspects of the present disclosure, it is possible to satisfactorily correct the captured image.

FIG. 1 is a block diagram showing the configuration of a tool system according to Embodiment 1. FIG. FIG. 2 is a schematic diagram showing the appearance of the same tool system. FIG. 3 is a schematic diagram showing an example of the mounting position of the imaging device in the same tool system. FIG. 4 is a schematic diagram showing an example of an image captured by the imaging device of the same. FIG. 5 is a schematic diagram showing another example of the mounting position of the imaging device of the same. FIG. 6 is a schematic diagram showing another example of an image captured by the imaging device of the same. FIG. 7 is a flow chart showing the operation of the imaging apparatus of the same. FIG. 8 is a sequence diagram showing the operation of the same tool system. FIG. 9 is a schematic diagram showing the appearance of the tool system according to Embodiment 2. FIG. FIG. 10 is a schematic diagram showing a main part of a jig in the same tool system. FIG. 11 is a schematic diagram showing an example of an image captured by an imaging device in the same tool system. 12 is a block diagram showing the configuration of a tool system according to Embodiment 3. FIG. FIG. 13 is a flow chart showing the operation of the imaging device in the same tool system.

Preferred embodiments of the present disclosure will be described in detail below with reference to the drawings. The following embodiment is but one of various embodiments of the present disclosure. The embodiments can be modified in various ways according to design and the like as long as the object of the present disclosure can be achieved. Each drawing described in this disclosure is a schematic drawing, and the ratio of the size and thickness of each component in each drawing does not necessarily reflect the actual dimensional ratio. It should be noted that the arrows indicating each direction in the drawings are only examples, and are not meant to define the directions when the tool system 1 is used. In addition, the arrows indicating each direction in the drawings are only shown for explanation and are not substantial.

It should be noted that the term “perpendicular” as used in the present disclosure means not only a state in which the angle between the two is strictly 90 degrees, but also a state in which the two are substantially perpendicular to each other within a certain margin of error. In other words, the angle between the two orthogonal angles is within a certain range of error (for example, 10 degrees or less) with respect to 90 degrees.

(Embodiment 1)
(1) Overview First, an overview of a tool system 1 according to Embodiment 1 will be described with reference to FIGS. 1 and 2. FIG.

As shown in FIG. 1, tool system 1 comprises imaging system 2 and tool 4 .

As shown in FIG. 2, the tool 4 is a portable tool that can be held by an operator with one hand. The tool 4 is, for example, an impact wrench having a motor and an impact mechanism. By using the tool 4, for example, the operator attaches a fastening part (for example, a bolt or nut) to the work (object to be processed) that is the work target X1, or performs processing such as drilling the work. You can In addition, the “work target” in the present disclosure means an object or part (location) or the like on which work is performed using the tool 4 .

The imaging system 2 of Embodiment 1 includes an imaging device 3 . The imaging device 3 is configured to be detachable from the tool 4 . As shown in FIG. 1 , the imaging device 3 includes an imaging section 32 , a correction section 331 and a setting section 332 .

The imaging unit 32 generates a captured image. The imaging unit 32 of the first embodiment is attached to the tool 4 so as to include the work target X1 of the tool 4 in the imaging range, for example.

In addition, the “captured image” referred to in the present disclosure includes still images (still images) and moving images (moving images). Furthermore, "moving image" includes an image composed of a plurality of still images (frames) obtained by frame-by-frame shooting or the like. The captured image may not be the data itself output from the imaging unit 32 . For example, the captured image may be subjected to processing such as data compression, conversion to another data format, brightness adjustment, or contrast adjustment as necessary. In the first embodiment, the captured image is, for example, a full-color still image.

The correction unit 331 performs image correction on the captured image captured by the imaging unit 32 . The “image correction” referred to in the present disclosure may include correction such as rotation correction of the captured image, size correction of the captured image, range correction (trimming) for cutting out a part of the captured image, and distortion correction of the captured image. Also, "rotational correction of the captured image" may include rotating the captured image by an arbitrary angle. Further, "correction of the size of the captured image" may include enlargement or reduction of the captured image captured by the imaging unit 32, magnification correction (focus adjustment) when the imaging unit 32 captures the image, and the like. Also, "distortion correction of the captured image" may include partially expanding or contracting the captured image by an arbitrary amount. Also, "distortion correction" may include keystone correction. “Keystone correction” includes, for example, obtaining a captured image corrected to make the subject look like a rectangle by applying keystone correction (distortion correction) to a captured image in which a rectangular subject looks like a trapezoid. obtain.

The setting unit 332 sets correction values for image correction by the correction unit 331 . The "correction value" referred to in the present disclosure may include a rotation angle in rotation correction, a scaling ratio in size correction, a magnification when the imaging unit 32 captures an image, a trimming range, a trimming position, an expansion/contraction magnification in distortion correction, and the like. .

According to the imaging system 2 of Embodiment 1, since the setting unit 332 sets the correction value for image correction, it is possible to satisfactorily correct the captured image.

(2) Details A detailed configuration of the tool system 1 according to the first embodiment will be described below with reference to FIGS. 1 to 8. FIG.

In the following, as an example, as shown in FIGS. 2, 3, and 5, three axes of X, Y, and Z orthogonal to each other are set and described. Here, the axis along the axis Ax1 of the output shaft 421 in the tool 4 is defined as "X axis". The axis along the direction in which the body portion 401 of the body 40 and the mounting portion 403 of the tool 4 are arranged is defined as the "Z axis". In the following description, the direction along the X-axis will be simply referred to as the "front-rear direction", the negative direction of the X-axis will be referred to as "forward", and the positive direction of the X-axis will be referred to as "rearward". Also, the direction along the Y-axis is sometimes simply referred to as the "horizontal direction", the negative direction of the Y-axis is sometimes referred to as "leftward", and the positive direction of the Y-axis is referred to as "rightward". Also, the direction along the Z-axis is sometimes simply referred to as the "vertical direction", the positive direction of the Z-axis is sometimes referred to as "upper", and the negative direction of the Z-axis is referred to as "downward".

(2.1) Tool System Configuration As shown in FIG. 1 , the tool system 1 includes an imaging system 2 and a tool 4 . The tool system 1 according to the first embodiment is used, for example, in an assembly line for assembling workpieces in a factory. In particular, as an example in Embodiment 1, the tool 4 included in the tool system 1 is a tightening tool, such as an impact wrench, used for tightening tightening parts (for example, bolts or nuts). More specifically, in the first embodiment, one workpiece has a plurality of locations to be tightened, and an operator uses the tool 4 to tighten each of the plurality of locations to be tightened in one work space. Assume a case in which attached parts are attached.

A “place to be tightened” as used in the present disclosure is a part of the work and a portion to which the tightening part is attached. For example, if the tightening part is a bolt, the screw hole through which the tightening part is tightened and the locations around the screw hole are tightened portions.

(2.2) Configuration of Tool As shown in FIG. 1 , the tool 4 has a tool-side communication section 41 , a drive section 42 and a tool-side control section 43 . Moreover, as shown in FIG. 2, the tool 4 has a body 40, a trigger 44, and a battery pack 45. As shown in FIG.

The tool 4 according to Embodiment 1 is an electric power tool that operates the drive unit 42 using electrical energy. Especially in Embodiment 1, it is assumed that the tool 4 is an impact wrench.

The tool 4 uses the battery pack 45 as a power source to operate the drive unit 42 with electric power (electrical energy) supplied from the battery pack 45 . In Embodiment 1, the battery pack 45 is included as a component of the tool 4, but it is not essential that the battery pack 45 is included as a component of the tool 4, and the battery pack 45 is included as a component of the tool 4. It does not have to be included.

The body 40 accommodates a driving portion 42 and an impact mechanism. Furthermore, in the tool 4 , the tool-side communication section 41 and the tool-side control section 43 are also housed in the body 40 .

A body 40 of the tool 4 has a body portion 401 , a grip portion 402 and a mounting portion 403 . The body portion 401 is formed in a tubular shape (cylindrical shape here).

The grip portion 402 protrudes (downward) from a portion of the peripheral surface of the body portion 401 along the radial direction of the body portion 401 .

The mounting portion 403 is provided at the lower end portion of the grip portion 402 . In other words, the body portion 401 and the mounting portion 403 are connected by the grip portion 402 . Mounting portion 403 is configured to detachably mount battery pack 45 .

The drive unit 42 has a motor. The drive unit 42 is configured to operate using electric power supplied to the motor from a battery pack 45 as a power source. An output shaft 421 protrudes from the tip (front end) of the body portion 401 . The output shaft 421 rotates about the axis Ax1 of the output shaft 421 as the drive unit 42 operates. That is, the drive unit 42 drives the output shaft 421 to rotate the output shaft 421 around the axis Ax1. In other words, when the drive unit 42 operates, torque acts on the output shaft 421 to rotate the output shaft 421 .

A tip tool 422 such as a socket for rotating the tightening part is detachably attached to the output shaft 421 . The tip tool 422 rotates together with the output shaft 421 about the axis Ax1. If the tip tool 422 is fitted to the fastening part, the fastening part rotates together with the tip tool 422, and the operation of tightening or loosening the fastening part is realized.

Trigger 44 protrudes from grip portion 402 . The trigger 44 is an operation unit that receives an operation for controlling rotation of the motor. By pulling the trigger 44, the motor can be turned on and off. Further, the rotational speed of the motor can be adjusted by the amount of retraction when the trigger 44 is pulled. The rotation speed of the motor increases as the retraction amount increases.

The tool-side communication unit 41 includes a communication interface configured to be capable of communicating with an imaging-side communication unit 31 (described later) of the imaging device 3 (imaging system 2). The term “communicable” as used in the present disclosure means that information can be exchanged directly or indirectly via a network, a repeater, or the like, using an appropriate communication method such as wired communication or wireless communication.

The tool-side communication unit 41 of Embodiment 1 is, for example, Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or a standard such as low-power radio (specified low-power radio) that does not require a license. communicates with the imaging side communication unit 31 by a wireless communication method conforming to the above.

Tool 4 comprises, for example, a microcomputer with a processor and memory. The computer system functions as the tool-side controller 43 by the processor executing appropriate programs. That is, the tool-side control section 43 is realized by a computer system having a processor and memory. The program may be prerecorded in a memory, or may be provided by being recorded in a non-temporary recording medium such as a memory card or through an electric communication line such as the Internet.

The tool-side control section 43 controls the driving section 42 . Specifically, the tool-side control section 43 operates the drive section 42 to rotate the output shaft 421 at a rotational speed based on the retraction amount of the trigger 44 .

Further, the tool-side control section 43 of the first embodiment controls the driving section 42 so that the tightening torque of the tightening component becomes the torque set value. The tool-side control unit 43 of Embodiment 1 has a torque estimation function of estimating the magnitude of tightening torque. In the first embodiment, as an example, the tool-side control unit 43 estimates the magnitude of the tightening torque based on the rotation speed of the drive unit 42 (motor) until the estimated value of the tightening torque reaches the seating determination level. do. When the estimated value of the tightening torque reaches the seating determination level, the tool-side control section 43 estimates the magnitude of the tightening torque based on the number of impacts of the impact mechanism. When the number of impacts of the impact mechanism reaches the threshold number of times based on the torque setting value, the tool-side control section 43 determines that the tightening torque has reached the torque setting value, and stops the driving section 42 (motor). As a result, the tool 4 can tighten the tightening component with the tightening torque exactly as the torque setting value.

Further, the tool-side control section 43 of Embodiment 1 controls the tool-side communication section 41 to transmit a completion notice to the imaging device 3 (imaging system 2) after the tightening of the fastening component is completed.

(2.3) Configuration of Imaging System Captured images for recording and the like are generated. In the first embodiment, it is assumed that the imaging system 2 generates captured images for recording as traceability.

As shown in FIG. 1 , the imaging system 2 of Embodiment 1 includes an imaging device 3 . In other words, the imaging system 2 of Embodiment 1 is implemented by the imaging device 3 .

(2.4) Configuration of Imaging Device The imaging device 3 is detachably fixed to the tool 4 by an appropriate mounting mechanism such as a screwing mechanism, fitting mechanism, sandwiching mechanism, and sticking mechanism.

Moreover, the fixing position of the imaging device 3 with respect to the tool 4 may be a position that includes the work target X1 in the imaging range of the imaging unit 32 . For example, the imaging device 3 may be fixed to the upper portion of the body portion 401 of the tool 4 as shown in FIG. 3, or may be fixed to the side portion of the body portion 401 of the tool 4 as shown in FIG. .

As shown in FIG. 1, the imaging device 3 of Embodiment 1 includes a body 30, an imaging-side communication unit 31, an imaging unit 32, an imaging-side control unit 33, a storage unit 34, and an operation unit 35. Prepare.

The body 30 is formed in a rectangular box shape. The body 30 houses an imaging side communication section 31 , at least part of the imaging section 32 , an imaging side control section 33 and a storage section 34 .

The imaging side communication section 31 includes a communication interface configured to be capable of communicating with a later-described tool side communication section 41 of the tool 4 . The imaging-side communication unit 31 of the first embodiment is, for example, a standard such as Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or a low-power radio that does not require a license (specified low-power radio). communicates with the tool-side communication unit 41 using a wireless communication method conforming to The imaging side communication section 31 of the first embodiment receives the completion notification from the tool side communication section 41 of the tool 4 .

The imaging unit 32 has, for example, an imaging device and a lens. The imaging unit 32 generates data as a captured image. The imaging unit 32 of the first embodiment is arranged at the tip (front surface) of the body 30 so that the work target X1 of the tool 4 is within the imaging range.

The operating portion 35 is provided on the upper portion of the body 30 . As shown in FIG. 3, the operation unit 35 has, for example, a plurality of (four in the example of FIG. 3) push button switches 351 . Further, the operation unit 35 may have a plurality of LEDs (Light Emitting Diodes) or a touch panel display. The operator can shift the state of the imaging device 3 to the "setting mode" by pressing one of the plurality of push button switches 351, for example.

The imaging device 3 comprises, for example, a microcomputer having a processor and memory. The computer system functions as the imaging-side control unit 33 by the processor executing appropriate programs. That is, the imaging-side control unit 33 is realized by a computer system having a processor and memory. The program may be prerecorded in a memory, or may be provided by being recorded in a non-temporary recording medium such as a memory card or through an electric communication line such as the Internet.

The imaging-side control unit 33 controls the imaging unit 32 . The imaging-side control unit 33 of the first embodiment controls the imaging unit 32 to start imaging when the state of the imaging device 3 shifts to the setting mode. Further, when the imaging-side communication unit 31 receives the completion notification from the tool 4, the imaging-side control unit 33 of the first embodiment controls the imaging unit 32 to start imaging.

The imaging-side control section 33 has a correction section 331 , a setting section 332 and a storage section 333 .

The correction unit 331 performs image correction on the captured image generated by the imaging unit 32 . As described above, the correction unit 331 performs image correction such as rotation correction, size correction, range correction (trimming), and distortion correction.

The setting unit 332 sets correction values for image correction by the correction unit 331 . More specifically, the setting unit 332 of the first embodiment sets a correction value for image correction when the imaging device 3 is in the setting mode.

FIG. 4 shows an example of a captured image when the imaging device 3 is attached to the upper portion of the body portion 401 of the tool 4 (see FIG. 3). Moreover, FIG. 6 shows an example of a captured image when the imaging device 3 is attached to the side portion of the body portion 401 of the tool 4 (see FIG. 5).

As shown in FIGS. 4 and 6 , the captured image generated by the imaging unit 32 differs depending on how the imaging device 3 (imaging unit 32 ) is attached to the tool 4 . For example, the image Im1 in FIG. 4 and the image Im2 in FIG. 6 are images of the same work target X1. However, the image Im1 and the image Im2 are different images because the mounting conditions of the imaging unit 32 (imaging device 3) are different. If the mounting condition of the imaging device 3 changes and the captured image changes, it is not preferable from the viewpoint of the accuracy of image recognition and quality determination, and traceability. Therefore, the setting unit 332 of the first embodiment sets a correction value for image correction based on the attachment status of the imaging device 3 (imaging unit 32 ) to the tool 4 . By setting the correction value based on the mounting situation such as the mounting position and mounting angle of the imaging device 3 with respect to the tool 4, the image correction of the captured image can be performed more satisfactorily. In addition, the “mounting situation” referred to in the present disclosure may include the mounting position of the imaging device 3 with respect to the tool 4, the mounting angle of the imaging device 3 with respect to the tool 4, and the like.

Further, the setting unit 332 of the first embodiment sets the correction value based on the setting image captured (generated) by the imaging unit 32 . A “setting image” referred to in the present disclosure may include a captured image generated by the imaging unit 32 when the imaging device 3 is in the setting mode.

The setting unit 332 performs a specifying process of specifying the orientation of the image of the output shaft 421 (tip of the tool 4) included in the setting image (the output shaft 421 shown in the setting image). Note that the setting unit 332 may specify the orientation of the image of the tip tool 422 (the tip of the tool 4) included in the setting image.

The setting unit 332 identifies the orientation of the image of the output shaft 421 included in the setting image by performing pattern recognition processing, for example. “Pattern recognition processing” as used in the present disclosure means image processing for recognizing what the image of an object is based on the shape of the image of the object included in the image. Examples of this type of pattern recognition processing include pattern matching processing, processing for recognizing an image of an object included in an image using a learned model created by machine learning, and the like. The pattern matching process referred to here is a process of comparing template data with a comparison target (setting image) using template data such as an image of the output shaft 421 (tip of the tool 4). As a machine learning method, an appropriate algorithm may be used, for example, a deep learning algorithm may be used.

The setting unit 332 of the first embodiment specifies the orientation of the image of the output shaft 421 included in the setting image, and then the image of the work target X1 included in the captured image has a good balance between the vertical and horizontal sizes. Set a correction value such as

When the imaging device 3 is attached to the upper part of the body portion 401 of the tool 4 (see FIG. 3), as shown in FIG. Along the upward direction. As shown in FIG. 4, when the orientation of the image on the output shaft 421 is from the bottom to the top of the setting image, the top of the image of the work target X1 is larger than the bottom of the work target X1. For this reason, the setting unit 332 of the first embodiment determines that the size (vertical) balance between the upper portion of the image of the work target X1 and the lower portion of the image of the work target X1 in image correction such as trapezoidal correction performed by the correction unit 331. Set the correction value so that it is good. Note that the setting unit 332 specifies the length (the length of the arrow Ar1) along the axis Ax1 of the image of the tool 4 included in the image for setting in the specifying process, and determines the length of the image of the tool 4 included in the image for setting. A correction value may be set based on the length of .

Further, when the imaging device 3 is attached to the side portion of the body portion 401 of the tool 4 (see FIG. 5), as shown in FIG. along the left-to-right direction of the As shown in FIG. 6, when the orientation of the image on the output shaft 421 is from the left to the right of the setting image, the right portion of the image of the work target X1 becomes larger than the left portion of the image of the work target X1. For this reason, the setting unit 332 of the first embodiment determines the size (left and right) of the left part of the image of the work target X1 and the right part of the image of the work target X1 in image correction such as trapezoidal correction performed by the correction unit 331. Set the correction value so that the balance is good. Note that the setting unit 332 may set the correction value based on the length (the length of the arrow Ar2) along the axis Ax1 of the image of the tool 4 included in the setting image.

The setting unit 332 sets the correction value for image correction based on the setting image captured by the imaging unit 32, so that the captured image can be corrected more satisfactorily.

The storage unit 333 stores the captured image after the image correction has been performed by the correction unit 331 in the storage unit 34 .

The storage unit 34 is a semiconductor memory such as ROM (Read Only Memory), RAM (Random Access Memory), or EEPROM (Electrically Erasable Programmable Read Only Memory). Note that the storage unit 34 is not limited to a semiconductor memory, and may be a hard disk drive or the like. The storage unit 34 stores captured images after image correction has been performed by the correction unit 331 .

(3) Operation of Tool System Next, operation of the tool system 1 will be described with reference to FIGS. 7 and 8. FIG.

FIG. 7 is a flowchart showing the operation of the imaging device 3 (imaging system 2). The processing shown in FIG. 7 is performed, for example, when the state of the imaging device 3 shifts to the setting mode. After shifting to the setting mode, the imaging device 3 starts imaging (S1). A setting image is generated when the imaging unit 32 of the imaging device 3 starts imaging.

Next, the setting unit 332 of the imaging device 3 performs a specifying process of specifying the orientation of the image of the output shaft 421 (tip of the tool 4) included in the setting image (S2). Then, the setting unit 332 of the imaging device 3 specifies the orientation of the image of the output shaft 421 included in the setting image, and the size of the image of the work target X1 included in the captured image is well balanced. A correction value is set so that (S3). When the setting unit 332 sets the correction value, the setting mode processing ends.

The flowchart shown in FIG. 7 is merely an example, and the order of processing may be changed as appropriate, and processing may be added or deleted as appropriate.

FIG. 8 is a sequence diagram showing the operation of the tool system 1. FIG. First, when the trigger 44 of the tool 4 is pulled by the operator, the tool 4 performs a tightening operation (S11). For example, when the tightening torque reaches the set torque value, the tool 4 completes the tightening operation, and transmits a completion notification to the imaging device 3 (S12).

When the imaging device 3 receives the completion notification from the tool 4, the imaging unit 32 starts imaging (S13). The correction unit 331 of the imaging device 3 performs image correction on the captured image using the correction value set by the setting unit 332 (S14). Then, the storage unit 333 of the imaging device 3 stores the captured image after image correction by the correction unit 331 in the storage unit 34 (S15). The processing ends when the storage unit 333 stores the captured image in the storage unit 34 .

The sequence diagram shown in FIG. 8 is merely an example, and the order of processing may be changed as appropriate, and processing may be added or deleted as appropriate.

(4) Modifications Embodiment 1 is merely an example of various embodiments of the present disclosure. Embodiment 1 can be modified in various ways according to design and the like, as long as the object of the present disclosure can be achieved.

Also, functions equivalent to those of the tool system 1 (imaging system 2) according to the first embodiment may be embodied by a setting method, a (computer) program, or a non-temporary recording medium recording the program. A setting method according to one aspect is a setting method used in an imaging system 2 that is attachable to and detachable from the tool 4 and includes an imaging unit 32 (imaging device 3) that generates a captured image. The setting method has a correction step and a setting step. In the correction step, image correction is performed on the captured image. In the setting step, correction values for image correction in the correction step are set. A program according to one aspect is a program for causing one or more processors to execute the above setting method.

The tool system 1 (imaging system 2) in the present disclosure includes, for example, a computer system. A computer system is mainly composed of a processor and a memory as hardware. The functions of the tool system 1 in the present disclosure are realized by the processor executing a program recorded in the memory of the computer system. The program may be recorded in advance in the memory of the computer system, may be provided through an electric communication line, or may be recorded in a non-temporary recording medium such as a computer system-readable memory card, optical disk, or hard disk drive. may be provided. A processor in a computer system is made up of one or more electronic circuits, including semiconductor integrated circuits (ICs) or large scale integrated circuits (LSIs). The integrated circuit such as IC or LSI referred to here is called differently depending on the degree of integration, and includes integrated circuits called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). In addition, a field-programmable gate array (FPGA) that is programmed after the LSI is manufactured, or a logic device capable of reconfiguring the bonding relationship inside the LSI or reconfiguring the circuit partitions inside the LSI may also be adopted as the processor. can be done. A plurality of electronic circuits may be integrated into one chip, or may be distributed over a plurality of chips. A plurality of chips may be integrated in one device, or may be distributed in a plurality of devices. A computer system, as used herein, includes a microcontroller having one or more processors and one or more memories. Accordingly, the microcontroller also consists of one or more electronic circuits including semiconductor integrated circuits or large scale integrated circuits.

The imaging system 2 may include at least the imaging section 32 , the correction section 331 and the setting section 332 .

At least some of the functions of the imaging system 2 may be realized by a cloud (cloud computing) or the like.

It is not an essential configuration of the imaging system 2 that at least some of the functions of the imaging system 2 are integrated in one housing (imaging device 3), and at least some of the functions of the imaging system 2 are provided in a plurality of housings. It may be provided dispersedly on the body. For example, the functions such as the setting unit 332 of the imaging system 2 may be provided in a housing (device) other than the imaging device 3 such as the tool 4 .

For example, the setting unit 332 may be provided in another device configured to be communicable with the imaging device 3 . For example, the imaging device 3 transmits the setting image captured by the imaging unit 32 to another device, and the setting unit 332 provided in the other device corrects the image based on the setting image received from the imaging device 3. Set the correction value. Then, another device transmits information about the correction value set by the setting unit 332 to the imaging device 3, and the correction unit 331 of the imaging device 3 performs image correction based on the information about the correction value received from the other device. .

It is not essential that the imaging device 3 include the storage unit 34 . The imaging system 2 may include a storage device separate from the imaging device 3 instead of the storage unit 34 .

It is not essential that the imaging device 3 shifts to the setting mode and the setting unit 332 sets the correction values based on the setting image. For example, the setting unit 332 may set the correction value by specifying the orientation of the image of the output shaft 421 included in the captured image of the work target X1. When the setting unit 332 sets the correction value by specifying the orientation of the image of the output shaft 421 included in the captured image of the work target X1, the imaging device 3 performs, for example, the processing of steps S13 and S14 shown in FIG. Steps S1 to S3 shown in FIG. 7 are performed between the processing of .

The setting unit 332 may set the correction value for image correction after the imaging unit 32 is attached to the tool 4 . For example, after the operator attaches the imaging device 3 to the tool 4, the operator operates the operation unit 35 to set the state of the imaging device 3 to the setting mode. 4, set the correction value for the image correction. After the imaging device 3 is attached to the tool 4, the setting unit 332 sets the correction value for image correction, so that the captured image can be corrected more satisfactorily. In addition, at least one of the imaging device 3 and the tool 4 can detect that the imaging device 3 is attached to the tool 4, and after it is detected that the imaging device 3 is attached to the tool 4, the setting unit 332 may set correction values for image correction. For example, after detecting that the imaging device 3 is attached to the tool 4, the imaging device 3 performs the processing of steps S1 to S3 shown in FIG. 7 to set a correction value for image correction.

It is not essential for the setting unit 332 to set the correction value for image correction based on the setting image captured by the imaging unit 32 . The imaging device 3 may be configured such that, for example, by operating the operation unit 35 , information regarding the attachment status of the imaging device 3 to the tool 4 can be input. The setting unit 332 sets a correction value for image correction based on the information about the mounting state input to the imaging device 3 . The information about the mounting state of the imaging device 3 may be input to the imaging device 3 not only by the operation on the operation unit 35 but also by communication from an external device such as a setting terminal.

The setting unit 332 may set the correction value for image correction based on the type of the tool 4 to which the imaging unit 32 is attached. The imaging device 3 may be configured to input information about the type (model) of the tool 4 to which the imaging device 3 is attached, for example, by operating the operation unit 35 . The setting unit 332 sets correction values for image correction based on the information about the type of the tool 4 input to the imaging device 3 . The information about the type of the tool 4 to which the imaging device 3 is attached may be input to the imaging device 3 by communication from an external device such as a setting terminal, not limited to the operation on the operation unit 35 . By setting the correction value by the setting unit 332 based on the type of the tool 4 to which the imaging unit 32 is attached, the image correction of the captured image can be performed more satisfactorily.

Also, the setting unit 332 may use a learned model generated by machine learning to set a correction value for image correction. The learned model is generated, for example, by supervised learning using a plurality of teacher data indicating the relationship between the setting image captured (generated) by the imaging unit 32 and the correction value for image correction. The setting unit 332 can obtain information indicating a correction value for image correction as an output of the learned model by inputting the setting image captured by the imaging unit 32 to the learned model. At least one of information regarding the mounting state of the imaging unit 32 and information regarding the type of the tool 4 to which the imaging unit 32 is mounted may be used as the input of the learned model instead of the setting image.

The machine learning algorithm is, for example, a neural network. However, machine learning algorithms are not limited to neural networks, for example, XGB (eXtreme Gradient Boosting) regression, Random Forest, decision tree, Logistic Regression, support vector machine ( Support vector machine (SVM), Naive Bayes classifier, k-nearest neighbors method, or the like may be used. Furthermore, the machine learning algorithm may be, for example, Gaussian Mixture Model (GMM), k-means clustering, or the like.

Also, the learning method is supervised learning as an example in the first embodiment. However, the learning method is not limited to supervised learning, and may be unsupervised learning or reinforcement learning.

Also, the trained model may be updated by performing additional learning.

Although the case where the tool 4 is an impact wrench has been described in the first embodiment, the tool 4 is not limited to an impact wrench, and may be, for example, a nut runner or an oil pulse wrench. Further, the tool 4 is not limited to a configuration using the battery pack 45 as a power source, and may be configured using an AC power source (commercial power source) as a power source. Moreover, the tool 4 is not limited to an electric tool, and may be an air tool having an air motor that operates using compressed air supplied from an air compressor as a power source.

(Embodiment 2)
As shown in FIG. 9 , the tool system 1 according to Embodiment 2 differs from the tool system 1 according to Embodiment 1 in that the imaging system 2 includes a jig 5 .

As described above, the imaging system 2 of Embodiment 2 further includes the jig 5 .

The jig 5 has a reference object 51 , a pair of first mounting portions 52 and a pair of second mounting portions 53 . Note that each of the pair of first mounting portions 52 is arranged along the Y-axis, and FIG. 9 is a plan view of the tool system 1 viewed along the Y-axis. Of the mounting portions 52, only the first mounting portion 52 on the front side of the drawing is shown. Similarly, since each of the pair of second mounting portions 53 is aligned along the Y-axis, FIG. showing. Further, in the following description, each of the pair of first mounting portions 52 may be simply referred to as "first mounting portion 52", and each of the pair of second mounting portions 53 may simply be referred to as "second mounting portion 53". .

The jig 5 of Embodiment 2 can be attached to and detached from the tool 4 by a pair of first attachment portions 52 and a pair of second attachment portions 53 . Also, the jig 5 is configured to face the imaging unit 32 of the imaging device 3 with a reference object 51 that serves as a reference for image correction. The jig 5 is attached to the tool 4 after the imaging device 3 is attached to the tool 4, for example. Moreover, the jig 5 of Embodiment 3 is removed from the tool 4 when the tool 4 is operated.

The reference object 51 of the second embodiment is a rectangular flat plate. The reference object 51 faces the imaging section 32 of the imaging device 3 in the X-axis direction. More specifically, the reference object 51 has a facing surface 510 that faces the imaging section 32 of the imaging device 3 in the X-axis direction.

As shown in FIG. 10, a lattice pattern 511 is formed on the facing surface 510 . Note that the lattice pattern 511 may be formed with a color difference from the surroundings, or may be formed with unevenness.

The setting unit 332 of the second embodiment sets a correction value for image correction based on the setting image of the reference object 51 captured by the imaging unit 32 . FIG. 11 shows a setting image obtained by imaging the grid pattern 511 of the reference object 51 by the imaging unit 32 . As shown in FIG. 11, the image Im3 of the pattern 511 included in the setting image has an angle in the circumferential direction D1 about the X-axis (see FIG. 10) compared to the actual pattern 511 (see FIG. 10). different. Also, the image Im3 of the pattern 511 included in the setting image is distorted.

The setting unit 332 of the second embodiment sets a correction value for image correction so that the shape of the image Im3 of the pattern 511 included in the setting image approximates the shape of the actual pattern 511 (see FIG. 10). Specifically, the setting unit 332 sets the rotation correction angle so that the angle of the image Im3 of the pattern 511 in the circumferential direction D1 is the same as the angle of the actual pattern 511 in the circumferential direction D1. Further, the setting unit 332 sets a correction value for distortion correction so that the shape of the image Im3 of the pattern 511 is the same as that of the actual pattern 511 .

The setting unit 332 may also set the correction value for size correction based on the distance between any two points in the image Im3 of the pattern 511 . The setting unit 332 sets a correction value for size correction based on, for example, the distance between the first intersection point P1 and the second intersection point P2.

The first attachment portion 52 protrudes from the reference object 51 along the X-axis. In other words, the first attachment portion 52 protrudes rearward from the reference object 51 . The first attachment portion 52 is attached to the side portion of the body portion 401 of the tool 4, for example. The first attachment portion 52 is attached to the body portion 401 of the tool 4 by an appropriate attachment mechanism such as a screwing mechanism, fitting mechanism, and sticking mechanism. The pair of first mounting portions 52 sandwich the body portion 401 of the tool 4 in the direction along the Y axis.

The second mounting portion 53 protrudes from the reference object 51 along the X-axis. In other words, the second attachment portion 53 protrudes rearward from the reference object 51 . The second attachment portion 53 is attached to the side portion of the grip portion 402 of the tool 4, for example. The second attachment portion 53 is attached to the grip portion 402 of the tool 4 by an appropriate attachment mechanism such as a screwing mechanism, fitting mechanism, and sticking mechanism. The pair of second mounting portions 53 sandwich the grip portion 402 of the tool 4 in the direction along the Y axis.

The imaging system 2 of the second embodiment is provided with the jig 5 as described above, so that even if the output shaft 421 (tip tool 422) of the tool 4 is not included in the imaging range of the imaging unit 32, for example, Also, by using the jig 5, the correction value for image correction can be set based on the setting image captured by the imaging unit 32. FIG. In addition, since the setting unit 332 sets the correction value based on the setting image of the reference object 51, the image correction of the captured image can be performed more satisfactorily.

Embodiment 2 is but one example of various embodiments of the present disclosure. Embodiment 2 can be modified in various ways according to the design, etc., as long as the object of the present disclosure can be achieved.

Various configurations (including modifications) described in the second embodiment can be employed in appropriate combination with various configurations (including modifications) described in the first embodiment.

(Embodiment 3)
As shown in FIG. 12, the tool system 1 according to Embodiment 3 is different from the tool systems according to Embodiments 1 and 2 in that the imaging system 2 (imaging device 3) includes a distance measurement unit 36 and an orientation detection unit 37. different from 1.

As described above, the imaging device 3 of Embodiment 3 further includes the distance measurement section 36 and the orientation detection section 37 .

The distance measuring unit 36 measures the distance between the imaging unit 32 (the tip of the body 30) and the work target X1 of the tool 4. FIG.

The distance measurement unit 36 includes, for example, a distance sensor such as RADAR (Radio Detection and Ranging), LiDAR (Light Detection and Ranging), or an ultrasonic sensor. A LiDAR is, for example, an infrared sensor. As an example in the third embodiment, the distance measurement unit 36 has an ultrasonic sensor. The ultrasonic sensor is a time-of-flight that measures the distance to an object by measuring the time it takes to emit ultrasonic waves and receive the ultrasonic waves reflected by the object (for example, work object X1). It is a distance sensor of the method. The ultrasonic sensor outputs an electrical signal according to the measured distance.

The orientation detection unit 37 detects the orientation of the imaging unit 32 . In other words, the posture detection unit 37 detects the mounting angle of the imaging unit 32 (imaging device 3 ) with respect to the tool 4 .

The orientation detection unit 37 includes, for example, motion sensors such as an acceleration sensor and a gyro sensor. As an example in the third embodiment, the posture detection unit 37 has a triaxial acceleration sensor and a triaxial gyro sensor as motion sensors. A three-axis acceleration sensor detects acceleration on each of three axes orthogonal to each other and outputs an electrical signal corresponding to the acceleration. A three-axis gyro sensor detects angular velocities around three axes orthogonal to each other and outputs electrical signals corresponding to the angular velocities.

The setting unit 332 of the third embodiment sets at least one of the distance between the imaging unit 32 and the work target X1 of the tool 4 measured by the distance measuring unit 36 and the orientation of the imaging unit 32 detected by the orientation detection unit 37. Based on this, the correction value for image correction is set. In other words, the setting unit 332 uses at least one of the distance between the imaging unit 32 and the tool 4 and the posture of the imaging unit 32 as information indicating the mounting state of the imaging unit 32 with respect to the tool 4 to determine the correction value. set.

Next, the operation of the imaging system 2 (imaging device 3) of Embodiment 3 will be described. FIG. 13 is a flowchart showing the operation of the imaging device 3 (imaging system 2). The processing shown in FIG. 13 is performed, for example, when the state of the imaging device 3 is shifted to the setting mode.

After shifting to the setting mode, the distance measuring unit 36 of the imaging device 3 measures the distance between the imaging unit 32 and the work target X1 of the tool 4 (S21). Next, the orientation detection unit 37 of the imaging device 3 detects the orientation of the imaging unit 32 (S22). The setting unit 332 of the imaging device 3 sets at least one of the distance between the imaging unit 32 and the work target X1 of the tool 4 measured by the distance measuring unit 36 and the posture of the imaging unit 32 detected by the posture detection unit 37. Based on this, a correction value for image correction is set (S23). When the setting unit 332 sets the correction value, the setting mode processing ends.

The flowchart shown in FIG. 13 is merely an example, and the order of processing may be changed as appropriate, and processing may be added or deleted as appropriate.

For example, the order of the processing in step S21 and the processing in step S22 may be reversed, or may be performed simultaneously.

The imaging system 2 of the third embodiment measures at least the distance measured by the distance measuring unit 36 (mounting position of the imaging unit 32) and the orientation of the imaging unit 32 detected by the orientation detection unit 37 (mounting angle of the imaging unit 32). By setting the correction value by the setting unit 332 based on one of them, the image correction of the captured image can be performed more satisfactorily.

Embodiment 3 is but one example of various embodiments of the present disclosure. Embodiment 3 can be modified in various ways according to design and the like as long as the object of the present disclosure can be achieved.

Various configurations (including modifications) described in the third embodiment can be employed in appropriate combination with various configurations (including modifications) described in the first and second embodiments.

(summary)
As described above, the imaging system (2) according to the first aspect includes an imaging section (32), a correction section (331), and a setting section (332). The imaging unit (32) is attachable to and detachable from the tool (4) and generates a captured image. A correction unit (331) performs image correction on the captured image. A setting unit (332) sets a correction value for image correction by the correction unit (331).

According to this aspect, since the setting unit (332) sets the correction value for the image correction, it is possible to excellently perform the image correction of the captured image.

In the imaging system (2) according to the second aspect, in the first aspect, the setting unit (332) sets the correction value for image correction after the imaging unit (32) is attached to the tool (4). .

According to this aspect, after the imaging unit (32) is attached to the tool (4), the setting unit (332) sets the correction value for image correction, so that the captured image can be corrected more satisfactorily. can be done.

In the imaging system (2) according to the third aspect, in the first or second aspect, the setting section (332) sets the correction value based on the setting image captured by the imaging section (32).

According to this aspect, the setting section (332) sets the correction value based on the setting image captured by the imaging section (32), so that the captured image can be corrected more satisfactorily.

The imaging system (2) according to the fourth aspect further comprises a jig (5) in the third aspect. The jig (5) is attachable to and detachable from the tool (4), and is configured to face the imaging unit (32) with a reference object (51) serving as a reference for image correction. A setting unit (332) sets a correction value based on the setting image of the reference object (51).

According to this aspect, the setting unit (332) sets the correction value based on the setting image of the reference object (51), so that the captured image can be corrected more satisfactorily.

In the imaging system (2) according to the fifth aspect, in any one of the first to fourth aspects, the setting unit (332) corrects based on the state of attachment of the imaging unit (32) to the tool (4) set the value.

According to this aspect, by setting the correction value based on the mounting situation such as the mounting position and mounting angle of the imaging unit (32) with respect to the tool (4), the image correction of the captured image can be performed more satisfactorily. .

The imaging system (2) according to the sixth aspect, in the fifth aspect, further comprises a distance measuring section (36). A distance measuring unit (36) measures the distance between the imaging unit (32) and the work object (X1) of the tool (4). A setting unit (332) sets a correction value based on the distance measured by the distance measuring unit (36).

According to this aspect, the setting unit (332) sets the correction value based on the distance measured by the distance measuring unit (36), so that the captured image can be corrected more satisfactorily.

The imaging system (2) according to the seventh aspect, in the fifth or sixth aspect, further comprises a posture detection section (37). A posture detection unit (37) detects the posture of the imaging unit (32). A setting unit (332) sets a correction value based on the orientation of the imaging unit (32) detected by the orientation detection unit (37).

According to this aspect, the setting unit (332) sets the correction value based on the orientation of the imaging unit (32) detected by the orientation detection unit (37), so that the image correction of the captured image is performed more satisfactorily. be able to.

In the imaging system (2) according to the eighth aspect, in any one of the first to seventh aspects, the setting unit (332) is configured based on the type of the tool (4) to which the imaging unit (32) is attached , to set the correction value.

According to this aspect, the setting unit (332) sets the correction value based on the type of the tool (4) to which the imaging unit (32) is attached, so that the captured image can be corrected more satisfactorily. can be done.

Configurations other than the first aspect are not essential to the imaging system (2), and can be omitted as appropriate.

A tool system (1) according to a ninth aspect comprises an imaging system (2) according to any one of the first to eighth aspects, and a tool (4).

According to this aspect, in the imaging system (2), the setting unit (332) sets the correction value for the image correction, so that the image correction of the captured image can be performed satisfactorily.

A setting method according to the tenth aspect is a setting method used in an imaging system (2) having an imaging unit (32) that is detachable from a tool (4) and that generates a captured image. The setting method has a correction step and a setting step. In the correction step, image correction is performed on the captured image. In the setting step, correction values for image correction in the correction step are set.

According to this aspect, since the correction value for the image correction is set, the image correction of the captured image can be performed satisfactorily.

A program according to an eleventh aspect is a program for causing one or more processors to execute the setting method according to the tenth aspect.

According to this aspect, since the correction value for the image correction is set, the image correction of the captured image can be performed satisfactorily.

1 tool system 2 imaging system 32 imaging unit 331 correction unit 332 setting unit 36 distance measurement unit 37 attitude detection unit 4 tool 5 jig 51 reference object X1 work target

Claims (11)

an imaging unit that is attachable to and detachable from a tool and generates a captured image;
a correction unit that performs image correction on the captured image;
a setting unit that sets a correction value for the image correction by the correction unit;
comprising
imaging system. The setting unit sets the correction value of the image correction after the imaging unit is attached to the tool.
The imaging system of Claim 1. The setting unit sets the correction value based on the setting image captured by the imaging unit.
3. The imaging system according to claim 1 or 2. a jig that is detachable from the tool and configured to face the imaging unit with a reference object that serves as a reference for image correction;
The setting unit sets the correction value based on the setting image of the reference object.
The imaging system according to claim 3. The setting unit sets the correction value based on the mounting status of the imaging unit with respect to the tool.
The imaging system according to any one of claims 1 to 4. further comprising a distance measuring unit that measures the distance between the imaging unit and the work target of the tool;
The setting unit sets the correction value based on the distance measured by the distance measurement unit.
The imaging system according to claim 5. further comprising an orientation detection unit that detects the orientation of the imaging unit;
The setting unit sets the correction value based on the orientation of the imaging unit detected by the orientation detection unit.
The imaging system according to claim 5 or 6. The setting unit sets the correction value based on the type of the tool to which the imaging unit is attached.
The imaging system according to any one of claims 1-7. an imaging system according to any one of claims 1 to 8;
the tool;
comprising a
tool system. A setting method used in an imaging system that is detachable from a tool and includes an imaging unit that generates a captured image,
a correction step of performing image correction on the captured image;
a setting step of setting a correction value for the image correction in the correcting step;
having
Setting method. for causing one or more processors to execute the setting method according to claim 10,
program.

Images