CN113838133A - A state detection method, apparatus, computer equipment and storage medium - Google Patents

Abstract

The present disclosure provides a state detection method, device, computer equipment and storage medium, wherein the method includes: acquiring a target image; identifying a hydraulic support in the target image, and determining a plurality of hydraulic supports corresponding to at least one target component in the hydraulic support Target key points; determine the attitude information of the hydraulic support based on the target position information of the target key points; determine the safety state of the hydraulic support based on the attitude information of the hydraulic support.

Description

A state detection method, apparatus, computer equipment and storage medium

technical field

The present disclosure relates to the technical field of deep learning and computer vision, and in particular, to a state detection method, apparatus, computer equipment and storage medium.

Background technique

In mine operations, it is necessary to use hydraulic supports to support the upper surface of the mine to prevent ore from falling, resulting in dangerous situations. For example, the hydraulic support is a kind of protective equipment used to control the mine pressure of the coal mining face. During the coal mining process, the falling of the coal mine will exert a lateral force on the hydraulic support, causing the hydraulic support to swing sideways, thereby driving the hydraulic pressure. The support is skewed; or, due to the movement of a certain hydraulic support, the bearing capacity of multiple hydraulic supports is unbalanced, thereby destroying the hydraulic support, etc., which will cause the coal to fall and be dangerous. Today, by setting up multiple sensor devices on the hydraulic support, collecting multiple sets of data collected by the multiple sensor devices, and performing fusion processing, the current position state of the hydraulic support is predicted.

However, the premise of the above prediction scheme is that sensor equipment needs to be installed on each beam of the hydraulic support to judge the posture change of the hydraulic support. Therefore, the cost requirements for the posture detection of the hydraulic support are relatively high; in addition, the working environment of coal mining is complex. , The sensor equipment on the hydraulic support is easy to be damaged, and the line of the sensor equipment is often damaged due to the movement of the hydraulic support, resulting in the failure of normal return of the collected data, which not only affects the work efficiency, but also increases the cost of maintaining and repairing the sensor equipment.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide at least a state detection method, apparatus, computer device, and storage medium.

In a first aspect, an embodiment of the present disclosure provides a state detection method, including:

get the target image;

Identifying the hydraulic support in the target image, and determining a plurality of target key points corresponding to at least one target component in the hydraulic support;

Determine the attitude information of the hydraulic support based on the target position information of the target key point;

Based on the attitude information of the hydraulic support, a safe state of the hydraulic support is determined.

In this aspect, the deep neural network model can learn all the key points of the hydraulic support in advance, and can predict the posture information of different target components for different key points. Therefore, by identifying the hydraulic support, determining multiple target key points corresponding to the target component, and using the target position information of the target key point, the posture information of the hydraulic support can be accurately calculated, such as the telescopic length of a certain target component, or The inclination angle of a target component, etc. By monitoring the posture information of the hydraulic support, the safety state of the hydraulic support, such as safe or dangerous, can be accurately obtained. After that, the staff can give timely warning when there is a dangerous situation according to the monitored safety status of the hydraulic support. Compared with the use of multiple sensors in the prior art, this aspect completely replaces the collection of data by traditional sensors. At the same time, since the way of acquiring images can be through existing surveillance cameras, there is no need to rearrange the shooting equipment, so the cost of the information acquisition equipment can be reduced; in addition, the above method is based on the deep neural network model for image detection, compared with the traditional method. The sensor fuses multiple collected data, which can improve the accuracy of the attitude information of the hydraulic support, and then can detect a relatively accurate safety state of the hydraulic support.

In an optional implementation manner, the target member includes a first member; the attitude information includes first inclination angle information of the first member relative to a preset coordinate axis in a preset coordinate system; the target The location information includes first location information;

The determining of the posture information of the hydraulic support based on the target position information of the target key points includes:

For the first component, based on the first position information of each target key point in the target image among the plurality of target key points located in the first component, it is determined that each target key point is located in the target image. the second position information in the preset coordinate system;

Based on second position information of each of the target key points in the preset coordinate system, first inclination angle information of the first member relative to a preset coordinate axis in the preset coordinate system is determined.

This embodiment is aimed at the situation where the attitude information is the first inclination angle information. Since the deep neural network model can accurately determine multiple target key points in each first component, the Accurate first position information of the target key point, and then obtain accurate second position information converted from the first position information to the preset coordinate system. Since the first inclination angle information is obtained through analysis in the preset coordinate system, accurate first inclination angle information can be obtained by using the accurate second position information.

In an optional implementation manner, the target member further includes a second member; the posture information further includes telescopic length information of the second member; the target key point includes a first key point and a second key point ; The target position information includes third position information and fourth position information;

The determining of the posture information of the hydraulic support based on the target position information of the target key points includes:

For the second component, determine a first key point located on the second component and a second key point corresponding to the second component; the second key point is located on other key points connected to the second component on the target component;

The first key point and the second key point are determined based on the third position information of the first key point in the target image and the fourth position information of the second key point in the target image distance information between points;

Based on the distance information, the telescopic length information of the second member is determined.

This embodiment is aimed at the situation where the posture information is telescopic length information, since the deep neural network model can accurately determine the first key point in each second component and the matching second key point set for the second component in advance, Therefore, the precise third position information and the fourth position information respectively corresponding to the accurate first key point and the second key point can be determined. Due to the accurate third position information and the fourth position information, according to the degree of change between the two, the accurate distance information between the first key point and the second key point can be determined, and then the accurate expansion and contraction of the second member can be determined. length information.

In an optional implementation manner, identifying the hydraulic support in the target image, and determining a plurality of target key points corresponding to at least one target component in the hydraulic support, includes:

determining the type information of the hydraulic support;

The target image is identified based on the type information, and multiple target key points corresponding to at least one target member in the hydraulic support are determined.

In this embodiment, due to the different types of hydraulic supports, the target key points used when detecting attitude information are different. Since different key points can be flexibly configured for different types of hydraulic supports in the training stage of the deep neural network model, in the state detection stage, the type information is used to identify the target image, and the hydraulic pressure corresponding to the type of information can be matched. The multiple target key points configured on the bracket improve the matching accuracy between the target key points and the target component.

In an optional implementation manner, determining the safety state of the hydraulic support based on the posture information of the hydraulic support includes:

acquiring first preset standard information corresponding to the first component;

In the case that the first inclination angle information of the first member matches the corresponding first preset standard information, it is determined that the safety state of the hydraulic support is safe.

In this embodiment, the first preset standard information may be the standard information of the first component in the working process obtained according to historical experience values or actual application scenarios; therefore, the detected accurate first inclination angle information is compared with the first preset standard information By comparing the standard information, the safety state of the hydraulic support can be predicted more accurately.

In an optional implementation manner, determining the safe state of the hydraulic support based on the posture information of the hydraulic support includes:

acquiring second preset standard information corresponding to the first component, and third preset standard information corresponding to the second component;

When the first inclination angle information of the first member matches the corresponding second preset standard information, and the telescopic length information of the second member matches the corresponding third preset standard information In this case, it is determined that the safety state of the hydraulic support is safe.

In this embodiment, the second preset standard information may be the standard information of the first component in the working process obtained according to historical experience values or actual working scenarios, and the third preset standard information may be the second component obtained according to the actual working scene Standard information in the working process, therefore, by comparing the detected accurate first inclination angle information with the second preset standard information, and comparing the detected telescopic length information with the second preset standard information, it is possible to predict A more accurate safety state of the hydraulic support can be obtained.

In an optional implementation manner, determining the safety state of the hydraulic support based on the posture information of the hydraulic support includes:

Obtain the posture information of other hydraulic supports working simultaneously with the hydraulic support;

Based on the situation that the posture information of the hydraulic support matches the posture information of the hydraulic supports in the other hydraulic supports, determine the matching number of the hydraulic supports in the other hydraulic supports;

When the number of matches is greater than a preset threshold, it is determined that the safety state of the hydraulic support is safe.

In this embodiment, during the working process of the hydraulic support, its safety state is also related to other hydraulic supports working at the same time. Therefore, when the attitude information of the hydraulic support is determined, the attitude information of other hydraulic supports working at the same time can also be carried out. By comparison, a more accurate safety state of the hydraulic support is further obtained.

In an optional implementation manner, the obtaining the target image includes:

acquiring the original image including the hydraulic support captured by the photographing device;

Identify the hydraulic support in the original image, and determine the detection frame corresponding to the hydraulic support;

The target image corresponding to the hydraulic support is intercepted from the original image based on the detection frame.

The detection frame corresponding to the hydraulic support is determined in this embodiment. The detection frame can include the entire hydraulic support in the smallest range. After that, a part of the original image framed by the detection frame is intercepted as the target image, that is, the size of the target image is smaller than the original image. , which can save the time for subsequent identification of the hydraulic support in the target image, thereby improving the efficiency of subsequent state detection.

In an optional embodiment, the target member includes a support beam in the hydraulic support.

In this embodiment, after it is determined that the hydraulic support includes a hydraulic support, the state detection method can be applied in the scene of a coal mine, which can accurately detect the attitude information of the support beam in the hydraulic support, and then determine the safety state of the hydraulic support, so as to be applied to the coal mine. Provides timely and effective security warnings.

In a second aspect, an embodiment of the present disclosure further provides a state detection device, including:

The image acquisition module is used to acquire the target image;

a key point determination module, configured to identify the hydraulic support in the target image, and determine a plurality of target key points corresponding to at least one target component in the hydraulic support;

an information determination module, configured to determine the attitude information of the hydraulic support based on the target position information of the target key point;

A state detection module, configured to determine the safe state of the hydraulic support based on the attitude information of the hydraulic support.

In an optional implementation manner, the target member includes a first member; the attitude information includes first inclination angle information of the first member relative to a preset coordinate axis in a preset coordinate system; the target The location information includes first location information;

The information determination module is configured to, for the first component, determine each of the target key points in the target image based on the first position information of each target key point in the target image. second position information of the target key point under the preset coordinate system; first inclination angle information of the preset coordinate axis in the preset coordinate system.

In an optional implementation manner, the target member further includes a second member; the posture information further includes telescopic length information of the second member; the target key point includes a first key point and a second key point ; The target position information includes third position information and fourth position information;

The information determination module is configured to, for the second component, determine a first key point located on the second component and a second key point corresponding to the second component; the second key point is located in the on other target components connected to the second component; based on the third position information of the first key point in the target image and the fourth position information of the second key point in the target image, determine distance information between the first key point and the second key point; based on the distance information, determine the telescopic length information of the second member.

In an optional implementation manner, the key point determination module is configured to determine the type information of the hydraulic support; identify the target image based on the type information, and determine at least one target in the hydraulic support Multiple target keypoints corresponding to the component.

In an optional implementation manner, the state detection module is configured to acquire first preset standard information corresponding to the first member; between the first inclination angle information of the first member and the corresponding In the case where the first preset standard information matches, it is determined that the safety state of the hydraulic support is safe.

In an optional implementation manner, the state detection module is configured to acquire second preset standard information corresponding to the first component and third preset standard information corresponding to the second component ; The first inclination angle information of the first member is matched with the corresponding second preset standard information, and the telescopic length information of the second member is matched with the corresponding third preset standard information In the case of , it is determined that the safety state of the hydraulic support is safe.

In an optional implementation manner, the state detection module is used to obtain the attitude information of other hydraulic supports working simultaneously with the hydraulic support; based on the attitude information of the hydraulic support, it is If the posture information of the hydraulic supports matches, determine the matching number of hydraulic supports in the other hydraulic supports that match; in the case that the matching number is greater than the preset threshold, determine that the safety state of the hydraulic supports is Safety.

In an optional implementation manner, the image acquisition module is configured to acquire an original image including the hydraulic support captured by a photographing device; identify the hydraulic support in the original image, and determine the corresponding hydraulic support. A detection frame; based on the detection frame, a target image corresponding to the hydraulic support is intercepted from the original image.

In an optional embodiment, the target member includes a support beam in the hydraulic support.

In a third aspect, embodiments of the present disclosure further provide a computer device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the computer device runs, the processing The processor and the memory communicate through a bus, and when the machine-readable instructions are executed by the processor, the first aspect or the steps of any possible state detection method in the first aspect are performed.

In a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the first aspect, or any one of the first aspect. Steps of a possible state detection method.

For the description of the effects of the above state detection apparatus, computer equipment and storage medium, please refer to the description of the above state detection method, which will not be repeated here.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

FIG. 1 shows a flowchart of a state detection method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram showing the target key points corresponding to each support beam in the hydraulic support provided by the embodiment of the present disclosure;

FIG. 3 is a schematic diagram showing the inclination angle of the fender provided by the embodiment of the present disclosure under various working states;

FIG. 4 is a schematic diagram showing the inclination angle of the roof beam provided by the embodiment of the present disclosure under various working states;

FIG. 5 is a schematic diagram showing a display state of the hydraulic support provided by the embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of a state detection apparatus provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

In addition, the terms "first", "second" and the like in the description and claims in the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein.

Reference herein to "a plurality or several" means two or more. "And/or", which describes the association relationship of the associated objects, means that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the associated objects are an "or" relationship.

The research found that the mine operation needs to use the hydraulic support to support the upper surface of the mine to prevent the ore from falling, resulting in the occurrence of dangerous situations. For example, the hydraulic support is a kind of protective equipment used to control the mine pressure of the coal mining face. During the coal mining process, the falling of the coal mine will exert a lateral force on the hydraulic support, causing the hydraulic support to swing sideways, thereby driving the hydraulic pressure. The support is skewed; or, due to the movement of a certain hydraulic support, the bearing capacity of multiple hydraulic supports is unbalanced, thereby destroying the hydraulic support, etc., which will cause the coal to fall and be dangerous. Today, by setting up multiple sensor devices on the hydraulic support, collecting multiple sets of data collected by the multiple sensor devices, and performing fusion processing, the current position state of the hydraulic support is predicted. However, the premise of the above prediction scheme is that sensor equipment needs to be installed on each beam of the hydraulic support to judge the posture change of the hydraulic support. Therefore, the cost requirements for the posture detection of the hydraulic support are relatively high; in addition, the working environment of coal mining is complex. , The sensor equipment on the hydraulic support is easy to be damaged, and the line of the sensor equipment is often damaged due to the movement of the hydraulic support, resulting in the failure of normal return of the collected data, which not only affects the work efficiency, but also increases the cost of maintaining and repairing the sensor equipment.

Based on the above research, the present disclosure provides a state detection method, device, computer equipment and storage medium. The deep neural network model can learn all the key points of the hydraulic support in advance, and can predict different targets for different key points. The pose information of the component. Therefore, by identifying the hydraulic support, determining multiple target key points corresponding to the target component, and using the target position information of the target key point, the posture information of the hydraulic support can be accurately calculated, such as the telescopic length of a certain target component, or The inclination angle of a target component, etc. By monitoring the posture information of the hydraulic support, the safety state of the hydraulic support, such as safe or dangerous, can be accurately obtained. After that, the staff can give timely warning when there is a dangerous situation according to the monitored safety status of the hydraulic support. Compared with the use of multiple sensors in the prior art, this aspect completely replaces the collection of data by traditional sensors. At the same time, since the way of acquiring images can be through existing surveillance cameras, there is no need to rearrange the shooting equipment, so the cost of the information acquisition equipment can be reduced; in addition, the above method is based on the deep neural network model for image detection, compared with the traditional method. The sensor fuses multiple collected data, which can improve the accuracy of the attitude information of the hydraulic support, and then can detect a relatively accurate safety state of the hydraulic support.

The defects existing in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions to the above problems proposed by the present disclosure hereinafter should be the inventors Contributions made to this disclosure during the course of this disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In order to facilitate the understanding of this embodiment, the application scenario of a state detection method disclosed in the embodiment of the present disclosure is first introduced in detail. Taking the application scenario of coal mining as an example, for the state detection of the hydraulic support, it is necessary to ensure the internal components of the hydraulic support. No deformation occurs to reduce the probability of safety accidents. Among them, the hydraulic support is a support device used to control the mine pressure of the coal mining face. The mine pressure of the coal collecting surface acts on the hydraulic support in the form of external load. In the mechanical system of the interaction between the hydraulic support and the surrounding rock of the coal collection surface, if the resultant force of each support of the hydraulic support and the external load force of the roof acting on the hydraulic support are exactly on the same line, the safe state of the hydraulic support can be determined as Safety.

Due to the continuous mining of the coal mine, the hydraulic support needs to move with the working face and adjust the attitude. It is determined to work on the current working face with the adjusted final attitude. Since it is uncertain whether the adjusted actual attitude information of the hydraulic support meets the requirements, therefore, External testing equipment is required for testing to further determine the safe state of the hydraulic support.

In addition, due to the movement of the top coal, an inclined force will also be generated on the hydraulic support, resulting in an unbalanced bearing capacity of the hydraulic support, thereby damaging the hydraulic support. Therefore, it is necessary to detect the safety state of the hydraulic support in real time to ensure coal mining in the mine. job security.

To this end, an embodiment of the present disclosure provides a state detection method, which will be described in detail below. The execution subject of the state detection method provided by the embodiment of the present disclosure is generally a computer device with a certain computing capability. In some possible implementations, the state detection method may be implemented by the processor calling computer-readable instructions stored in the memory.

The state detection method provided by the embodiment of the present disclosure will be described below by taking the execution subject as a computer device as an example.

Referring to FIG. 1, which is a flowchart of a state detection method provided by an embodiment of the present disclosure, the method includes steps S101-S104, wherein:

S101: Acquire a target image.

The target image obtained in this step includes the hydraulic support. Among them, the hydraulic support includes a guard plate, a front beam, a top beam, a shield beam, a front connecting rod, a base and other components.

In some embodiments, the target image may be an image obtained by marking the hydraulic support in the original image captured by the photographing device with a detection frame, and cropping a part of the original image corresponding to the hydraulic support marked by the detection frame.

S102: Identify the hydraulic support in the target image, and determine multiple target key points corresponding to at least one target component in the hydraulic support.

In this step, the deep neural network model can be used to identify the hydraulic support in the target image. In the training process, the deep neural network model uses a large number of sample images marked with key points of the hydraulic support for iterative training, and finally can learn all the hydraulic support. Therefore, in the process of identifying the target image, by determining the hydraulic support, a plurality of target key points corresponding to at least one target member in the hydraulic support can be determined.

The hydraulic support includes at least one target member. The target components may include support beams such as fenders, front beams, roof beams, shield beams, and front linkages.

Exemplarily, the multiple target key points corresponding to the target component may include multiple target key points set on the target component, or, a target key point set on the target component and other preset targets connected to the target component. At least one target key set on the component.

Continuing the above example, see FIG. 2 , which is a schematic diagram showing the target key points corresponding to each support beam in the hydraulic support. The multiple target key points corresponding to the guard board 21 may include the front end point 1 of the guard board and the rear end point 2 of the guard board; the multiple target key points corresponding to the front beam 22 may include the front end point 3 of the front beam and the front end point 4 of the top beam connected with the front beam; a plurality of target key points corresponding to the top beam 23 can include the front end point 4 of the top beam and the rear end point 5 of the top beam; a plurality of targets corresponding to the cover beam 24 The key points may include the front end point 6 of the shield beam and the rear end point 7 of the shield beam; the multiple target key points corresponding to the front link 25 may include the upper end point 8 of the front link and the lower end 9 of the front link.

In some embodiments, the hydraulic support may have multiple different types (including models and types, which are not specifically limited in the embodiments of the present disclosure). For different types of hydraulic supports, when the deep neural network model is used for the training process, the marked The key point locations are different. Therefore, in the process of identifying the hydraulic support in the target image, first, the type of the hydraulic support is determined, and then, according to the positions of the key points corresponding to the hydraulic support of the type learned by pre-training, determine the hydraulic support in the target image. Multiple target key points corresponding to at least one target component in the hydraulic support.

Here, since in the training stage of the deep neural network model, different key points can be flexibly configured for different types of hydraulic supports, therefore, in the state detection stage, the type information is used to identify the target image, which can be matched to the corresponding type of information. The multiple target key points configured by the hydraulic support improve the matching accuracy between the target key points and the target components.

S103: Determine the attitude information of the hydraulic support based on the target position information of the target key point.

In this step, the target position information of the target key points is the target position information of the plurality of target key points determined in S103 in the target image. The attitude information of the hydraulic support may include attitude information of at least one target member.

During specific implementation, a preset coordinate system can be established in the target image, and the target position information can be converted into coordinate information in the preset coordinate system; then, based on the coordinate information of multiple target key points, the attitude information of the target component is determined, Based on the attitude information of the at least one target member, attitude information of the hydraulic support is determined.

Exemplarily, the attitude information may include first tilt angle information of the target member relative to a preset coordinate axis in a preset coordinate system, and the first tilt angle information may include a tilt angle or a pitch angle. For example, taking the target component as the guard plate as an example, determine the first inclination angle information of the guard plate relative to the preset coordinate axis in the preset coordinate system, that is, the inclination angle α ₁ , as shown in FIG. 3 , which is Schematic diagram showing the inclination angle of the guard board in various working states, wherein 31 represents the preset coordinate system established in the target image, including the x-coordinate axis 311 and the y-coordinate axis 312; the solid line in FIG. 3 represents The straight line connecting the two target key points (ie P ₁ and P ₂ ) corresponding to the guard board, the dashed straight line represents the straight line parallel to the x-coordinate axis 311 , wherein the solid straight line includes the x-coordinate axis 311 as α ₁ =-180 The straight line 321 with α ₁ =-150° with the x-coordinate axis 311, the line 323 with α ₁ =-90° with the x-coordinate axis 311, and the line 324 with α ₁ =0° with the x-coordinate axis 311 .

Here, it can be known from the structure of the hydraulic support that the first inclination angle information of the front beam is the same as the first inclination angle information of the top beam. Therefore, when the first inclination angle information of the top beam is determined, the front beam can be determined. The first inclination angle information does not need to be calculated repeatedly.

For another example, taking the target member as the top beam as an example, determine the first inclination angle information of the top beam relative to the preset coordinate axis in the preset coordinate system, that is, the inclination angle α ₂ , as shown in FIG. 4 , which is the top beam Schematic diagram showing the tilt angle under various working states. Among them, 41 represents the preset coordinate system established in the target image, including the x-coordinate axis 411 and the y-coordinate axis 412; the solid line in FIG. ₄ represents the two target key points corresponding to the top beam (ie _P3 and P4 ), the dashed straight line represents a straight line parallel to the x-coordinate axis 411, wherein the solid straight line includes a straight line 421 which is α ₂ =30° with the x-coordinate axis 411, and a ₂ =-30° with the x-coordinate axis 411. Line 422.

S104: Determine the safe state of the hydraulic support based on the attitude information of the hydraulic support.

Here, the attitude information of the hydraulic support can represent the safety state of the hydraulic support. Specifically, the attitude information of each target component in the hydraulic support can meet the preset attitude requirements, and the safety state of the hydraulic support can be determined as Safety. If the attitude information of any target component in the hydraulic support does not meet the preset attitude requirements, it can be determined that the safety state of the hydraulic support is dangerous. Once the safety state of the hydraulic support is determined to be dangerous, in order to protect the life and safety of the staff, it is necessary to cut off the working power of the hydraulic support in time, and adjust the target components in the dangerous state in time under the condition of safety protection of the staff. The adjustment method may include adjusting the inclination angle of the target member according to the first preset standard information of the current state of the target member; or adjusting the telescopic length of the target member according to the third preset standard information. If the attitude information of any target component in the hydraulic support does not meet the preset attitude requirements, it can be determined that the safety state of the hydraulic support is dangerous. Once the safety state of the hydraulic support is determined to be dangerous, in order to protect the life and safety of the staff, it is necessary to cut off the working power of the hydraulic support in time, and adjust the target components in the dangerous state in time under the condition of safety protection of the staff. The adjustment method may include adjusting the inclination angle of the target member according to the first preset standard information of the current state of the target member; or adjusting the telescopic length of the target member according to the third preset standard information.

In some embodiments, the preset attitude requirement may include first preset standard information for the first tilt angle information. The first preset standard information may be defined according to historical experience values and actual working scenarios, which are not specifically limited in the embodiments of the present disclosure. For example, with respect to the inclination angle α ₁ of the guard board, when it is determined that the working scene of the guard board is in the closed state, the tilt angle of the guard board is detected, and at this time, the first inclination angle of the guard board can be determined. A preset standard information is α ₁ =-180°. If it is detected that the inclination angle α ₁ ≠-180°, it is determined that the safety state of the fender is dangerous. When it is determined that the working scene of the guard board is in the open state, the inclination angle of the guard board is detected, and at this time, it can be determined that the first preset standard information of the first tilt angle of the guard board is α ₁ =0°. If it is detected that the inclination angle α ₁ ≠0°, it is determined that the safety state of the fender is dangerous.

In the above S101 to S104, all the key points of the hydraulic support can be learned in advance through the deep neural network model, and the posture information of different target components can be predicted for different key points. Therefore, by identifying the hydraulic support, determining multiple target key points corresponding to the target component, and using the target position information of the target key point, the posture information of the hydraulic support can be accurately calculated, such as the telescopic length of a certain target component, or The inclination angle of a target component, etc. By monitoring the posture information of the hydraulic support, the safety state of the hydraulic support, such as safe or dangerous, can be accurately obtained. After that, the staff can give timely warning when there is a dangerous situation according to the monitored safety status of the hydraulic support. Compared with the use of multiple sensors in the prior art, this aspect completely replaces the collection of data by traditional sensors. At the same time, since the way of acquiring images can be through existing surveillance cameras, there is no need to rearrange the shooting equipment, so the cost of the information acquisition equipment can be reduced; in addition, the above method is based on the deep neural network model for image detection, compared with the traditional method. The sensor fuses multiple collected data, which can improve the accuracy of the attitude information of the hydraulic support, and then can detect a relatively accurate safety state of the hydraulic support.

For S103, in some embodiments, in the case where it is determined that the target member includes the first member, the attitude information of the hydraulic support is determined. At this time, the attitude information may include first inclination angle information of the first member relative to the preset coordinate axis in the preset coordinate system; the target position information may include first position information.

In a specific implementation, for the first component, based on the first position information of each target key point in the target image among the multiple target key points located in the first component, the position of each target key point in the preset coordinate system is determined respectively. second position information; based on the second position information of each target key point in the preset coordinate system, determine first inclination angle information of the first member relative to the preset coordinate axis in the preset coordinate system.

Here, according to the first position information of the target key point in the target image, the second position information of the target key point in the preset coordinate system is determined. Specifically, according to the preset coordinate system that has been established in the target image, The determined first position information is converted into coordinate information in the preset coordinate system, and the coordinate information is used as the second position information of the target key point in the preset coordinate system.

After that, by using the second position information of each target key point corresponding to the first component, the position of the first component in the preset coordinate system can be represented, for example, the coordinates of the two target key points in the preset coordinate system , the straight line equation of the first component in the preset coordinate system can be determined. Further, the straight line of the first member in the preset coordinate system can be determined, and the angle information between the straight line and the preset coordinate axis in the preset coordinate system can be used as the first inclination angle information of the first member, or, according to According to the needs of the detection task, the supplementary angle information of the included angle between the straight line and the preset coordinate axis in the preset coordinate system can also be used as the first inclination angle information of the first member.

Exemplarily, the first member may include a fender, a top beam, a shield beam, and a front link. Referring to FIG. 5 , which is a schematic diagram of a display state of the hydraulic support; wherein, the inclination angle of the guard plate relative to the preset coordinate axis in the preset coordinate system is α ₁ , and the top beam is relative to the preset coordinate system. _The inclination angle _of the preset coordinate axis in the The inclination angle is α ₄ , and 51 represents the preset coordinate system established in the target image, including the x-coordinate axis 511 and the y-coordinate axis 512; the line 521 connecting the target key point P ₁ and the target key point P ₂ in FIG. 5 Represents the guard board, the line 522 connecting the target key point P ₃ and the target key point P ₄ represents the top beam, the line 523 connecting the target key point P ₅ and the target key point P ₆ represents the cover beam, the target key point P ₇ and the target The line 524 connecting the key point _P8 represents the front link. The dashed straight line represents a straight line parallel to the x-coordinate axis 511 . Among them, the inclination angle α ₁ of the straight line 521 is the angle α ₁ with the x-coordinate axis 511 =-30°; the inclination angle α ₂ of the straight line 522 is the angle α ₂ with the x-coordinate axis 511 =30°; The inclination angle α ₃ of the line 523 is α ₃ =60° with the x-coordinate axis 511 ; the inclination angle α ₄ of the straight line 524 is α ₄ =150° with the x-coordinate axis 511 . Alternatively, it can also be determined that the inclination angle α ₄ =30° of the front link relative to the preset coordinate axis in the preset coordinate system, that is, the supplementary angle of the included angle with the x coordinate axis 511 .

Here, since the deep neural network model can accurately determine multiple target key points in each first component, the precise first position information of the target key point can be determined according to the multiple accurate target key points, and then the first key point can be obtained. The position information is converted into accurate second position information under the preset coordinate system. Since the first inclination angle information is obtained through analysis in the preset coordinate system, accurate first inclination angle information can be obtained by using the accurate second position information.

For S103, in some embodiments, in the case where it is determined that the target member includes the second member, the attitude information of the hydraulic support is determined. At this time, the posture information includes the telescopic length information of the second member; the multiple target key points corresponding to the second member may include the first key point and the second key point. The target location information may include third location information and fourth location information. Here, the second member includes a telescopic member, such as a front beam in a hydraulic support, and the front beam is sleeved in the top beam and is telescopic. During the working process of the front beam, the telescopic length can be adjusted according to the needs of the task. Exemplarily, when the current task is an extension task, the front beam can be controlled to extend a certain distance L ₁ ; when the next task is a retraction task, the front beam can be controlled to retract a certain distance from the current position. L ₂ . Here, the protruding amount L ₁ or the retracting amount L ₂ of the front beam is adjusted according to the needs of the task, and is not fixed.

The telescoping length information may include the amount of extension or retraction of the second member, eg, the amount of extension of the front beam, or the amount of retraction of the front beam. Taking the extension of the front beam as an example, the extension process of the front beam includes multiple states, for example, the stop state - the extension state - the stop state, and the length of the front beam extending from the previous stop state to the next stop state is the current Front beam extension during working hours. For another example, taking the retraction of the front beam as an example, the retraction process of the front beam includes multiple states, such as a stop state-retraction state-stop state, and the length of the front beam retracted from the previous stop state to the next stop state , which is the retraction amount of the front beam at the current moment.

During specific implementation, with respect to the second component, a first key point located on the second component and a second key point corresponding to the second component are determined; the second key point is located on other target components connected to the second component; based on the first key point on the second component The third position information of a key point in the target image and the fourth position information of the second key point in the target image, determine the distance information between the first key point and the second key point; based on the distance information, determine the second key point The telescopic length information of the component.

Here, according to the third position information of the first key point in the target image, the fifth position information of the first key point in the preset coordinate system is determined; according to the fourth position information of the second key point in the target image, The sixth position information of the second key point in the preset coordinate system is determined. Specifically, according to the preset coordinate system that has been established in the target image, the determined third position information can be converted into coordinate information in the preset coordinate system, and the coordinate information can be used as the first key point in the preset coordinate system. The fifth position information under the coordinate system; convert the determined fourth position information into coordinate information in the preset coordinate system, and use the coordinate information as the sixth position information of the second key point under the preset coordinate system .

After that, the coordinates indicated by the fifth position information of the first key point corresponding to the second component and the coordinates indicated by the sixth position information of the second key point can be used to determine the distance between the two coordinates, which can also be expressed as The distance between the first keypoint and the second keypoint. Here, the first key point may be a key point on the second component, and the second key point may be a key point on other target components connected to the second component.

For other target members, in some embodiments, in the case where the second member is a single member, the other target members may be other types of members connected to the second member, for example, when the second member is a front beam, and the front beam In the case of a single telescopic beam, the other target frame may be a top beam connected to the front beam.

In other embodiments, when there are multiple second members, other target members may include other types of other members (not the second member, such as a roof beam) connected to the second member, or include other target members connected to the second member. A second member of the same class to which the second member is connected. For example, the front beam includes multiple telescopic beams, namely the first telescopic beam, the second telescopic beam, and the third telescopic beam, wherein one end of the first telescopic beam is connected to the top beam, and the other end is telescopic with the second section Beam connection, one end of the third telescopic beam is connected with the second telescopic beam, and the other end is connected with the guard board; in the case where the second component is the first telescopic beam, other target structures can be connected with the first telescopic beam Connected roof beam; in the case where the second member is the second telescopic beam, the other target frame can be the first telescopic beam or the third telescopic beam connected to the second telescopic beam; when the second member is the first telescopic beam or the third telescopic beam; In the case of three telescopic beams, the other target frame may be the second telescopic beam connected to the third telescopic beam.

The key points on other target members corresponding to the second member can be defined in advance, that is, the second key point. Continuing the above example, when the second member is the front beam, and the front beam is a single telescopic beam, the pre-defined first The two key points may be the front end point 4 on the top beam connected to the front beam (see FIG. 2 ), and the first key point of the front beam may be the front end point 3 of the front beam (see FIG. 2 ). Continuing the above example, in the case where the second member is the front beam, and the front beam is a multi-section telescopic beam, the second key point is pre-defined. At this time, the first key point of the third section of the telescopic beam may be the front end point of the third section of the telescopic beam. Alternatively, the front end point of the first telescopic beam connected to the second telescopic beam is included. In this case, the first key point of the second telescopic beam may be the front end of the second telescopic beam. Alternatively, the front end point on the top beam connected to the first telescopic beam section is included. In this case, the first key point of the first telescopic beam section may be the front end point of the first telescopic beam section.

Afterwards, according to the distance between the first key point and the second key point, the telescopic length information of the second member is determined. The distance is the extension length of the front beam. If there are multiple sets of first key points and second key points, respectively determine the distance between each set of first key points relative to the second key point, and accumulate them to obtain the total extension length of the front beam.

Here, since the deep neural network model can accurately determine the first key point in each second component and the matching second key point set for the second component in advance, it can be based on the accurate first key point and the first key point. The two key points determine their corresponding precise third position information and fourth position information respectively. Due to the accurate third position information and the fourth position information, according to the degree of change between the two, the accurate distance information between the first key point and the second key point can be determined, and then the accurate expansion and contraction of the second member can be determined. length information.

For S104, when the posture information of the hydraulic support is determined, the safety state of the hydraulic support may be further determined based on the posture information of the hydraulic support.

如果检测出护帮板的倾斜角

则确定该护帮板的安全状态为安全；同理，分别确定液压支架中的顶梁、掩护梁、前连杆等支架梁的倾斜角，在每个支架梁的倾斜角分别位于与其各自对应的第一预设标准信息所指示的预设倾斜角范围的情况下，确定顶梁、掩护梁、前连杆的安全状态为安全，即确定液压支架的安全状态为安全。反之，针对液压支架中的任意一个第一构件，如果任意一个第一构件的第一倾斜角信息与其对应的第一预设标准信息不相匹配，即，任意一个第一构件的倾斜角不位于预设倾斜角范围，则可以确定该液压支架的安全状态为危险。延续上例，如果检测出护帮板的倾斜角

或

则确定该护帮板的安全状态为危险。此时不必再继续确定其他第一构件的安全状态，即可确定液压支架的安全状态为危险。The safety state of the hydraulic support is judged by using the first inclination angle information. In some embodiments, first, the first preset standard information corresponding to the first member may be obtained first; then, when the first inclination angle information of the first member matches the corresponding first preset standard information Only then can the safety state of the hydraulic support be determined to be safe. Exemplarily, for each first member in the hydraulic support, when the inclination angle of each first member is located in the preset inclination angle range indicated by the corresponding first preset standard information, it can be determined that the The safe state of the hydraulic support is safe. For example, with respect to the inclination angle α ₁ of the guard plate, when it is determined that the working scene of the guard plate is in the closed state, the inclination angle of the guard plate is detected, and the preset inclination angle range in the closed state of the guard plate is determined, which is

If the tilt angle of the guard plate is detected

Then it is determined that the safety state of the guard plate is safe; in the same way, the inclination angles of the support beams such as the top beam, the cover beam, and the front connecting rod in the hydraulic support are respectively determined, and the inclination angle of each support beam is located at its corresponding In the case of the preset tilt angle range indicated by the first preset standard information, the safety state of the top beam, the cover beam, and the front link is determined to be safe, that is, the safety state of the hydraulic support is determined to be safe. On the contrary, for any first member in the hydraulic support, if the first inclination angle information of any first member does not match its corresponding first preset standard information, that is, the inclination angle of any first member is not located in With a preset tilt angle range, it can be determined that the safe state of the hydraulic support is dangerous. Continuing the above example, if the tilt angle of the guard plate is detected

or

Then it is determined that the safety state of the guard board is dangerous. At this time, it is not necessary to continue to determine the safety state of other first components, and it can be determined that the safety state of the hydraulic support is dangerous.

Here, the preset inclination angle ranges of the first component in different working states are different, and the preset inclination angle range indicated by the first preset standard information may be defined according to historical experience values or actual working scenarios. No specific limitation is made.

Exemplarily, the hydraulic support in the coal mine needs to cooperate with the coal cutter and the scraper to realize the three-machine linkage work in actual work. Plate collision accident, resulting in a major safety production accident. If the guard plate in the hydraulic support is not opened in time after being stowed, it will also lead to accidents such as the collapse of the roof, affecting the safety of underground workers. Therefore, when it is determined that the safety state of the hydraulic support is dangerous, it is necessary to send early warning information to the staff in time, or directly control the background system to stop the hydraulic support to avoid underground safety and production accidents.

The safety state of the hydraulic support is judged by using the first inclination angle information and the telescopic length information. In other embodiments, the second preset standard information corresponding to the first member and the third preset standard information corresponding to the second member may also be acquired; after that, at the first inclination angle of the first member When the information matches the corresponding second preset standard information, and the telescopic length information of the second member matches the corresponding third preset standard information, it is determined that the safety state of the hydraulic support is safe. Conversely, if the first inclination angle information of any first member in the hydraulic support does not match its corresponding second preset standard information, or, the telescopic length information of any second member in the hydraulic support and its corresponding first If the three preset standard information do not match, it can be determined that the safety state of the hydraulic support is dangerous.

Here, both the second preset standard information and the third preset standard information may be defined according to historical experience values or actual working scenarios, which are not specifically limited in the embodiments of the present disclosure.

Exemplarily, only the inclination angles corresponding to the fenders, the top beam, the shield beam and the front link are all within the preset inclination angle range indicated by the corresponding second preset standard information, and the telescopic length of the front beam is Only when the information matches the third preset standard information, it can be determined that the safety state of the hydraulic support is safe. If any one of the support beams, such as the front beam, is determined to be in the fully extended state, it is known that the third preset standard information corresponding to the front beam in the fully extended state indicates that the extension of the front beam is 1.5m , at this time, if it is determined that the actual extension length of the front beam is 1m, the safety state of the front beam is determined to be dangerous, and then the safety state of the entire hydraulic support is determined to be dangerous.

Here, the telescopic length information of the front beam is detected, that is, the actual telescopic length, including the extension of the front beam or the retraction of the front beam. Specifically, the distance between adjacent pixels in the known target image, and the actual length and the image are known. The ratio C of the length, since the ratio remains unchanged, the actual telescopic length L of the front beam can be determined by detecting the distance L ₃ between the first key point and the second key point of the front beam in the target image, that is, the image length ₄ =C×L ₃ .

In addition, the second inclination angle information of the second member may be the first inclination angle information of other target members connected to the second member. In some embodiments, the first inclination angle information of the first member matches the corresponding second preset standard information, the telescopic length information of the second member matches the corresponding third preset standard information, and the second When the second inclination angle information of the component matches the corresponding fourth preset standard information, it is determined that the safety state of the hydraulic support is safe. The fourth preset standard information may include second preset standard information corresponding to other target components connected to the second component.

In other embodiments, the posture information of other hydraulic supports working simultaneously with the hydraulic support may be recorded in real time, and there may be multiple other hydraulic supports. Only by ensuring that most or all of the hydraulic supports working at the same time have the same attitude information, the safe state of the mine working environment can be ensured. Therefore, in order to determine the safety state of the hydraulic support, the detected attitude information of the hydraulic support can also be matched with the attitude information of other hydraulic supports working simultaneously with the hydraulic support.

In the specific implementation, the attitude information of other hydraulic supports working simultaneously with the hydraulic support can be obtained first; based on the attitude information of the hydraulic support, if the attitude information of the hydraulic support in the other hydraulic supports matches with the attitude information of the hydraulic supports in the other hydraulic supports, other matching hydraulic supports are determined. The matching number of hydraulic supports in the middle; when the matching number is greater than the preset threshold, it is determined that the safety state of the hydraulic supports is safe.

Here, the preset threshold may be set according to the total number of hydraulic supports working simultaneously, actual application scenarios, and empirical values, which are not specifically limited in this embodiment of the present disclosure. For example, in general, there are 10 hydraulic supports working at the same time, namely 1, 2, ... 10. If the current hydraulic support 10 matches the posture information of the first 8 hydraulic supports (2-9), only If the posture information of the hydraulic support 1 does not match, it can be determined that the current safety state of the hydraulic support 10 is safe, and the safety state of the hydraulic support 1 is dangerous.

Exemplarily, taking a coal mining application scenario as an example, there are multiple hydraulic supports working together, and it is necessary to ensure that the attitude information of each hydraulic support is the same. Therefore, when detecting the safety state of the hydraulic support, it is only possible to compare whether the attitude information of the hydraulic support matches the attitude information of the previous hydraulic support (that is, other hydraulic supports) of the hydraulic support, and if they match, the safety of the hydraulic support can be determined. Status is safe. If it does not match, it is determined that the safety state of the hydraulic support is dangerous, and then the entire coal mining task is determined to be dangerous, and early warning information is sent to the staff law in time.

For S101, the process of determining the target image, specifically, acquiring the original image including the hydraulic support captured by the photographing device; identifying the hydraulic support in the original image, and determining the detection frame corresponding to the hydraulic support; intercepting the original image based on the detection frame The target image corresponding to the hydraulic support.

Here, the photographing device may be a camera, etc., a device for photographing the hydraulic support. The detection frame is the identification information marking the hydraulic support in the original image, including coordinate information indicating the position of the detection frame. According to the coordinate information indicating the position of the detection frame, determine the region image of the hydraulic support in the original image (ie, part of the original image), and cut the region image from the original image to obtain the target image corresponding to the hydraulic support.

Exemplarily, a target detection network, such as Faster RCNN, SSD, YOLO v2&v3 and other detection networks, can be used to identify the hydraulic support in the original image, and determine the detection frame corresponding to the hydraulic support.

Here, the detection frame corresponding to the hydraulic support is determined, and the detection frame can include the entire hydraulic support in the smallest range. After that, a part of the original image framed by the detection frame is intercepted as the target image, that is, the size of the target image is smaller than the original image. The time for subsequent identification of the hydraulic support in the target image is saved, thereby improving the efficiency of subsequent state detection.

In some embodiments, the embodiments of the present disclosure may be performed using a pre-trained deep neural network model, wherein the training process of the deep neural network model may include the following steps:

Step 1. Obtain the target data set to be trained.

Among them, the target dataset may include multiple pictures containing hydraulic supports. Exemplarily, the picture may be a picture of a hydraulic support with different postures extracted from a video of the hydraulic support.

Step 2: For each picture, obtain the mark information input by the user for marking the hydraulic support in the picture, and perform mark processing for the picture to obtain preset key points corresponding to the hydraulic support.

Wherein, the marking information may be position information of preset key points.

Step 3. Identify the hydraulic support in each picture, and determine the training detection frame corresponding to the hydraulic support.

Here, the process of determining the training detection frame of the hydraulic support in the training process can be referred to as the process of determining the detection frame of the hydraulic support in the state detection process in practical applications, and the repeated parts will not be repeated here.

Step 4: Determine a training image corresponding to the training detection frame based on the training detection frame corresponding to the hydraulic support.

Specifically, a part of the picture corresponding to the training detection frame may be cut out to obtain the training image.

Step 5. Use the training images marked with preset key points to train the deep neural network model to be trained to obtain a trained deep neural network model.

Here, the trained target neural network can output the position information of the target key points corresponding to the hydraulic support.

Exemplarily, the above-mentioned deep neural network model may be an attitude estimation model, such as HRnet in attitude estimation mmpose.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same inventive concept, the embodiment of the present disclosure also provides a state detection device corresponding to the state detection method. Reference may be made to the implementation of the method, and repeated descriptions will not be repeated.

Referring to FIG. 6 , which is a schematic diagram of a state detection device provided by an embodiment of the present disclosure, the device includes: an image acquisition module 601 , a key point determination module 602 , an information determination module 603 and a state detection module 604 ; wherein,

an image acquisition module 601, configured to acquire a target image;

A key point determination module 602, configured to identify the hydraulic support in the target image, and determine a plurality of target key points corresponding to at least one target member in the hydraulic support;

an information determination module 603, configured to determine the attitude information of the hydraulic support based on the target position information of the target key point;

The state detection module 604 is configured to determine the safe state of the hydraulic support based on the attitude information of the hydraulic support.

In an optional implementation manner, the target member includes a first member; the attitude information includes first inclination angle information of the first member relative to a preset coordinate axis in a preset coordinate system; the target The location information includes first location information;

The information determination module 603 is configured to, for the first component, determine, respectively, each target key point based on the first position information of each target key point in the target image among the multiple target key points of the first component. second position information of each of the target key points under the preset coordinate system; First inclination angle information of a preset coordinate axis in the preset coordinate system.

In an optional implementation manner, the target member further includes a second member; the posture information further includes telescopic length information of the second member; the target key point includes a first key point and a second key point ; The target position information includes third position information and fourth position information;

The information determination module 603 is configured to determine, for the second component, a first key point located on the second component and a second key point corresponding to the second component; the second key point is located at on other target components connected to the second component; based on the third position information of the first key point in the target image and the fourth position information of the second key point in the target image, Determine the distance information between the first key point and the second key point; and determine the telescopic length information of the second member based on the distance information.

In an optional implementation manner, the key point determination module 602 is configured to determine the type information of the hydraulic support; identify the target image based on the type information, and determine at least one of the hydraulic supports. Multiple target key points corresponding to the target component.

In an optional implementation manner, the state detection module 604 is configured to acquire first preset standard information corresponding to the first member; When the first preset standard information matches, it is determined that the safety state of the hydraulic support is safe.

In an optional implementation manner, the state detection module 604 is configured to acquire second preset standard information corresponding to the first component, and a third preset standard corresponding to the second component. information; the first inclination angle information of the first member matches the corresponding second preset standard information, and the telescopic length information of the second member matches the corresponding third preset standard information In the case of matching, it is determined that the safety state of the hydraulic support is safe.

In an optional implementation manner, the state detection module 604 is used to acquire the attitude information of other hydraulic supports working simultaneously with the hydraulic support; based on the attitude information of the hydraulic support, it is If the posture information of the hydraulic supports matches, determine the matching number of hydraulic supports in the other hydraulic supports that match; in the case that the matching number is greater than the preset threshold, determine the safety state of the hydraulic supports for safety.

In an optional implementation manner, the image acquisition module 601 is configured to acquire an original image including the hydraulic support captured by a photographing device; identify the hydraulic support in the original image, and determine the corresponding hydraulic support. The detection frame; based on the detection frame, the target image corresponding to the hydraulic support is intercepted from the original image.

In an optional embodiment, the target member includes a support beam in the hydraulic support.

For the description of the processing flow of each module in the state detection apparatus and the interaction flow between the modules, reference may be made to the relevant description in the above-mentioned embodiment of the state detection method, which will not be described in detail here.

Based on the same technical concept, the embodiments of the present application also provide a computer device. Referring to FIG. 7 , a schematic structural diagram of a computer device provided by an embodiment of the present application includes:

Processor 71 , memory 72 and bus 73 . The memory 72 stores machine-readable instructions executable by the processor 71, and the processor 71 is configured to execute the machine-readable instructions stored in the memory 72. When the machine-readable instructions are executed by the processor 71, the processor 71 executes the instructions. The following steps: S101: acquire a target image; S102: identify the hydraulic support in the target image, and determine multiple target key points corresponding to at least one target component in the hydraulic support; S103: determine the target position information based on the target key point attitude information of the hydraulic support; S104: Determine the safety state of the hydraulic support based on the attitude information of the hydraulic support.

The above-mentioned memory 72 includes a memory 721 and an external memory 722; the memory 721 here is also called an internal memory, which is used to temporarily store the operation data in the processor 71 and the data exchanged with the external memory 722 such as the hard disk. The external memory 722 performs data exchange. When the computer device is running, the processor 71 communicates with the memory 72 through the bus 73, so that the processor 71 executes the execution instructions mentioned in the above method embodiments.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the state detection method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present disclosure further provide a computer program product, including computer instructions, which implement the steps of the above state detection method when the computer instructions are executed by a processor. Wherein, the computer program product can be any product that can realize the above state detection method, and some or all of the solutions that contribute to the prior art in the computer program product can be software products (for example, a software development kit (Software Development Kit, SDK)) ), the software product can be stored in a storage medium, and the computer instructions contained therein cause the relevant device or processor to execute some or all of the steps of the above state detection method.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined. Or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or modules, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present disclosure may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module.

If the functions are implemented in the form of software function modules and sold or used as independent products, they may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure rather than limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing The embodiments describe the present disclosure in detail. Those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims (12)

1. a state detection method, is characterized in that, comprises: get the target image; Identifying the hydraulic support in the target image, and determining a plurality of target key points corresponding to at least one target component in the hydraulic support; Determine the attitude information of the hydraulic support based on the target position information of the target key point; Based on the attitude information of the hydraulic support, a safe state of the hydraulic support is determined. 2 . The method according to claim 1 , wherein the target member comprises a first member; and the attitude information comprises a first inclination of the first member relative to a preset coordinate axis in a preset coordinate system. 3 . angle information; the target position information includes first position information; The determining of the posture information of the hydraulic support based on the target position information of the target key points includes: For the first component, based on the first position information of each target key point in the target image among the plurality of target key points located in the first component, it is determined that each target key point is located in the target image. the second position information in the preset coordinate system; Based on second position information of each of the target key points in the preset coordinate system, first inclination angle information of the first member relative to a preset coordinate axis in the preset coordinate system is determined. 3. The method according to claim 2, wherein the target member further comprises a second member; the posture information further comprises telescopic length information of the second member; the target key point comprises a first key point and a second key point; the target position information includes third position information and fourth position information; The determining of the posture information of the hydraulic support based on the target position information of the target key points includes: For the second component, determine a first key point located on the second component and a second key point corresponding to the second component; the second key point is located on other key points connected to the second component on the target component; The first key point and the second key point are determined based on the third position information of the first key point in the target image and the fourth position information of the second key point in the target image distance information between points; Based on the distance information, the telescopic length information of the second member is determined. 4. The method according to any one of claims 1 to 3, wherein the identifying the hydraulic support in the target image is to determine a plurality of target key points corresponding to at least one target member in the hydraulic support ,include: determining the type information of the hydraulic support; The target image is identified based on the type information, and multiple target key points corresponding to at least one target member in the hydraulic support are determined. 5. The method according to claim 2, wherein the determining the safety state of the hydraulic support based on the attitude information of the hydraulic support comprises: acquiring first preset standard information corresponding to the first component; In the case that the first inclination angle information of the first member matches the corresponding first preset standard information, it is determined that the safety state of the hydraulic support is safe. 6 . The method according to claim 3 , wherein the determining the safety state of the hydraulic support based on the attitude information of the hydraulic support comprises: 6 . acquiring second preset standard information corresponding to the first component, and third preset standard information corresponding to the second component; When the first inclination angle information of the first member matches the corresponding second preset standard information, and the telescopic length information of the second member matches the corresponding third preset standard information In this case, it is determined that the safety state of the hydraulic support is safe. 7. The method according to any one of claims 1 to 6, wherein the determining the safety state of the hydraulic support based on the posture information of the hydraulic support comprises: Obtain the posture information of other hydraulic supports working simultaneously with the hydraulic support; Based on the situation that the posture information of the hydraulic support matches the posture information of the hydraulic supports in the other hydraulic supports, determine the matching number of the hydraulic supports in the other hydraulic supports; When the number of matches is greater than a preset threshold, it is determined that the safety state of the hydraulic support is safe. 8. The method according to any one of claims 1 to 7, wherein the acquiring a target image comprises: acquiring the original image including the hydraulic support captured by the photographing device; Identify the hydraulic support in the original image, and determine the detection frame corresponding to the hydraulic support; The target image corresponding to the hydraulic support is intercepted from the original image based on the detection frame. 9. The method of any one of claims 1 to 8, wherein the target member comprises a support beam in the hydraulic support. 10. A state detection device, characterized in that, comprising: The image acquisition module is used to acquire the target image; a key point determination module, configured to identify the hydraulic support in the target image, and determine a plurality of target key points corresponding to at least one target component in the hydraulic support; an information determination module, configured to determine the attitude information of the hydraulic support based on the target position information of the target key point; A state detection module, configured to determine the safe state of the hydraulic support based on the attitude information of the hydraulic support. 11. A computer device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device runs, the processor and the The memories communicate with each other through a bus, and when the machine-readable instructions are executed by the processor, the steps of the state detection method according to any one of claims 1 to 9 are executed. 12. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the state detection method according to any one of claims 1 to 9 when the computer program is run by a processor A step of.

Images