CN113780706A - On-site operation and maintenance operation method and device based on visual enhancement - Google Patents
On-site operation and maintenance operation method and device based on visual enhancement Download PDFInfo
- Publication number
- CN113780706A CN113780706A CN202011167513.3A CN202011167513A CN113780706A CN 113780706 A CN113780706 A CN 113780706A CN 202011167513 A CN202011167513 A CN 202011167513A CN 113780706 A CN113780706 A CN 113780706A
- Authority
- CN
- China
- Prior art keywords
- image
- maintenance
- maintained
- state image
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012423 maintenance Methods 0.000 title claims abstract description 134
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000000007 visual effect Effects 0.000 title claims abstract description 41
- 230000006854 communication Effects 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 12
- 238000005259 measurement Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 11
- 238000004590 computer program Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 3
- 238000010561 standard procedure Methods 0.000 description 3
- 239000002131 composite material Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 238000011282 treatment Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000011418 maintenance treatment Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000001556 precipitation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Life Sciences & Earth Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a field operation and maintenance operation method and device based on visual enhancement, wherein the method comprises the following steps: acquiring an identifier of equipment to be maintained for a field operation and maintenance task; calling an operation flow sheet corresponding to the field operation and maintenance task according to the identification, wherein the operation flow sheet comprises at least one reference state image of the equipment to be maintained during the field operation and maintenance task and an operation to be executed corresponding to the reference state image; acquiring current space image information of equipment to be maintained, displaying the space image information, and determining a current state image of the equipment to be maintained according to the space image information; matching the current state image with at least one reference state image, and if the matched reference state image is matched, determining the operation to be executed corresponding to the matched reference state image; and acquiring a guide image and/or a guide animation by combining the display visual enhancement with the spatial image of the equipment to be operated and maintained, and displaying the guide image and/or the guide animation.
Description
Technical Field
The disclosure relates to the technical field of computer technology, and more particularly, to a field operation and maintenance operation method and device based on visual enhancement.
Background
With the enlargement of the scale of the data center, more workers are needed to cover more equipment operation and maintenance work under the current operation and maintenance work efficiency; with the new technologies of various data centers, the infinite architecture design, the innovation of hardware technology and the change of knowledge bodies of field operation and maintenance work bring new challenges; on-site operation and maintenance personnel need to continuously learn operation and maintenance treatment schemes of different equipment, so that higher requirements are generated on the knowledge storage and professional skills of the operation and maintenance personnel.
In the course of implementing the disclosed concept, the inventors found that there are at least the following problems in the prior art:
1) most of the operation manuals or SOPs of the current operation and maintenance work are documents maintained by paper or electronic versions, and operation and maintenance personnel need to be familiar with the content of the manuals before the operation is executed; in the operation and maintenance execution process, the content in the manual document and the actual situation on site are mutually split, and the manual document is difficult to visually contrast when facing a complex mechanical structure or the situation on site, so that the intuitive and efficient guidance function cannot be generated.
2) Data centers are distributed all over the country, and experienced operation and maintenance engineers with high qualification need to frequently go on a business trip if going to carry out operation and maintenance on site, so that the traffic and time costs are high;
3) if a complicated problem is encountered in the field operation and maintenance execution process, remote request assistance is generally adopted as telephone voice communication or video guidance, but the guidance process is not intuitive enough, communication may need to be confirmed repeatedly, and the guidance efficiency is poor.
Disclosure of Invention
In view of the above, the present disclosure provides a field operation and maintenance operation method and device based on visual enhancement, which can intuitively guide the field operation and maintenance operation.
One aspect of the present disclosure provides a field operation and maintenance operation method based on visual enhancement, including: acquiring an identifier of equipment to be maintained for a field operation and maintenance task; calling an operation flow sheet corresponding to the field operation and maintenance task according to the identification, wherein the operation flow sheet comprises at least one reference state image of equipment to be maintained during the field operation and maintenance task and an operation to be executed corresponding to the reference state image; acquiring current space image information of the equipment to be maintained, displaying the space image information, and determining a current state image of the equipment to be maintained according to the space image information; matching the current state image with the at least one reference state image, and if a matched reference state image is matched, determining an operation to be executed corresponding to the matched reference state image; and acquiring a guide image and/or a guide animation by combining the display visual enhancement with the operation to be performed and the space image of the equipment to be maintained, and displaying the guide image and/or the guide animation.
According to an embodiment of the present disclosure, the acquiring the current spatial image information of the device to be maintained includes: and acquiring a real-time image of the equipment to be maintained through image acquisition, and acquiring ranging information through acquiring the time when light touches the equipment to be maintained and returns.
According to an embodiment of the present disclosure, the method further comprises: and if the matched reference state image is not matched, sending a remote assistance request to a remote server, and acquiring the remote assistance sent by the remote server.
According to an embodiment of the present disclosure, the acquiring that the remote server sends the remote assistance includes: and receiving a spatial image information acquisition instruction sent by a remote server, and sending the spatial image information of the equipment to be maintained to the remote server.
According to an embodiment of the present disclosure, the determining the current state image of the device to be maintained according to the aerial image information includes: and displaying the image of the equipment to be operated and maintained at the preset angle and distance obtained by visual enhancement by utilizing the spatial image information as a current state image.
According to an embodiment of the present disclosure, the acquiring the remote assistance sent by the remote server further includes: and receiving an image annotation instruction sent by a remote server, and annotating the space image according to the image annotation instruction.
According to an embodiment of the present disclosure, the acquiring the remote assistance sent by the remote server further includes: and carrying out voice communication or video communication with the far-field server.
According to an embodiment of the present disclosure, the obtaining an identifier of a device to be maintained for a field operation and maintenance task includes: and acquiring the equipment bar code through image acquisition, or acquiring the equipment type through image identification.
According to an embodiment of the present disclosure, the method further comprises: and responding to an operation and maintenance operation guiding instruction from a user, sending a request to a server, and acquiring a field operation and maintenance operation task and an operation flow sheet corresponding to the field operation and maintenance operation task.
Another aspect of the present disclosure provides a field operation and maintenance work device based on visual enhancement, including: the identification acquisition module is used for acquiring an identification of the equipment to be maintained, which is aimed at by the field operation and maintenance task; the operation flow sheet calling module is used for calling an operation flow sheet corresponding to the field operation and maintenance operation task according to the identification, wherein the operation flow sheet comprises at least one reference state image of the equipment to be maintained and corresponding operation to be executed during the field operation and maintenance operation task; the state acquisition module is used for acquiring a real-time image of the equipment to be maintained and acquiring a current state image of the equipment to be maintained according to the real-time image; the matching module is used for matching the current state image with the reference state image in the operation flow sheet, and if the matched reference state image is matched, determining the operation to be executed corresponding to the matched reference state image; and the display module acquires a guide image and/or a guide animation by combining the display visual enhancement with the operation to be executed and the spatial image of the equipment to be maintained, and displays the guide image and/or the guide animation.
According to an embodiment of the present disclosure, the apparatus further comprises: and the remote assistance module is used for realizing remote assistance by interacting with the remote server.
Another aspect of the present disclosure provides an electronic device including: one or more processors; storage means for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as previously described.
Another aspect of the disclosure provides a computer-readable storage medium having stored thereon executable instructions for, when executed, implementing a method as previously described.
According to the embodiment of the disclosure, the operation and maintenance operation disposal difficulty can be at least partially reduced, the problem that the operation and maintenance operation manual and the actual situation are mutually split is solved, and therefore the technical effect of improving the operation and maintenance efficiency can be achieved.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an exemplary system architecture for a visual enhancement based on field operations and maintenance operations method and apparatus according to an embodiment of the present disclosure;
FIG. 2A schematically illustrates a flow diagram of a method of a visual enhancement based on-site operation and maintenance work according to an embodiment of the present disclosure;
FIG. 2B schematically illustrates a diagram of a tutorial interface obtained with display visual enhancement according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow diagram of a method of a visual enhancement based on-site operation and maintenance work according to yet another embodiment of the present disclosure;
FIG. 4A schematically illustrates a flow diagram of a method of a visual enhancement based on-site operation and maintenance work according to yet another embodiment of the present disclosure;
fig. 4B schematically shows a flowchart of a method for acquiring that the remote server sends remote assistance according to an embodiment of the present disclosure;
fig. 4C schematically illustrates obtaining the remote server sending remote assistance diagram according to an embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of a visual enhancement based on-site operation and maintenance work device, according to an embodiment of the present disclosure;
fig. 6 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides a field operation and maintenance operation method based on visual enhancement, which comprises the following steps: acquiring an identifier of equipment to be maintained for a field operation and maintenance task; calling an operation flow sheet corresponding to the field operation and maintenance task according to the identification, wherein the operation flow sheet comprises at least one reference state image of equipment to be maintained during the field operation and maintenance task and an operation to be executed corresponding to the reference state image; acquiring current space image information of the equipment to be maintained, displaying the space image information, and determining a current state image of the equipment to be maintained according to the space image information; matching the current state image with the at least one reference state image, and if a matched reference state image is matched, determining an operation to be executed corresponding to the matched reference state image; and acquiring a guide image and/or a guide animation by combining the display visual enhancement with the operation to be performed and the space image of the equipment to be maintained, and displaying the guide image and/or the guide animation.
Fig. 1 schematically illustrates an example system architecture 100 for a visual enhancement based on field operation and maintenance work method and apparatus according to an embodiment of this disclosure. It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the system architecture 100 according to this embodiment may include a terminal device 101, remote servers 102, 103, a network 104, and an operation and maintenance server 105. The network 104 is used to provide a medium for a communication link between the terminal device 101 and the operation and maintenance server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal device 101 to interact with operation and maintenance server 105 via network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal device 101.
The terminal device 101 may be various electronic devices having a display screen and supporting network communication, including but not limited to a smartphone, tablet, laptop, desktop, and AR/MR glasses, etc. having a visual enhancement function. Further, the terminal device 101 has a scanning device or a camera, and acquires a device barcode through the scanning device, or identifies and determines the device type through a camera image, where the device barcode and the device type are used as identifiers of devices to be maintained for the on-site operation and maintenance task. And the operation flow sheet corresponding to the field operation and maintenance operation task can be called through the identification.
It is understood that the equipment to be maintained may be any equipment entity requiring manual operation, such as servers, diesel engines, air conditioners, power system equipment, water circulation system equipment, valves, water pumps and other common equipment of data centers, and the equipment to be maintained is not limited to the equipment of the data centers.
Further, the terminal device 101 may further have a spatial image collecting device, such as a camera of a composite LiDAR, where the spatial image collecting device acquires a real-time image of the device to be maintained through image collection, acquires distance measuring information through acquiring time when light touches the device to be maintained and returns, and acquires a spatial image of the device to be maintained through combination of the two, including the size of the device to be maintained and the distance between the terminal device 101 and the device to be maintained.
The operation and maintenance server 105 may be a server that provides various services, such as a background management server (for example only) that provides support for a website browsed by a user using the terminal device 101. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
The remote servers 102 and 103 may adopt remote terminal devices or remote servers for providing intuitive expert guidance operation in the field operation and maintenance operation process. In the operation and maintenance handling process, if a more complex handling process is encountered, the current operation and maintenance personnel are difficult to complete, and remote assistance can be initiated to the remote server 102 or the remote server 103 through the terminal device 101 at any time to request the current operation and maintenance expert on duty. In the remote assistance process, the operation and maintenance expert may mark the operation content on the real-time screen by a marking tool provided in the application based on the screen currently captured by the terminal device 101. And displaying the operation content to field personnel in a 3D human-computer interface prompting mode of displaying the enhanced display.
The remote servers 102 and 103 may be different guidance experts, and a specific remote server may be specified by the system or determined according to a task of the on-site operation and maintenance work.
It should be noted that the field operation and maintenance operation method based on visual enhancement provided by the embodiment of the present disclosure may be generally executed by the terminal device 101. Accordingly, the field operation and maintenance work device based on visual enhancement provided by the embodiment of the present disclosure may be generally disposed in the terminal device 101. The field operation and maintenance operation method based on visual enhancement provided by the embodiment of the disclosure can also be executed by a terminal device which is different from the terminal device 101 and can communicate with the operation and maintenance server 105. Accordingly, the field operation and maintenance operation device based on visual enhancement provided by the embodiment of the present disclosure may also be disposed in a terminal device different from the terminal device 101 and capable of communicating with the operation and maintenance server 105.
It should be understood that the number of terminal devices, remote servers, networks, and servers in fig. 1 are merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2A schematically illustrates a flow diagram of a method of a visual enhancement based on-site operation and maintenance work according to an embodiment of the present disclosure.
As shown in fig. 2A, the method includes acquiring an identifier of a device to be maintained for a field operation and maintenance task in operation S201.
Then, in operation S202, a work flow sheet corresponding to the on-site operation and maintenance work task is called according to the identifier, where the work flow sheet includes at least one reference state image of the device to be maintained during the on-site operation and maintenance work task and an operation to be executed corresponding to the reference state image.
In operation S203, current spatial image information of the device to be maintained is obtained, the spatial image information is displayed, and a current state image of the device to be maintained is determined according to the spatial image information.
In operation S204, the current state image is matched with the at least one reference state image, and if a matching reference state image is matched, an operation to be executed corresponding to the matching reference state image is determined.
In operation S205, a guidance image and/or a guidance animation is obtained by combining the operation to be performed and the aerial image of the device to be maintained with display visual enhancement, and is displayed.
The method shown in fig. 2A is further described below with reference to specific embodiments.
In operation S201, an identifier of a device to be maintained, to which a field operation and maintenance task is directed, is obtained through a scanning device or a camera. Illustratively, the identifier includes a device barcode or a type of the device barcode, and the acquiring the identifier of the to-be-maintained device for the on-site operation and maintenance task includes acquiring the device barcode through a scanning device, or identifying and acquiring the device type through a device image acquired by a camera.
In operation S202, the job flow sheet corresponding to the on-site operation and maintenance job task can be called through the identifier call acquired in operation S201.
Before the operation and maintenance operation of the operator, the terminal device 101 may send a download request to the operation and maintenance server 105 in response to an operation and maintenance operation instruction from the user, and download and obtain the on-site operation and maintenance operation task and the corresponding operation flow sheet. The operation flow sheet comprises at least one reference state image of equipment to be maintained during the on-site operation and maintenance task and an operation to be executed corresponding to the reference state image.
In operation S203, the terminal device 101 acquires current spatial image information of the device to be maintained through a spatial image acquisition device, displays the spatial image information, and determines a current state image of the device to be maintained according to the spatial image information. For example, a camera of the composite LiDAR, the spatial image acquisition device acquires a real-time image of the device to be maintained through image acquisition, acquires distance measurement information through acquiring time when light touches the device to be maintained and returns, and acquires a spatial image of the device to be operated through combination of the two, including the size of the device to be operated and the distance between the terminal device 101 and the device to be operated. By integrating the real-time image and the ranging information of the device to be maintained, the terminal device 101 will determine the current state image of the device to be maintained, which is currently handled. And the current state image is an image of the equipment to be operated and maintained at a preset angle and distance acquired by displaying visual enhancement by utilizing the spatial image information.
In operation S204, the current state image of the device to be maintained is matched with at least one reference state image in the job flow sheet, and if a matching reference state image is obtained, the operation to be executed corresponding to the matching reference state image is determined according to the job flow sheet. And the matching process comprises the step of setting a matching degree threshold value, and when the matching degree of the current state image of the equipment to be maintained and one reference state image in the operation flow sheet is judged to be greater than the matching degree threshold value, the matched reference state image is considered. And if the matching degree of the current state image of the equipment to be maintained and one reference state image in the operation flow sheet is judged to be less than or equal to the threshold value of the matching degree, the equipment to be maintained is not successfully matched.
In operation S205, the terminal device 101 performs augmented reality, aligns the device with the camera of the mobile device 101, displays the operation specification interface of the current treatment step on the display device of the mobile device, and performs operation instruction and voice prompt.
FIG. 2B schematically illustrates a guidance interface obtained with display visual enhancement according to an embodiment of the disclosure. As shown in fig. 2B, the guidance image is obtained by combining the to-be-executed operation and the spatial image of the to-be-maintained device with display visual enhancement, and the to-be-executed operation of the current treatment step is visually displayed on a real-time picture of the to-be-maintained device, so that the operation and maintenance personnel can conveniently understand the operation and maintenance operation, and the operation and maintenance work efficiency is improved. IT can be understood that, in this embodiment, the to-be-operated and maintained device is an IT device, but the to-be-operated and maintained device of the present disclosure includes, but is not limited to, an IT device, and any device entity that needs to be manually operated may be used as the to-be-operated and maintained device.
The operation and maintenance operation manual and SOP based on AR technology and in the form of enhanced video display are realized, the real-time picture of the field operation equipment is shot through the camera of the mobile equipment, the operation and maintenance operation content is prompted on the current operation content picture, the field operation and maintenance operation mode can be more visually presented, the operation and maintenance operation disposal difficulty and the learning cost are reduced, the field operation and maintenance overall efficiency is improved, and the labor cost is saved; meanwhile, the operation and maintenance operation manual, the SOP and other documented knowledge precipitation contents are combined into the actual operation and maintenance disposal process through the AR technology, so that the knowledge application coverage is improved, the problem that the operation and maintenance operation manual and the actual situation are mutually split is solved, and the operation and maintenance efficiency is improved.
Fig. 3 schematically illustrates a flow chart of a method of a field operation and maintenance work based on visual enhancement according to yet another embodiment of the present disclosure.
As shown in fig. 3, in operation S301, in response to an operation and maintenance work instruction from a user, the method sends a request to a server to obtain a field operation and maintenance work task and a work flow sheet corresponding to the field operation and maintenance work task, where the work flow sheet includes at least one reference state image of a device to be maintained during the field operation and maintenance work task and an operation to be executed corresponding to the reference state image.
In operation S302, an identifier of a device to be maintained for a field operation and maintenance task is obtained.
Then, in operation S303, a work flow sheet corresponding to the on-site operation and maintenance work task is called according to the identifier.
In operation S304, current spatial image information of the device to be maintained is obtained, the spatial image information is displayed, and a current state image of the device to be maintained is determined according to the spatial image information.
In operation S305, the current state image is matched with the at least one reference state image, and if a matching reference state image is matched, an operation to be executed corresponding to the matching reference state image is determined.
In operation S306, a guidance image and/or a guidance animation is obtained by combining the operation to be performed and the aerial image of the device to be maintained with display visual enhancement, and the guidance image and/or the guidance animation is displayed.
Fig. 4A schematically illustrates a flow chart of a method of a visual enhancement based on-site operation and maintenance work according to yet another embodiment of the present disclosure.
As shown in fig. 4A, the method includes acquiring an identifier of a device to be maintained for a field operation and maintenance task in operation S401.
Then, in operation S402, a work flow sheet corresponding to the on-site operation and maintenance work task is called according to the identifier, where the work flow sheet includes at least one reference state image of the device to be maintained during the on-site operation and maintenance work task and an operation to be executed corresponding to the reference state image.
In operation S403, current space image information of the device to be maintained is obtained, the space image information is displayed, and a current state image of the device to be maintained is determined according to the space image information.
In operation S404, the current state image is matched with the at least one reference state image, and if a matching reference state image is not matched, a remote assistance request is sent to a remote server to obtain remote assistance sent by the remote server. After the expert video guidance is initiated, both sides can perform simulation operation on the current physical entity based on the space image; communication cost of understanding the meeting needs in the language communication process is eliminated; the operation and maintenance expert role can directly draw a virtual enhancement processing method and operation animation in a display picture of the terminal equipment.
Fig. 4B schematically shows a flowchart of a method for acquiring that the remote server sends remote assistance according to an embodiment of the present disclosure. As shown in fig. 4B, the acquiring that the remote server sends the remote assistance includes:
s4041, receiving a spatial image information acquisition instruction sent by a remote server, and sending the spatial image information of the equipment to be maintained to the remote server.
S4042, receiving an image annotation instruction sent by a remote server, and annotating the space image according to the image annotation instruction.
S4043, performing voice or video communication with the remote server.
For example, when the problem faced by the operation and maintenance is complex or the reference state image cannot be matched, the remote assistance function of the expert engineer on duty can be initiated. Fig. 4C schematically illustrates a schematic diagram of obtaining the remote server sending remote assistance according to an embodiment of the disclosure. As shown in fig. 4C, in the remote assistance process, the remote engineer may control the camera of the terminal device to take a picture; the method comprises the steps that a camera of a terminal device is used for shooting a picture, a remote engineer can mark the current shooting picture in real time through an in-application software tool such as a painting brush, draw an operation mode and display the operation mode to a field operation and maintenance engineer in a prompt mode of a 3D image interface; meanwhile, the expert engineer on duty communicates with the on-site operation and maintenance personnel through voice or video in the remote assistance process.
It should be noted that the remote assistance request may be initiated automatically after the terminal device fails to match the reference state image, or the terminal device may be initiated again in response to the remote assistance instruction of the user.
The method of the embodiment reduces the probability that the operation and maintenance operation cannot be executed and the operation and maintenance work efficiency is slowed due to the insufficient personal ability of the on-site operation and maintenance personnel through a remote guidance mode; meanwhile, the range of the data center covered by the service capability of the operation and maintenance expert can be enlarged, the communication cost and the traffic time cost are reduced through remote guidance, and the capability output of the operation and maintenance expert is improved to the maximum extent.
Fig. 5 schematically illustrates a block diagram of a visual enhancement based on-site operation and maintenance work device, according to an embodiment of the present disclosure.
As shown in fig. 5, the visual enhancement-based field operation and maintenance work device includes an identification acquisition module 510, a work flow sheet calling module 520, a status acquisition module 530, a matching module 540, a display module 550, and a remote assistance module 560.
The identifier obtaining module 510 is configured to obtain an identifier of a device to be maintained, to which the on-site operation and maintenance task is directed; the work flow sheet calling module 520 is configured to call a work flow sheet corresponding to the on-site operation and maintenance work task according to the identifier, where the work flow sheet includes at least one reference state image of the device to be maintained and a corresponding operation to be executed during the on-site operation and maintenance work task; the state obtaining module 530 is configured to obtain current spatial image information of the device to be maintained, display the spatial image information, and determine a current state image of the device to be maintained according to the spatial image information; the matching module 540 is configured to match the current state image with a reference state image in the job flow sheet, and if a matching reference state image is matched, determine an operation to be executed corresponding to the matching reference state image; the display module 550 acquires a guidance image and/or a guidance animation by using the display visual enhancement in combination with the operation to be performed and the spatial image of the equipment to be maintained, and displays the guidance image and/or the guidance animation; the remote assistance module 560 is used for interacting with a remote server to realize remote assistance.
Any of the modules according to embodiments of the present disclosure, or at least part of the functionality of any of them, may be implemented in one module. Any one or more of the modules according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules according to the embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging the circuit, or in any one of three implementations, or in any suitable combination of any of the software, hardware, and firmware. Alternatively, one or more of the modules according to embodiments of the disclosure may be implemented at least partly as computer program modules which, when executed, may perform corresponding functions.
For example, any of the identity acquisition module 510, the workflow single invocation module 520, the status acquisition module 530, the matching module 540, the display module 550, and the remote assistance module 560 may be combined into one module to be implemented, or any one of the modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the identification acquisition module 510, the workflow single invocation module 520, the status acquisition module 530, the matching module 540, the display module 550, and the remote assistance module 560 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or in any one of three implementations of software, hardware, and firmware, or in any suitable combination of any of them. Alternatively, at least one of the identification acquisition module 510, the workflow single invocation module 520, the status acquisition module 530, the matching module 540, the display module 550, and the remote assistance module 560 may be implemented at least in part as a computer program module that, when executed, may perform a corresponding function.
Fig. 6 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device 600 includes a processor 610, a computer-readable storage medium 620. The electronic device 600 may perform a method according to an embodiment of the present disclosure.
In particular, the processor 610 may comprise, for example, a general purpose microprocessor, an instruction set processor and/or related chip set and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 610 may also include onboard memory for caching purposes. The processor 610 may be a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.
Computer-readable storage medium 620, for example, may be a non-volatile computer-readable storage medium, specific examples including, but not limited to: magnetic storage devices, such as magnetic tape or Hard Disk Drives (HDDs); optical storage devices, such as compact disks (CD-ROMs); a memory, such as a Random Access Memory (RAM) or a flash memory; and so on.
The computer-readable storage medium 620 may include a computer program 621, which computer program 621 may include code/computer-executable instructions that, when executed by the processor 610, cause the processor 610 to perform a method according to an embodiment of the disclosure, or any variation thereof.
The computer program 621 may be configured with, for example, computer program code comprising computer program modules. For example, in an example embodiment, code in computer program 621 may include one or more program modules, including 621A, 621B, … …, for example. It should be noted that the division and number of the modules are not fixed, and those skilled in the art may use suitable program modules or program module combinations according to actual situations, so that the processor 610 may execute the method according to the embodiment of the present disclosure or any variation thereof when the program modules are executed by the processor 610.
According to an embodiment of the present disclosure, at least one of the identification acquisition module 510, the workflow single invocation module 520, the status acquisition module 530, the matching module 540, the display module 550 and the remote assistance module 560 may be implemented as a computer program module as described with reference to fig. 6, which, when executed by the processor 610, may implement the respective operations described above.
The present disclosure also provides a computer-readable storage medium, which may be included in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It will be understood by those skilled in the art that while the present disclosure has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Accordingly, the scope of the present disclosure should not be limited to the above-described embodiments, but should be defined not only by the appended claims, but also by equivalents thereof.
Claims (13)
1. A field operation and maintenance operation method based on visual enhancement comprises the following steps:
acquiring an identifier of equipment to be maintained for a field operation and maintenance task;
calling an operation flow sheet corresponding to the field operation and maintenance task according to the identification, wherein the operation flow sheet comprises at least one reference state image of equipment to be maintained during the field operation and maintenance task and an operation to be executed corresponding to the reference state image;
acquiring current space image information of the equipment to be maintained, displaying the space image information, and determining a current state image of the equipment to be maintained according to the space image information;
matching the current state image with the at least one reference state image, and if a matched reference state image is matched, determining an operation to be executed corresponding to the matched reference state image; and
and acquiring a guide image and/or a guide animation by utilizing display visual enhancement in combination with the operation to be performed and the space image of the equipment to be maintained, and displaying the guide image and/or the guide animation.
2. The on-site operation and maintenance operation method according to claim 1, wherein the acquiring current aerial image information of the device to be maintained includes:
acquiring a real-time image of a device to be maintained by image acquisition, an
And acquiring the distance measurement information by acquiring the time when the light touches the equipment to be maintained and returns.
3. The on-site operation and maintenance operation method according to claim 1, wherein the determining the current state image of the device to be maintained according to the spatial image information comprises:
and displaying the image of the equipment to be operated and maintained at the preset angle and distance obtained by visual enhancement by utilizing the spatial image information as a current state image.
4. The on-site operation and maintenance work method according to claim 3, further comprising:
and if the matched reference state image is not matched, sending a remote assistance request to a remote server, and acquiring the remote assistance sent by the remote server.
5. The on-site operation and maintenance operation method according to claim 4, wherein the acquiring the remote assistance sent by the remote server comprises:
and receiving a spatial image information acquisition instruction sent by a remote server, and sending the spatial image information of the equipment to be maintained to the remote server.
6. The on-site operation and maintenance operation method according to claim 4, wherein the acquiring the remote assistance sent by the remote server further comprises:
and receiving an image annotation instruction sent by a remote server, and annotating the space image according to the image annotation instruction.
7. The on-site operation and maintenance operation method according to claim 4, wherein the acquiring the remote assistance sent by the remote server further comprises:
and carrying out voice or video communication with the remote server.
8. The on-site operation and maintenance work method according to claim 1, wherein the obtaining of the identifier of the device to be maintained for the on-site operation and maintenance work task comprises:
acquiring the device barcode by image acquisition, or
The device type is obtained through image recognition.
9. The on-site operation and maintenance work method according to claim 1, further comprising:
and responding to an operation and maintenance operation guiding instruction from a user, sending a request to a server, and acquiring a field operation and maintenance operation task and an operation flow sheet corresponding to the field operation and maintenance operation task.
10. A visual enhancement based field operation and maintenance work device comprising:
the identification acquisition module is used for acquiring an identification of the equipment to be maintained, which is aimed at by the field operation and maintenance task;
the operation flow sheet calling module is used for calling an operation flow sheet corresponding to the field operation and maintenance operation task according to the identification, wherein the operation flow sheet comprises at least one reference state image of the equipment to be maintained and corresponding operation to be executed during the field operation and maintenance operation task;
the state acquisition module is used for acquiring the current space image information of the equipment to be maintained, displaying the space image information and determining the current state image of the equipment to be maintained according to the space image information;
the matching module is used for matching the current state image with the reference state image in the operation flow sheet, and if the matched reference state image is matched, determining the operation to be executed corresponding to the matched reference state image; and
and the display module acquires a guide image and/or a guide animation by utilizing display visual enhancement in combination with the operation to be executed and the spatial image of the equipment to be maintained, and displays the guide image and/or the guide animation.
11. The on-site operations and maintenance work apparatus of claim 10, further comprising:
and the remote assistance module is used for realizing remote assistance by interacting with the remote server.
12. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the field operation and maintenance work method of any of claims 1-9.
13. A computer readable storage medium having stored thereon executable instructions for implementing the method of on-site operations of any one of claims 1 to 9 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011167513.3A CN113780706A (en) | 2020-10-27 | 2020-10-27 | On-site operation and maintenance operation method and device based on visual enhancement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011167513.3A CN113780706A (en) | 2020-10-27 | 2020-10-27 | On-site operation and maintenance operation method and device based on visual enhancement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113780706A true CN113780706A (en) | 2021-12-10 |
Family
ID=78835154
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011167513.3A Pending CN113780706A (en) | 2020-10-27 | 2020-10-27 | On-site operation and maintenance operation method and device based on visual enhancement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113780706A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114401383A (en) * | 2021-12-22 | 2022-04-26 | 上海智能制造功能平台有限公司 | A remote expert guidance system and method |
| CN114508389A (en) * | 2021-12-29 | 2022-05-17 | 东方电气集团东方汽轮机有限公司 | Method for monitoring motion and fault of steam turbine valve |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710449A (en) * | 2012-06-14 | 2012-10-03 | 华为技术有限公司 | Maintenance method and system for network element equipment, mobile equipment and network management service equipment |
| CN104506346A (en) * | 2014-12-09 | 2015-04-08 | 北京百度网讯科技有限公司 | Method and device for equipment operation and maintenance |
| CN106339094A (en) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation maintenance system and method based on augmented reality technology |
| CN207319184U (en) * | 2017-10-23 | 2018-05-04 | 北京华锐同创系统技术有限公司 | A kind of field device maintenance maintenance and current check system |
| CN109034418A (en) * | 2018-07-26 | 2018-12-18 | 国家电网公司 | Operation field information transferring method and system |
| CN109102087A (en) * | 2018-08-02 | 2018-12-28 | 广东电网有限责任公司 | Guidance method, device and server on equipment O&M line |
| CN110191162A (en) * | 2019-05-17 | 2019-08-30 | 广东电网有限责任公司 | A kind of power distribution communication device context O&M method, server, terminal and system |
| CN110210628A (en) * | 2019-05-09 | 2019-09-06 | 广东电网有限责任公司 | A kind of automation equipment intelligence O&M upkeep operation method of augmented reality |
| CN110609913A (en) * | 2019-09-18 | 2019-12-24 | 中国工商银行股份有限公司 | Information processing method and device, electronic device and medium |
-
2020
- 2020-10-27 CN CN202011167513.3A patent/CN113780706A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710449A (en) * | 2012-06-14 | 2012-10-03 | 华为技术有限公司 | Maintenance method and system for network element equipment, mobile equipment and network management service equipment |
| CN104506346A (en) * | 2014-12-09 | 2015-04-08 | 北京百度网讯科技有限公司 | Method and device for equipment operation and maintenance |
| CN106339094A (en) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | Interactive remote expert cooperation maintenance system and method based on augmented reality technology |
| CN207319184U (en) * | 2017-10-23 | 2018-05-04 | 北京华锐同创系统技术有限公司 | A kind of field device maintenance maintenance and current check system |
| CN109034418A (en) * | 2018-07-26 | 2018-12-18 | 国家电网公司 | Operation field information transferring method and system |
| CN109102087A (en) * | 2018-08-02 | 2018-12-28 | 广东电网有限责任公司 | Guidance method, device and server on equipment O&M line |
| CN110210628A (en) * | 2019-05-09 | 2019-09-06 | 广东电网有限责任公司 | A kind of automation equipment intelligence O&M upkeep operation method of augmented reality |
| CN110191162A (en) * | 2019-05-17 | 2019-08-30 | 广东电网有限责任公司 | A kind of power distribution communication device context O&M method, server, terminal and system |
| CN110609913A (en) * | 2019-09-18 | 2019-12-24 | 中国工商银行股份有限公司 | Information processing method and device, electronic device and medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114401383A (en) * | 2021-12-22 | 2022-04-26 | 上海智能制造功能平台有限公司 | A remote expert guidance system and method |
| CN114508389A (en) * | 2021-12-29 | 2022-05-17 | 东方电气集团东方汽轮机有限公司 | Method for monitoring motion and fault of steam turbine valve |
| CN114508389B (en) * | 2021-12-29 | 2024-02-13 | 东方电气集团东方汽轮机有限公司 | Method for monitoring valve movement and faults of steam turbine |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2736008A1 (en) | Dynamic tag generating apparatus and dynamic tag generating method thereof for use in display apparatus | |
| US11805202B2 (en) | Real-time crime center solution with dispatch directed digital media payloads | |
| CN112631910A (en) | Front-end testing method and device, computer equipment and storage medium | |
| CN110059623B (en) | Method and apparatus for generating information | |
| CN113780706A (en) | On-site operation and maintenance operation method and device based on visual enhancement | |
| CN109784738B (en) | Approval method and approval device | |
| CN107817962B (en) | Remote control method, device, control server and storage medium | |
| CN103064897B (en) | Geographical entity information loading method and system based on generalized information system | |
| KR20120095325A (en) | Method and system for providing social-computing-based location information using mobile augmented reality | |
| CN114283565A (en) | Multi-screen seat machine early warning method and device, electronic equipment and storage medium | |
| CN113570083A (en) | Operation and maintenance flow determination method and device for equipment, electronic equipment and storage medium | |
| CN111372057B (en) | Information interaction method, system, device, augmented reality equipment and medium | |
| CN109819026B (en) | Method and device for transmitting information | |
| CN104142882A (en) | Test method, test device and test system based on data processing | |
| CN115756256A (en) | Information labeling method, system, electronic equipment and storage medium | |
| CN111338642B (en) | Method, device, terminal and storage medium for determining application download path | |
| CN115294283A (en) | Digital twin factory construction method, device, equipment and storage medium | |
| US20200322293A1 (en) | Information processing system and method | |
| CN114245076B (en) | Method, device, electronic device and readable medium for generating real-time dynamic video of road | |
| KR102525305B1 (en) | Method, server and system for providing integrated information management service for ship maintenance | |
| US20230124200A1 (en) | Secure material movement to prevent malware propagation | |
| CN114140089A (en) | Portable tax self-service terminal and business handling method | |
| Alyassi | Empowering Frontline Workers to Work Effectively with Hands-Free Computing Devices | |
| CN115496231A (en) | Loss assessment processing method, device and equipment | |
| KR20250023057A (en) | Instant messenger operation method and apparatus for executing instant messenger |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |