[go: up one dir, main page]

WO2019041516A1 - 屏幕状态自动检测机器人、方法及计算机可读存储介质 - Google Patents

屏幕状态自动检测机器人、方法及计算机可读存储介质 Download PDF

Info

Publication number
WO2019041516A1
WO2019041516A1 PCT/CN2017/108740 CN2017108740W WO2019041516A1 WO 2019041516 A1 WO2019041516 A1 WO 2019041516A1 CN 2017108740 W CN2017108740 W CN 2017108740W WO 2019041516 A1 WO2019041516 A1 WO 2019041516A1
Authority
WO
WIPO (PCT)
Prior art keywords
preset
display
service device
detected
image
Prior art date
Application number
PCT/CN2017/108740
Other languages
English (en)
French (fr)
Inventor
严海锐
周宝
肖京
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Priority to SG11202001417WA priority Critical patent/SG11202001417WA/en
Priority to US16/632,859 priority patent/US11361422B2/en
Priority to JP2019556812A priority patent/JP6911149B2/ja
Publication of WO2019041516A1 publication Critical patent/WO2019041516A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0259—Control of position or course in two dimensions specially adapted to land vehicles using magnetic or electromagnetic means
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/028—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using a RF signal
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0285—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using signals transmitted via a public communication network, e.g. GSM network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
    • G06K7/1417—2D bar codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30121—CRT, LCD or plasma display

Definitions

  • the present application relates to the field of terminal technologies, and in particular, to a screen state automatic detecting robot, a method, and a computer readable storage medium.
  • unmanned network areas for example, bank self-service outlets
  • the store has business equipment for different businesses, such as deposit and withdrawal machines, ATMs, self-service payment machines, and PCs for customers (for example, PCs designed for large customers, you can check financial management, funds , stock information, etc.), self-checking equipment, numbering machine, promotional equipment, etc., these business equipment for different services usually need to configure the terminal screen.
  • the terminal screen is a delicate electronic device, and display problems often occur. Once the display problem affects the customer's use, the industry usually needs to periodically check and maintain the status of the screen of each self-service device in the unmanned network area.
  • the scheme for detecting the screen state of the self-service device usually includes: scheme 1, feedback through the manual inspection, and scheme 2, and feedback through the customer complaint.
  • the drawback of the above scheme 1 is that the manual inspection method is labor-intensive, the timeliness is relatively poor, and some details may not be carefully checked. For example, the small crack artificial is not necessarily carefully observed, and the error is likely to occur.
  • the shortcoming of the above solution 2 is that it will affect the user experience, and many users may not give feedback. Even if the user gives feedback, the problem usually is not comprehensive and objective, not only the timeliness is bad, but also the error will occur. Therefore, how to automatically and accurately detect the screen state of each self-service device in the unmanned network area has become a technical problem to be solved urgently.
  • the present application provides a screen state automatic detecting robot, a method, and a computer readable storage medium, the main purpose of which is to automatically detect the screen state of each self-service device in the unmanned network point area without human power.
  • the present application provides a screen state automatic detecting robot, which includes a memory and a processor, and the memory stores a screen state automatic detecting program, when the screen state automatic detecting program is executed by the processor Implement the following operations:
  • the service device corresponding to the preset area is used as a service device to be detected, and a preset graphic code is sent to the service device to be detected, according to the preset graphic code. Determining, by the content displayed on the display screen, whether the circuit to be detected has a circuit failure;
  • the device to be detected is controlled to perform image display according to the preset display parameter, and the image displayed on the display screen is analyzed to analyze the service to be detected. Whether the preset type of exception has occurred on the display screen of the device.
  • the present application further provides a method for automatically detecting a screen state, the method comprising:
  • the service device corresponding to the preset area is regarded as a to-be-tested industry. And sending a preset graphic code to the service device to be detected, to determine whether a circuit fault occurs in the service device to be detected according to the content displayed on the display screen by the preset graphic code;
  • the device to be detected is controlled to perform image display according to the preset display parameter, and the image displayed on the display screen is analyzed to analyze the service to be detected. Whether the preset type of exception has occurred on the display screen of the device.
  • the present application further provides a computer readable storage medium having a screen state automatic detection program stored thereon, the screen state automatic detection program being configurable by one or more processors Execute to implement the following steps:
  • the service device corresponding to the preset area is used as a service device to be detected, and a preset graphic code is sent to the service device to be detected, according to the preset graphic code. Determining, by the content displayed on the display screen, whether the circuit to be detected has a circuit failure;
  • the device to be detected is controlled to perform image display according to the preset display parameter, and the image displayed on the display screen is analyzed to analyze the service to be detected. Whether the preset type of exception has occurred on the display screen of the device.
  • the screen state automatic detecting robot, the method and the computer readable storage medium proposed by the application respectively moves to a preset area of each service device of the unmanned network point area, and if the robot moves to a preset area of a special service device, And sending a preset graphic code to the service device to be detected, to determine whether a circuit fault occurs in the service device to be detected according to the content displayed on the display screen by the preset graphic code, and if the circuit fault does not occur, controlling the service device to be detected
  • the graphic display is performed according to the preset display parameter, and the image displayed on the display screen is analyzed according to the preset analysis rule to analyze whether the display screen of the service device to be detected has a preset type abnormality, and the solution does not need manual Participate in the automatic detection of the circuit fault of the device and the screen display state by moving the robot to the corresponding area.
  • FIG. 1 is a schematic diagram of a preferred embodiment of a screen state automatic detecting robot of the present application
  • 2a to 2d are two-dimensional codes in which partial information is occluded
  • 3a is a two-dimensional code in which perspective is generated;
  • FIG. 3b is an outer contour model of the two-dimensional code after seeing;
  • FIG. 3c is an effect of reversely seeing the two-dimensional code after see-through;
  • FIG. 4 is a schematic view of display areas of different sizes set on a display screen
  • Figure 5 is a boundary of a small-sized rectangular area and a medium-sized rectangular display area
  • Figure 6 is an outer contour curve of a large-sized rectangular area
  • Figure 7 is a schematic illustration of a plurality of points taken from a rectangular boundary
  • FIG. 8 is a schematic diagram of a straight line obtained by a least squares straight line fitting algorithm for acquired points
  • Figure 9 is a screen state before performing noise filtering enhancement preprocessing
  • Figure 10 is a screen state after performing noise filtering enhancement preprocessing
  • Figure 11 is a schematic view showing a state in which a bad line appears on the screen
  • Figure 12 is a schematic view showing a state in which a bad spot appears on the screen
  • Figure 13 is a schematic view showing a state in which a crack occurs on the screen
  • Figure 14 is a schematic view showing a state of a lateral crack appearing on a screen
  • 15 is a schematic diagram of a program module of an automatic screen state detecting program in an embodiment of a screen state automatic detecting robot of the present application;
  • 16 is a flowchart of a preferred embodiment of a screen state automatic detecting method according to the present application.
  • FIG. 17 is a flowchart of a second embodiment of a method for automatically detecting a screen state according to the present application.
  • the application provides a screen state automatic detecting robot.
  • FIG. 1 a schematic diagram of a preferred embodiment of a screen state automatic detecting robot of the present application is shown.
  • the screen state automatic detecting robot includes a memory 11, a processor 12, a communication bus 13, a network interface 14, and a camera 15.
  • the memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like.
  • the memory 11 may be an internal storage unit of the screen state automatic detection robot in some embodiments, for example, the screen state automatically detects the hard disk of the robot.
  • the memory 11 may also be an external storage device of the screen state automatic detection robot in other embodiments, such as a plug-in hard disk equipped on the screen state automatic detection robot, a smart memory card (SMC), and a secure digital (Secure) Digital, SD) cards, flash cards, etc.
  • the memory 11 may also include both an internal storage unit of the screen state automatic detection robot and an external storage device.
  • the memory 11 can be used not only for storing application software and various types of data installed in the screen state automatic detection robot, such as code of the screen state automatic detection program, but also for temporarily storing data that has been output or is to be output.
  • the processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data processing chip for running program code or processing stored in the memory 11. Data, such as performing a screen state automatic detection program, and the like.
  • CPU Central Processing Unit
  • controller microcontroller
  • microprocessor or other data processing chip for running program code or processing stored in the memory 11.
  • Data such as performing a screen state automatic detection program, and the like.
  • Communication bus 13 is used to implement connection communication between these components.
  • the network interface 14 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and is typically used to establish a communication connection between the robot and other electronic devices.
  • the robot can connect to the user terminal through the network interface 14, receive the detection command sent by the user terminal, or connect to the service device through the network interface 14, and control the service device to display the image according to the preset display parameter.
  • the camera 15 is used to collect the display content of the display screen of the business device.
  • Figure 1 shows only the screen state automatic detection robot with components 11-15 and the screen state automatic detection program, but it should be understood that not all illustrated components are required to be implemented, and alternative implementations may be implemented more or less. Component.
  • the robot may further include a user interface
  • the user interface may include a display, an input unit such as a keyboard, a physical button for triggering the detection instruction, and the like
  • the optional user interface may further include a standard wired Interface, wireless interface.
  • the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch sensor, or the like.
  • the display may also be referred to as a display screen or a display unit as appropriate.
  • the robot may further include an RF (Radio Frequency) circuit, a sensor, a WiFi module, and the like.
  • the sensor may be a light sensor, a distance sensor, or the like.
  • a screen state automatic detection program is stored in the memory 11; when the processor 12 executes the program, the following operations are implemented:
  • the robot provided in this embodiment may be placed in an unmanned network point area, and the service equipment in the unmanned network point area may have one or more.
  • the robot can be moved to the unmanned network point area in real time, timing, in an idle state or upon receiving a detection command, and the detection of the screen display state of each service device in the unmanned network point area is turned on. Wherein, when the robot is in the process of no need to process the service for a preset period of time, it is determined that it is in an idle state.
  • Each service device in the unmanned network site area is divided into preset areas, and the preset area of one service device refers to an area where the distance from the service machine is less than or equal to a preset distance, and the robot passes during the moving process.
  • Positioning its own position determines the position coordinates in the unmanned network point area, and combines the known position coordinates of each service device in the unmanned network point area to calculate the current position and the predetermined position of each service device.
  • Distance if current The distance between the location and the location where the service device is located is less than or equal to the preset distance, and then the robot is determined to move to the preset region of the service device.
  • each service device needs to be detected one by one.
  • the control robot moves to the preset area of each service device in the unmanned network point area, and two of them are described below.
  • the movement of the robot is controlled according to the preset mobile navigation path.
  • the control robot controls the mobile navigation according to the movement. The path continues to move to the undetected business device until the display screen of all business devices is detected and the mobile is finished.
  • the robot may be controlled to move randomly. After moving to a preset area of a service device and detecting the display screen of the service device to be detected, the control robot marks the service device to be detected as an obstacle. The object, the obstacle avoidance movement, after the obstacle avoidance movement is completed, the control robot continues to perform random movement, moves to another business device for detection, until all the business equipment is marked as an obstacle and ends the movement, and the business equipment The obstacle mark is cleared.
  • the principle of the obstacle avoiding movement algorithm is: when the robot detects a business device, shifts the preset angle to the left or right in the current moving direction, and judges Whether it can move unimpeded after the offset angle, if it can pass after the offset angle, continue to move randomly along the direction after the offset, if it is blocked after the offset angle, it will continue to shift in the same direction.
  • the algorithm that constantly adjusts the offset angle circumvents obstacles that hinder movement.
  • the service device corresponding to the preset area is used as a service device to be detected, and a preset graphic code is sent to the service device to be detected, according to the preset.
  • the content displayed on the display screen by the graphic code determines whether a circuit failure has occurred in the service device to be detected.
  • the service device corresponding to the preset area is used as a service device to be detected, and the service device corresponding to the preset area is detected.
  • the robot can adjust the angle of the camera 15 until the display screen of the current display screen of the service device can be captured, and then send a preset graphic code to the service device to be detected to display the content on the display screen according to the preset graphic code. Determining whether a fault occurs in the service device to be detected.
  • the control robot performs wireless communication with the service device to be detected (ie, the service device corresponding to the preset area currently described), and sends a display instruction for displaying the preset graphic code including the preset information to the service device to be detected.
  • the service device displays the preset graphic code according to the received display instruction; the control robot scans and analyzes the display content of the display screen of the service device to be detected; if the preset information is scanned and analyzed from the display screen of the service device to be detected, It is determined that the display screen of the service device to be detected does not have a circuit fault; if the preset information is not scanned and analyzed from the display screen of the service device to be detected, it is determined that a circuit fault occurs on the display screen of the service device to be detected.
  • the preset graphic code may be a two-dimensional code with preset information.
  • the use of two-dimensional code for circuit fault detection has the following advantages: the two-dimensional code has strong fault tolerance and has error correction capability, as shown in Figures 2a to 2d, even if there are stains, spots, cracks, etc. on the display screen. Some information is blocked, and the QR code can still be recognized. It does not affect whether the power can be powered by the QR code, that is, whether the circuit fault occurs.
  • the QR code can contain information, such as the ID of the screen.
  • the service device corresponding to the preset area can be identified according to the screen ID included in the two-dimensional code, because the outer contour of the two-dimensional code is square,
  • the robot is not required to be strictly identifiable to the target screen.
  • the robot can easily restore the distortion caused by the perspective, as shown in Figures 3a to 3c.
  • the position of the robot may cause a certain degree of perspective, which may cause the outer contour of the two-dimensional code to be displayed in the lens of the robot. Rectangle. It can be seen from Fig. 3b that the two-dimensional code does not show a square, but a quadrilateral, resulting in a "nearly large and small" perspective. However, the two-dimensional code recognition can perform reverse fluoroscopy in the case of generating perspective so that the image can be recognized normally.
  • the QR code and the reverse perspective method are existing mature methods and will not be explained in detail here. Referring to Fig. 3c, the effect of the reverse perspective of the two-dimensional code is shown.
  • a prompt message is sent to the predetermined monitoring device, indicating that a circuit fault occurs on the display screen of the service device to be detected needs to be processed.
  • the prompt information is sent to the background server, where the prompt information includes the unique ID of the service device in which the circuit is faulty, and the network identifier of the unmanned network area where the service device to be detected is located.
  • the prompt information format may be "The unattended dot area of the dot is marked as *****, and the display screen of the service device whose ID number is ***** has a circuit failure that needs to be processed.”
  • control the service device to be detected to perform image display according to preset display parameters, and analyze an image displayed on the display screen to analyze the to-be-processed Check whether the display screen of the service device has a preset type of abnormality.
  • S3 includes the following specific operations:
  • the display screens of the service device to be detected are respectively displayed in a solid color display manner, for example, three preset colors of red, green, and blue, wherein different preset colors correspond to display areas of different sizes.
  • the shape of the display area corresponding to each preset color corresponds to the shape of the maximum display area of the display screen, and the display area corresponding to one of the preset colors is the maximum display area of the display screen.
  • S34 Determine, according to the acquired anti-interference image, a maximum display area of the display screen, and perform image extraction of the actual display area for the mask image corresponding to the preset color of the maximum display area according to the determined maximum display area. Extract the maximum display image, because if there is interference area in the perimeter of the screen, for example, there is “water inlet” around the screen, the screen boundary area cannot be displayed normally, then the maximum map obtained by taking the mask is incomplete, which is equivalent. The boundary interference problem is excluded from the detection, so the maximum display area needs to be obtained in the above manner.
  • the screen width of the small-sized rectangle is L1
  • the screen width of the medium-size rectangle is L2
  • the screen width of the large-size rectangle is L3.
  • the proportional relationship between the length and width of the screen is also known data.
  • the maximum display area of the display screen is determined according to the anti-interference image of the preset color corresponding to the display area of the smaller size, and the preset color corresponding to the maximum display area is determined according to the determined maximum display area.
  • the mask image is used to extract the image of the actual display area, and the principle of extracting the maximum display image is as follows:
  • the width of the large-sized rectangle is denoted as L" 3 .
  • the average value of the two values is preferably taken as the width of the large-size rectangle according to Formula 1:
  • the area range of the large-sized rectangle can be obtained, and the area range of the large-size rectangle is the maximum display area of the screen, that is, the outer contour curve of the largest-size rectangular area shown in FIG. 6.
  • the quadrilateral is inversely transformed into a rectangle, and the anti-interference image is acquired according to the position of the mask image rectangle.
  • the boundary of the calculated small-sized rectangle and the boundary of the medium-sized rectangle are respectively calculated.
  • an extension line of the boundary line can be obtained, and the extension lines of the four boundary lines are respectively obtained by the above method, and then the intersection points of the four boundary lines are determined, according to the intersection of the four boundary lines and the The four boundary lines determine the position of the quadrilateral of the mask image, that is, the range of the medium size rectangle or the small size rectangle, thereby determining the exact values of L1 and L2.
  • the principle of filtering and enhancing the maximum display image is as follows:
  • the image of the abnormal state of the screen is affected by non-uniform illumination, irregular gaps on the screen surface and equipment, so that the acquired image has Noise, so you need to remove these noises, while maintaining the details of the abnormal state of the screen, and as much as possible to improve the contrast between the crack and the background, which is conducive to later segmentation and recognition. It can be seen from the pre-processing of FIG. 9 and the pre-processed FIG. 10 that the noise-improved graph has less noise interference and the problem is more clearly visible.
  • the present application preprocesses an image using a method of a pilot filter.
  • the algorithm steps using the pilot filter are as follows: the grayscaled screen image is used as the input image P, and the grayscale screen image is also used as the guide image I, and the image filtered by the noise filter is used as q. Equation 2 enhances the input image P.
  • m is the enhancement factor, which can be determined according to the actual situation.
  • the filter here can be selected according to the actual situation.
  • the ⁇ of the following formula is a satisfaction weight, and the enchanted satisfaction is evaluated according to the P_enhanced image effect, and the highest satisfaction of the enhanced is set to a fixed value, for example, 500.
  • T n The nth iteration is denoted as T n
  • T n+1 the n+1th time can be recorded as T n+1
  • T n+1 the value of m after n iterations is Then there are:
  • the iterative process is as follows: First we give an initial weight m as Then, substituting into formula 2 to solve the image after P_enhanced, and then evaluating the satisfaction of P_enhanced according to formula 3, and then digitizing the obtained satisfaction weight ⁇ , and then using the alphaned value and formula 5 to m weight The value is updated. Then repeat the above steps to update m. After several rounds of iteration, you can get an ideal m weight. Because each time the iteration will modify the weight m, the ⁇ value will decrease when the effect of the image P_enhanced is poor, and the final m weight will also decrease, and the weight will increase accordingly.
  • Each adjustment will drive the image P_enhanced effect to a better direction, set the number of iterations to a fixed value (here set to 10), and it will have a convergence state after n iterations, that is, the image P_enhanced effect is close to the most Good state.
  • the enhanced image P_enhanced is used as the input image P, and the grayscale screen image is used as the guide image I, and the filtered image is obtained as the final enhanced and smoothed image.
  • S36 analyzing the enhanced image to analyze whether a preset type of abnormality occurs in a display screen of the service device to be detected, where the abnormality of the preset type includes a bad spot, a bad line, and a crack.
  • the identification process of the bad line is as follows: the image preprocessed by the above steps is acquired, and the straight line is detected by using the Hough transform. When the number of bad lines is detected to be greater than 1, it is determined that there is a bad line; otherwise, the judgment does not exist. Bad line.
  • the Hough transform is a feature extraction technique.
  • the classic Hough transform recognizes the lines in the image and later develops to recognize arbitrary shapes, but the more common identifying shapes are circles and ellipses, which we use to identify the state of the screen loop.
  • a straight line is defined as:
  • ⁇ 0 is used to represent the algebraic distance of the origin to the line, and ⁇ 0 is the angle between the orthogonal line of the line and the x-axis, then:
  • ( ⁇ , ⁇ ) is a representation of polar coordinates. However, if ( ⁇ , ⁇ ) is also expressed in the form of Cartesian coordinates, that is, ⁇ and ⁇ are orthogonally processed, then ( ⁇ , ⁇ ) is called a Hough space.
  • a point in the Cartesian coordinate system corresponds to a sinusoid of the Hough space.
  • a straight line is composed of an infinite number of points. In the Hough space, there are an infinite number of sinusoids, but these sinusoids intersect at a point ( ⁇ 0 , ⁇ 0 ), and the point is brought into Equation 7 and Equation 8 to obtain the slope of the line. And the intercept, so a straight line is determined. Therefore, when the line is identified by the Hough transform, the maximum value in the Hough space may correspond to a straight line.
  • the conventional method of checking the bad line is by manual observation.
  • the robot automatically detects whether there is a bad line in the preprocessed image by the above method.
  • Performing a bad line detection analysis on the enhanced image to determine whether the display screen of the service device to be detected occurs The bad line. Referring to Fig. 12, there is a case where a bad spot appears on the screen.
  • the process of determining the bad spot is to obtain the pre-processed picture, and use the SimpleBlobDetector algorithm to detect the presence of spots, when the number of detected spots is greater than one. , to determine the presence of bad spots on the screen, otherwise it is judged that there is no bad spot on the screen.
  • the embodiment also introduces a method for selecting parameters in the SimpleBlobDetector algorithm.
  • Table 1 solves the value of feature k
  • the meaning of the above formula is that when the feature value is k, the probability that the judgment is that the spot is correct is as high as possible, and the lower the probability that the judgment is the spot but not actually the spot, the larger the weight wk.
  • the parameter that needs to be determined is selected as the feature point of the SimpleBlobDetector algorithm with the largest wk value of each feature parameter. Therefore, it is not necessary to artificially determine the specific value of this parameter.
  • the conventional method of inspecting bad spots is by manual observation.
  • the robot automatically detects whether there is a bad spot on the preprocessed image by the above method.
  • the screen is tested for cracks as follows:
  • the crack is identified by the segmented image: after the segmentation, some connected regions can be seen, where the connected region refers to the image region formed after the crack is segmented, and when the number of connected regions is greater than 1, the image can be judged. There are cracks, and the cracks may be mesh or a single crack across the screen. If there is only one connected region, we obtain the X-axis projection distance H of the crack and the projection distance R from the y-axis, as shown in Fig. 14. If H ⁇ threshold D and R ⁇ threshold D, there is no crack in the screen.
  • the threshold D is a distance value, and when H and R are simultaneously smaller than this value, the line can be considered as a crack, but a noise point. If any of H and R is greater than this threshold, it can be judged that there is a lateral or longitudinal crack on the screen, but the crack does not span the entire screen, as shown in Fig. 14 is a form of lateral crack.
  • the conventional method of inspecting cracks is by manual observation, and by the method of the embodiment, the robot passes the above method.
  • the preprocessed image automatically detects the presence of cracks.
  • control robots are respectively moved to the preset areas of the service devices of the unmanned network point area. If the robot moves to the preset area of the service device, the preset graphic code is sent to the service device to be detected. Determining whether a circuit fault occurs in the service device to be detected according to the content displayed on the display screen by the preset graphic code. If no circuit failure occurs, controlling the service device to be detected to perform graphic display according to the preset display parameter, and displaying on the display screen The image is analyzed according to a preset analysis rule to analyze whether a preset type of abnormality has occurred on the display screen of the service device to be detected. The scheme does not require manual participation, and the circuit fault and the screen display state of the device are moved to the corresponding area by the robot. Perform automatic detection.
  • the screen state automatic detection program may also be divided into one or more modules, one or more modules are stored in the memory 11 and executed by one or more processors (this implementation)
  • the processor 12 is executed to complete the application
  • a module referred to herein refers to a series of computer program instructions that are capable of performing a particular function.
  • FIG. 15 it is a schematic diagram of a program module for automatically detecting a screen state in an embodiment of a screen state automatic detecting robot of the present application.
  • the screen state automatic detecting program may be divided into a control moving module 10 and a first
  • the detection module 20 and the second detection module 30, the functions or operational steps performed by the modules 10-30 are substantially the same as those of the above embodiments, and will not be described in detail herein.
  • the control mobile module 10 is configured to control the robot to move to a preset area of each service device of the unmanned network point area, respectively;
  • the first detecting module 20 is configured to: if the mobile device moves to a preset area, the service device corresponding to the preset area is used as a service device to be detected, and then send a preset graphic code to the to-be-detected service device, Determining whether a circuit fault occurs in the service device to be detected according to the content displayed on the display screen by the preset graphic code;
  • the second detecting module 30 is configured to: if the circuit of the service device to be detected does not have a circuit fault, control the service device to be detected to perform image display according to the preset display parameter, and analyze the image displayed on the display screen, It is analyzed whether a preset type of abnormality has occurred on the display screen of the service device to be detected.
  • the present application also provides a method for automatically detecting a screen state.
  • a flowchart of a preferred embodiment of the screen state automatic detecting method of the present application is shown. The method can be performed by a device that can be implemented by software and/or hardware.
  • the screen state automatic detecting method includes:
  • Step S10 controlling the robot to move to a preset area of each service device in the unmanned network point area, respectively;
  • the robot in this embodiment can be placed in the unmanned network point area, and the service equipment in the unmanned network point area can have one or more.
  • the robot can be moved to the unmanned network point area in real time, timing, in an idle state or upon receiving a detection command, and the detection of the screen display state of each service device in the unmanned network point area is turned on. Wherein, when the robot is in the process of no need to process the service for a preset period of time, it is determined that it is in an idle state.
  • Each service device in the unmanned network site area is divided into preset areas, and the preset area of one service device refers to an area where the distance from the service machine is less than or equal to a preset distance, and the robot passes during the moving process.
  • Positioning its own position determines the position coordinates in the unmanned network point area, and combines the known position coordinates of each service device in the unmanned network point area to calculate the current position and the predetermined position of each service device. The distance, if the distance between the current location and the location of a service device is less than or equal to the preset distance, determines that the robot moves to the preset area of the service device.
  • each service device needs to be detected one by one.
  • the control robot moves to the preset area of each service device in the unmanned network point area, and two of them are described below.
  • the movement of the robot is controlled according to the preset mobile navigation path.
  • the control robot controls the mobile navigation according to the movement. The path continues to move to the undetected business device until the display screen of all business devices is detected and the mobile is finished.
  • the robot may be controlled to move randomly. After moving to a preset area of a service device and detecting the display screen of the service device to be detected, the control robot marks the service device to be detected as an obstacle. The object, the obstacle avoidance movement, after the obstacle avoidance movement is completed, the control robot continues to perform random movement, moves to another business device for detection, until all the business equipment is marked as an obstacle and ends the movement, and the business equipment The obstacle mark is cleared.
  • the principle of the obstacle avoiding movement algorithm is: when the robot detects a business device, shifts the preset angle to the left or right in the current moving direction, and judges Whether it can move unimpeded after the offset angle, if it can pass after the offset angle, continue to move randomly along the direction after the offset, if it is blocked after the offset angle, it will continue to shift in the same direction.
  • the algorithm that constantly adjusts the offset angle circumvents obstacles that hinder movement.
  • step S20 if the robot moves to a preset area, the service device corresponding to the preset area is used as a service device to be detected, and a preset graphic code is sent to the service device to be detected, according to the The content displayed on the display screen by the graphic code determines whether a circuit fault has occurred in the service device to be detected.
  • the service device corresponding to the preset area is used as a service device to be detected, and the service device corresponding to the preset area is detected.
  • the robot can adjust the angle of the camera 15 until the display screen of the current display screen of the service device can be captured, and then send a preset graphic code to the service device to be detected to display the content on the display screen according to the preset graphic code. Determining whether a fault occurs in the service device to be detected.
  • the control robot performs wireless communication with the service device to be detected (ie, the service device corresponding to the preset area currently described), and sends a display instruction for displaying the preset graphic code including the preset information to the service device to be detected.
  • the service device displays the preset graphic code according to the received display instruction; the control robot scans and analyzes the display content of the display screen of the service device to be detected; if the preset information is scanned and analyzed from the display screen of the service device to be detected, It is determined that the display screen of the service device to be detected does not have a circuit fault; if the preset information is not scanned and analyzed from the display screen of the service device to be detected, it is determined that a circuit fault occurs on the display screen of the service device to be detected.
  • the preset graphic code may be a two-dimensional code with preset information.
  • the use of two-dimensional code for circuit fault detection has the following advantages: the two-dimensional code has strong fault tolerance and has error correction capability, as shown in Figures 2a to 2d, even if there are stains, spots, cracks, etc. on the display screen. Some information is blocked, and the QR code can still be recognized. It does not affect whether the power can be powered by the QR code, that is, whether the circuit fault occurs.
  • the QR code can contain information, such as the ID of the screen.
  • the service device corresponding to the preset area can be identified according to the screen ID included in the two-dimensional code; since the outer contour of the two-dimensional code is square, Therefore, it is not required that the robot can be recognized strictly against the target screen.
  • the robot can easily restore the distortion caused by the perspective, as shown in Figs. 3a to 3c.
  • the position of the robot may cause a certain degree of perspective, which may cause the outer contour of the two-dimensional code to be displayed in the lens of the robot. Rectangle. It can be seen from Fig. 3b that the two-dimensional code does not show a square, but a quadrilateral, resulting in a "nearly large and small" perspective.
  • the two-dimensional code recognition can perform reverse fluoroscopy in the case of generating perspective so that the two-dimensional code can be recognized normally.
  • the reverse fluoroscopy method is an existing mature method and will not be explained in detail here. Referring to Fig. 3c, the effect of the reverse perspective of the two-dimensional code is shown.
  • a prompt message is sent to the predetermined monitoring device, indicating that a circuit fault occurs on the display screen of the service device to be detected needs to be processed.
  • the prompt information is sent to the background server, where the prompt information includes the unique ID of the service device in which the circuit is faulty, and the network identifier of the unmanned network area where the service device to be detected is located.
  • the prompt information format may be "The unattended dot area of the dot is marked as *****, and the display screen of the service device whose ID number is ***** has a circuit failure that needs to be processed.”
  • Step S30 If the display screen of the service device to be detected does not have a circuit failure, control the to-be-detected industry.
  • the device displays the image according to the preset display parameter, and analyzes the image displayed on the display screen to analyze whether a preset type of abnormality has occurred on the display screen of the service device to be detected.
  • step S30 in the preferred embodiment of the screen state automatic detecting method of the present application, and step S30 includes:
  • Step S301 the display screen of the service device to be detected is controlled to display a plurality of preset colors in a solid color display manner, for example, three preset colors of red, green, and blue, wherein different preset colors correspond to different size displays.
  • the area, the shape of the display area corresponding to each preset color corresponds to the shape of the maximum display area of the display screen, and the display area corresponding to one of the preset colors is the maximum display area of the display screen.
  • step S302 when a preset color is displayed in a solid color display manner, a mask is displayed on the display graphic of the display screen for the preset color, that is, the other colors of the display screen except the preset color are masked to obtain a mask.
  • Code image when a preset color is displayed in a solid color display manner, a mask is displayed on the display graphic of the display screen for the preset color, that is, the other colors of the display screen except the preset color are masked to obtain a mask.
  • Step S303 performing anti-interference processing on the mask image corresponding to the preset color of the display area of the smaller size to obtain an anti-interference image, wherein the display area of the smaller size is other sizes than the maximum display area. Display area.
  • Step S304 determining a maximum display area of the display screen according to the acquired anti-interference image, and performing image extraction of the actual display area for the mask image of the preset color corresponding to the maximum display area according to the determined maximum display area.
  • extract the maximum display image because if there is interference area in the perimeter of the screen, for example, there is “water inlet” around the screen, the screen boundary area cannot be displayed normally, then the maximum map obtained by taking the mask is incomplete, quite In order to exclude the boundary interference problem from the detection, it is necessary to obtain the maximum display area by the above method.
  • the screen width of the small-sized rectangle is L1
  • the screen width of the medium-size rectangle is L2
  • the screen width of the large-size rectangle is L3.
  • the proportional relationship between the length and width of the screen is also known data.
  • the maximum display area of the display screen is determined according to the anti-interference image of the preset color corresponding to the display area of the smaller size, and the preset color corresponding to the maximum display area is determined according to the determined maximum display area.
  • the mask image is used to extract the image of the actual display area, and the principle of extracting the maximum display image is as follows:
  • the width of the large-sized rectangle is denoted as L" 3 .
  • the average value of the two values is preferably taken as the width of the large-size rectangle according to Formula 1:
  • the area range of the large-sized rectangle can be obtained, and the area range of the large-size rectangle is the maximum display area of the screen, that is, the outer contour curve of the largest-size rectangular area shown in FIG. 6.
  • the quadrilateral is inversely transformed into a rectangle, and the anti-interference image is acquired according to the position of the mask image rectangle.
  • the boundary of the calculated small-sized rectangle and the boundary of the medium-sized rectangle are respectively calculated.
  • an extension line of the boundary line can be obtained, and the extension lines of the four boundary lines are respectively obtained by the above method, and then the intersection points of the four boundary lines are determined, according to the intersection of the four boundary lines and the The four boundary lines determine the position of the quadrilateral of the mask image, that is, the range of the medium size rectangle or the small size rectangle, thereby determining the exact values of L1 and L2.
  • Step S305 performing noise filtering enhancement preprocessing on the maximum display image to obtain an enhanced image.
  • the collected abnormal image of the screen is affected by non-uniform illumination, irregular gaps on the screen surface, equipment, etc., so that the acquired image is noisy, it is necessary to remove the noise while maintaining the details of the abnormal state of the screen, and As much as possible to improve the contrast between the crack and the background, which is conducive to the later segmentation and recognition. It can be seen from the pre-processing of FIG. 9 and the pre-processed FIG. 10 that the noise-improved graph has less noise interference and the problem is more clearly visible.
  • the present application preprocesses an image using a method of a pilot filter.
  • the algorithm steps using the bootstrap filter are as follows:
  • the grayscale image of the screen is used as the input image P.
  • the grayscale screen image is used as the guide image I
  • the image filtered by the noise filter is q
  • the input image P is enhanced by Equation 2.
  • m is the enhancement factor, which can be determined according to the actual situation.
  • the filter here can be selected according to the actual situation.
  • the ⁇ of the following formula is a satisfaction weight, and the enchanted satisfaction is evaluated according to the P_enhanced image effect, and the highest satisfaction of the enhanced is set to a fixed value, for example, 500.
  • T n The nth iteration is denoted as T n
  • T n+1 the n+1th time can be recorded as T n+1
  • T n+1 the value of m after n iterations is Then there are:
  • the iterative process is as follows: First we give an initial weight m as Then, substituting into formula 2 to solve the image after P_enhanced, and then evaluating the satisfaction of P_enhanced according to formula 3, and then digitizing the obtained satisfaction weight ⁇ , and then using the alphaned value and formula 5 to m weight The value is updated. Then repeat the above steps to update m. After several rounds of iteration, you can get an ideal m weight. Because each time the iteration will modify the weight m, the ⁇ value will decrease when the effect of the image P_enhanced is poor, and the final m weight will also decrease, and the weight will increase accordingly.
  • Each adjustment will drive the image P_enhanced effect to a better direction, set the number of iterations to a fixed value (here set to 10), and it will have a convergence state after n iterations, that is, the image P_enhanced effect is close to the most Good state.
  • the enhanced image P_enhanced is used as the input image P, and the grayscale screen image is used as the guide image I, and the filtered image is obtained as the final enhanced and smoothed image.
  • step S306 the enhanced image is analyzed to analyze whether a preset type of abnormality has occurred in the display screen of the service device to be detected, and the abnormality of the preset type includes bad spots, bad lines and cracks.
  • the identification process of the bad line is as follows: the image preprocessed by the above steps is acquired, and the straight line is detected by using the Hough transform. When the number of bad lines is detected to be greater than 1, it is determined that there is a bad line; otherwise, the judgment does not exist. Bad line.
  • the Hough transform is a feature extraction technique.
  • the classic Hough transform recognizes the lines in the image and later develops to recognize arbitrary shapes, but the more common identifying shapes are circles and ellipses, which we use to identify the state of the screen loop.
  • a straight line is defined as:
  • ⁇ 0 is used to represent the algebraic distance of the origin to the line, and ⁇ 0 is the angle between the orthogonal line of the line and the x-axis, then:
  • ( ⁇ , ⁇ ) is a representation of polar coordinates. However, if ( ⁇ , ⁇ ) is also expressed in the form of Cartesian coordinates, that is, ⁇ and ⁇ are orthogonally processed, then ( ⁇ , ⁇ ) is called a Hough space.
  • a point in the Cartesian coordinate system corresponds to a sinusoid of the Hough space.
  • a straight line is composed of an infinite number of points. In the Hough space, there are an infinite number of sinusoids, but these sinusoids intersect at a point ( ⁇ 0 , ⁇ 0 ), and the point is brought into Equation 7 and Equation 8 to obtain the slope of the line. And the intercept, so a straight line is determined. Therefore, when the line is identified by the Hough transform, the maximum value in the Hough space may correspond to a straight line.
  • the conventional method of checking the bad line is by manual observation.
  • the robot automatically detects whether there is a bad line in the preprocessed image by the above method.
  • the process of determining the bad spot is to obtain the pre-processed picture, and use the SimpleBlobDetector algorithm to detect the presence of spots, when the number of detected spots is greater than one. , to determine the presence of bad spots on the screen, otherwise it is judged that there is no bad spot on the screen.
  • the embodiment also introduces a A method for selecting parameters in the SimpleBlobDetector algorithm.
  • the meaning of the above formula is that when the feature value is k, the probability that the judgment is that the spot is correct is as high as possible, and the lower the probability that the judgment is the spot but not actually the spot, the larger the weight wk.
  • the parameter that needs to be determined is selected as the feature point of the SimpleBlobDetector algorithm with the largest wk value of each feature parameter. Therefore, it is not necessary to artificially determine the specific value of this parameter.
  • the conventional method of inspecting bad spots is by manual observation.
  • the robot automatically detects whether there is a bad spot on the preprocessed image by the above method.
  • the screen is tested for cracks as follows:
  • the crack is identified by the segmented image: after the segmentation, some connected regions can be seen, where the connected region refers to the image region formed after the crack is segmented, and when the number of connected regions is greater than 1, the image can be judged. There are cracks, and the cracks may be mesh or a single crack across the screen. If there is only one connected region, we obtain the X-axis projection distance H of the crack and the projection distance R from the y-axis, as shown in Fig. 14. If H ⁇ threshold D and R ⁇ threshold D, there is no crack in the screen.
  • the threshold D is a distance value, and when H and R are simultaneously smaller than this value, the line can be considered as a crack, but a noise point. If any of H and R is greater than this threshold, it can be judged that there is a lateral or longitudinal crack on the screen, but the crack does not span the entire screen, as shown in Fig. 14 is a form of lateral crack.
  • the conventional method for inspecting cracks is by manual observation.
  • the robot automatically detects whether there is a crack in the preprocessed image by the above method.
  • the control robot moves to a preset area of each service device in the unmanned network point area, and if the robot moves to a preset area of a special service device, the service device to be detected is Sending a preset graphic code to determine whether a circuit fault occurs in the service device to be detected according to the content displayed on the display screen by the preset graphic code. If no circuit failure occurs, controlling the service device to be detected to perform graphic display according to the preset display parameter. And analyzing the image displayed on the display screen according to the preset analysis rule to analyze whether the display screen of the service device to be detected has a preset type of abnormality, the scheme does not need manual participation, and the robot moves to the corresponding area to the device. The circuit fault and the screen display status are automatically detected.
  • the embodiment of the present application further provides a computer readable storage medium, where the screen state automatic detection program is stored, and the screen state automatic detection program can be executed by one or more processors, Implement the following operations:
  • the service device corresponding to the preset area is used as a service device to be detected, and a preset graphic code is sent to the service device to be detected, according to the preset graphic code. Determining, by the content displayed on the display screen, whether the circuit to be detected has a circuit failure;
  • the device to be detected is controlled to perform image display according to the preset display parameter, and the image displayed on the display screen is analyzed to analyze the service to be detected. Whether the preset type of exception has occurred on the display screen of the device.
  • the robot performs scan analysis on the content displayed on the display screen by the service device to be detected based on the display instruction; if the preset information is scanned and analyzed from the display screen of the service device to be detected, the display of the service device to be detected is determined. No circuit failure occurs on the screen; if the preset information is not scanned and analyzed from the display screen of the service device to be detected, it is determined that a circuit fault has occurred on the display screen of the service device to be detected.
  • the display screens of the service device to be detected are respectively displayed in a plurality of preset colors in a manner of displaying in a solid color, wherein different preset colors correspond to display areas of different sizes, shapes of the display areas corresponding to the respective preset colors, and the display
  • the shape of the maximum display area of the screen corresponds to, and the display area corresponding to one of the preset colors is the maximum display area of the display screen; when a preset color is displayed in a solid color display manner, for the preset color Masking the display graphic of the display screen to obtain a mask image; performing anti-interference processing on the mask image corresponding to the preset color of the display area of the smaller size to obtain an anti-interference image, wherein the smaller size
  • the display area is a display area of a size other than the maximum display area; the maximum display area of the display screen is determined according to the acquired anti-interference image, and the preset corresponding to the maximum display area is determined according to the determined maximum display area.
  • the mask image of the color is used to extract the image of the actual display area, and the maximum display image is extracted; Maximum enhancement display image noise filtering pretreatment, to obtain an enhanced image; enhancement of the image is analyzed to analyze whether the display screen of the device to be detected traffic abnormality occurs in the preset type.
  • the technical solution of the present application which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM as described above). , a disk, an optical disk, including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the various embodiments of the present application.
  • a terminal device which may be a mobile phone, a computer, a server, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Electromagnetism (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Toxicology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Manipulator (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
  • Image Analysis (AREA)

Abstract

一种屏幕状态自动检测机器人及屏幕状态自动检测方法，该屏幕状态自动检测机器人包括存储器（11）和处理器（12），存储器（11）上存储有屏幕状态自动检测程序，该程序被处理器执行时实现如下步骤：控制机器人分别移动到无人网点区的各个业务设备的预设区域内；若机器人移动到一个业务设备的预设区域内，则检查该业务设备的显示屏幕是否发生了电路故障；若未发生，则控制该业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出该业务设备的显示屏幕是否发生了预设类型的异常。该屏幕状态自动检测机器人及屏幕状态自动检测方法实现了不通过人力，自动对无人网点区各个自助业务设备的屏幕状态进行检测。

Description

屏幕状态自动检测机器人、方法及计算机可读存储介质
优先权申明
本申请基于巴黎公约申明享有2017年08月29日递交的申请号为201710754580.7、名称为“屏幕状态自动检测机器人、方法及计算机可读存储介质”的中国专利申请的优先权，该中国专利申请的整体内容以参考的方式结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及一种屏幕状态自动检测机器人、方法及计算机可读存储介质。
背景技术
目前，在金融、保险等领域，为了降低人力成本、提高业务办理的便捷性，业界越来越多的部署无人网点区(例如，银行自助网点区)进行业务拓展和办理，无人网点区是一个自助提供银行业务、保险业务等的区域，分布在不同区域的门店。而门店里面有办理不同业务的业务设备，例如，存取款一体机、自动取款机、自助缴费机、为客户准备的PC机(例如，专为大客户使用的PC机，可查询理财、基金、股票信息等)、自助查询设备、排号机、宣传设备等，这些办理不同业务的业务设备通常都需配置终端屏幕。而终端屏幕属于娇贵的电子器件，经常会发生显示问题，一旦发生显示的问题就会影响客户的使用，所以业界通常需要定时检查与维护无人网点区各个自助业务设备的屏幕的状态。
目前，检测自助业务设备的屏幕状态的方案通常包括：方案1、通过人工巡查进行问题的反馈；方案2、通过客户投诉进行问题反馈。
然而，上述方案1的缺陷在于人工巡查的方式比较浪费人力，时效性比较差，而且有些细节还不一定能仔细检查到，如细小裂纹人工不一定仔细察觉，容易发生错漏。上述方案2的缺陷在于会很影响用户的体验，而且很多用户未必会进行反馈，即使用户进行反馈，通常反应的问题不够全面和客观，不仅时效性差，而且同样会发生错漏。因此，如何自动且准确地对无人网点区各个自助业务设备的屏幕状态进行检测，已经成为一个亟待解决的技术问题。
发明内容
本申请提供一种屏幕状态自动检测机器人、方法及计算机可读存储介质，其主要目的在于实现不通过人力，自动地对无人网点区各个自助业务设备的屏幕状态进行检测。
为实现上述目的，本申请提供一种屏幕状态自动检测机器人，该机器人包括存储器和处理器，所述存储器上存储有屏幕状态自动检测程序，所述屏幕状态自动检测程序被所述处理器执行时实现如下操作：
控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
此外，为实现上述目的，本申请还提供一种屏幕状态自动检测方法，该方法包括：
控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业 务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
此外，为实现上述目的，本申请还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有屏幕状态自动检测程序，所述屏幕状态自动检测程序可被一个或者多个处理器执行，以实现如下步骤：
控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
本申请提出的屏幕状态自动检测机器人、方法及计算机可读存储介质，控制机器人分别移动到无人网点区的各个业务设备的预设区域内，若机器人移动到一个特务设备的预设区域内，则向待检测业务设备发送预设图形码，以根据预设图形码在显示屏幕上显示的内容判断待检测业务设备是否发生了电路故障，若未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图形显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常，该方案不需要人工参与，通过机器人移动到对应区域对设备的电路故障以及屏幕显示状态进行自动检测。
附图说明
图1为本申请屏幕状态自动检测机器人较佳实施例的示意图；
图2a至图2d为部分信息被遮挡的二维码；
[根据细则91更正 11.01.2018]　
图3a为发生了透视的二维码；图3b为透视后的二维码的外轮廓模型；图3c为将透视后的二维码进行反透视后的效果；
图4为在显示屏幕上设置的不同尺寸的显示区域的示意图；
图5为小尺寸矩形区域和中尺寸矩形显示区域的边界；
图6为大尺寸矩形区域的外轮廓曲线；
图7为从矩形边界上取的多个点的示意图；
图8为对获取的点通过最小二乘法直线拟合算法得到的直线的示意图；
图9为进行滤噪增强预处理前的屏幕状态；
图10为进行滤噪增强预处理后的屏幕状态；
图11为显示屏幕出现坏线的状态示意图；
图12为显示屏幕出现坏斑的状态示意图；
图13为显示屏幕出现裂缝的状态示意图；
图14为显示屏幕出现的横向裂缝的状态示意图；
图15为本申请屏幕状态自动检测机器人一实施例中的屏幕状态自动检测程序的程序模块示意图；
图16为本申请屏幕状态自动检测方法较佳实施例的流程图；
图17为本申请屏幕状态自动检测方法第二实施例的流程图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请提供一种屏幕状态自动检测机器人。参照图1所示，为本申请屏幕状态自动检测机器人较佳实施例的示意图。
在本实施例中，屏幕状态自动检测机器人包括存储器11、处理器12，通信总线13，网络接口14以及摄像头15。
其中，存储器11至少包括一种类型的可读存储介质，所述可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、磁性存储器、磁盘、光盘等。存储器11在一些实施例中可以是屏幕状态自动检测机器人的内部存储单元，例如该屏幕状态自动检测机器人的硬盘。存储器11在另一些实施例中也可以是屏幕状态自动检测机器人的外部存储设备，例如屏幕状态自动检测机器人上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，存储器11还可以既包括屏幕状态自动检测机器人的内部存储单元也包括外部存储设备。存储器11不仅可以用于存储安装于屏幕状态自动检测机器人的应用软件及各类数据，例如屏幕状态自动检测程序的代码等，还可以用于暂时地存储已经输出或者将要输出的数据。
处理器12在一些实施例中可以是一中央处理器(Central Processing Unit,CPU)、控制器、微控制器、微处理器或其他数据处理芯片，用于运行存储器11中存储的程序代码或处理数据，例如执行屏幕状态自动检测程序等。
通信总线13用于实现这些组件之间的连接通信。
网络接口14可选的可以包括标准的有线接口、无线接口(如WI-FI接口)，通常用于在该机器人与其他电子设备之间建立通信连接。在该实施例中，机器人可以通过网络接口14与用户终端连接，接收用户终端发送的检测指令，也可以通过网络接口14与业务设备连接，控制业务设备按照预设显示参数进行图像的显示。
摄像头15用于对业务设备的显示屏幕的显示内容进行采集。
图1仅示出了具有组件11-15以及屏幕状态自动检测程序的屏幕状态自动检测机器人，但是应理解的是，并不要求实施所有示出的组件，可以替代的实施更多或者更少的组件。
可选地，该机器人还可以包括用户接口，用户接口可以包括显示器(Display)、输入单元比如键盘(Keyboard)、用于触发检测指令的物理按键等，可选的用户接口还可以包括标准的有线接口、无线接口。可选地，在一些实施例中，显示器可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED(Organic Light-Emitting Diode，有机发光二极管)触摸器等。其中，显示器也可以适当的称为显示屏或显示单元。
可选地，该机器人还可以包括RF(Radio Frequency，射频)电路，传感器、WiFi模块等。其中，传感器可以是光传感器、距离传感器等。
在图1所示的实施例中，存储器11中存储有屏幕状态自动检测程序；处理器12执行该程序时实现如下操作：
S1、控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内。
该实施例提供的机器人可以放置于无人网点区，无人网点区的业务设备可以有一个或者多个。可以实时、定时、在处于空闲状态时或者在接收到检测指令时，控制机器人移动到无人网点区，开启对无人网点区内的各个业务设备的屏幕显示状态的检测。其中，当机器人在预设时长内处于无需处理业务时，确定其处于空闲状态。
在无人网点区中的每一台业务设备划分有预设区域，一个业务设备的预设区域指的是距离该业务机器的距离小于或者等于预设距离的区域，机器人在移动过程中，通过对自身位置进行定位确定在无人网点区中的位置坐标，并结合已知的各个业务设备在无人网点区中的位置坐标，计算出当前位置与预先确定的各个业务设备所处位置之间的距离，若当前 位置与一个业务设备所处位置之间的距离小于或者等于预设距离，则确定该机器人移动到该业务设备的预设区域。
当无人网点区中设置有多台业务设备时，需要逐个地对各个业务设备进行检测。控制机器人分别移动到无人网点区的各个业务设备的预设区域内的方式可以有多种，以下列举其中两种方式进行说明。
在一实施例中，按照预设移动导航路径控制机器人的移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，控制机器人按照所述移动导航路径继续向未检测的业务设备进移动，直到所有业务设备的显示屏幕均检测完毕后结束移动。
或者，在其他的实施例中，可以控制机器人随机移动，在移动到一个业务设备的预设区域内且对该待检测业务设备的显示屏幕检测完毕后，控制机器人标记该待检测业务设备为障碍物，进行障碍物规避移动，避障移动完毕后，控制机器人继续进行随机移动，移动到另一台业务设备进行检测，直到所有业务设备均被标记为障碍物后结束移动，并将业务设备的障碍物标记清除。
具体地，在机器人随机移动的过程中，障碍物规避移动算法的原理是：当机器人对一台业务设备检测完毕后，在当前的移动方向上向左或者向右偏移预设角度，并判断偏移角度后是否可以不受阻碍地移动，若偏移角度后可以通行，则沿着偏移后的方向继续随机移动，若偏移角度后受到阻碍无法通行，则继续沿同一方向偏移预设角度直至能够不受阻碍的移动，通过这样不断调整偏移角度的算法对阻碍移动的障碍物进行规避。
S2、若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障。
当机器人在进入某一预设区域后，将该预设区域对应的业务设备作为待检测的业务设备，对该预设区域对应的业务设备进行检测。机器人可以调整摄像头15的角度直至能够拍摄到该业务设备当前的显示屏幕的显示画面，然后向该待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障。具体地，控制机器人与待检测的业务设备(即当前所述的预设区域对应的业务设备)进行无线通信，向待检测的业务设备发送显示包含有预设信息的预设图形码的显示指令，业务设备根据接收到的显示指令显示该预设图形码；控制机器人对待检测业务设备的显示屏幕的显示内容进行扫描分析；若从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕没有发生电路故障；若没有从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕发生了电路故障。
在一些实施例中，所述预设图形码可以是带有预设信息的二维码。使用二维码进行电路故障的检测具有以下优点：二维码的容错能力强，并且具有纠错能力，参照图2a至图2d所示，即使显示屏幕上有污迹、斑点、裂缝等干扰情况造成有部分信息被遮挡，二维码仍然可以被识别出来，不会影响通过二维码的方式判断是否能上电，即判断是否发生电路故障；二维码可包含信息，例如屏幕的ID，也就是说即使当机器人同时看到多台业务设备的显示屏幕，也可以根据二维码所包含的屏幕ID中识别出所在预设区域对应的业务设备由于二维码的外轮廓是正方形，因此，不要求机器人严格正对目标屏幕也可以识别，当在机器人视觉里面产生透视时，机器人很容易对于透视造成的变形进行还原，如图3a至图3c所示。
由图3a和图3b可以得知，机器人在用摄像头对屏幕内容进行处理的时候，由于机器人所处的位置可能会造成一定程度的透视，导致二维码外轮廓显示在机器人的镜头中可能是非矩形的。从图3b可以看出二维码显示的并不是一个正方形，而是四边形，产生了“近大远小”的透视。但是二维码识别可以在产生透视的情况下进行反透视从而能正常识别该 二维码，反透视方法是现有成熟方法，此处不做详细解释。参照图3c所示，为将二维码进行反透视后的效果。
进一步地，在一些实施例中，若显示屏幕发生了电路故障，则向预先确定的监控设备发送提示信息，提示所述待检测业务设备的显示屏幕发生了电路故障需要进行处理。例如，向后台服务器发送提示信息，该提示信息包含发生电路故障的业务设备的唯一身份ID，及所述待检测业务设备所处的无人网点区的网点标识，例如，该提示信息格式可以为“网点标识为*****的无人网点区，ID号为*****的业务设备的显示屏幕发生了电路故障需要进行处理”。
S3、若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
S3包括如下具体操作：
S31，控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，例如，红、绿、蓝三种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域。
S32，在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，即将显示屏幕的除了该预设颜色的其他颜色屏蔽掉，以获取掩码图像。
S33，为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域。
S34，根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像，因为如果屏幕周边范围有干扰区域，例如，屏幕周围有“进水”情况，屏幕边界区域无法正常显示，那么取完掩码获取到的最大图也是不完整的，相当于把边界干扰问题排除在检测之外了，因此需要通过上述方式获取最大显示区域。
参照图4所示，以在显示屏幕设置三个不同尺寸的显示区域为例，其中，小尺寸矩形的屏幕宽度为L1、中尺寸矩形的屏幕宽度为L2、大尺寸矩形的屏幕宽度为L3。机器人在控制屏幕显示的时候可以设定L1、L2和L3的比例关系：L3＝θ1L2＝θ2L1，即θ1和θ2均为已知数。此外，屏幕的长宽之间的比例关系ζ也为已知数据。当根据显示内容获取L1或者L2的值之后，可以根据上述L1、L2和L3的比例关系计算出显示屏幕的宽L3，继而根据屏幕的长宽比例关系L/W＝ζ计算出显示屏幕的长W。
具体地，根据对应较小尺寸的显示区域的预设颜色的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像的原理说明如下：
利用小尺寸矩形区域的宽L1通过L3＝θ1L2＝θ2L1可以求解出大尺寸矩形框的宽，记为L'3，同样，利用中尺寸矩形框的宽L2也可以求解出大尺寸矩形的宽，并记为L″3，本申请为了减少误差，优选地，根据公式1取这两个值的平均值作为大尺寸矩形的宽：
Figure PCTCN2017108740-appb-000001
然后根据屏幕的长宽比例关系L/W＝ζ计算出大尺寸矩形的长W。从而可以得到大尺寸矩形的区域范围，而大尺寸矩形的区域范围就是屏幕最大显示区域，即图6所示的最大尺寸矩形区域的外轮廓曲线。
但是，在分别取不同颜色的掩码图像后，由于屏幕边界本身可能有其他颜色的干扰，例如，屏幕边界可能本身就存在斑点，通过不同颜色分别取掩码后的图可能依然不是完整 的图。针对取掩码后的图像存在不完整的现象，本申请采用的解决方案如下：
获取对应较小尺寸的显示区域的预设颜色的掩码图像的每条边界线上的多个点的坐标；
分别基于所述各个边界线上的多个点的坐标，通过最小二乘法直线拟合算法获取所述掩码图像的每条边界线的延长线；
根据所述掩码图像的每条边界线的延长线获取所述四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置，也就是中尺寸或者小尺寸矩形边框的范围。
将所述四边形进行反透视变换为矩形，根据所述掩码图像矩形的位置获取所述抗干扰图像。
参照图5所示，分别为计算出来的小尺寸矩形的边界和中尺寸矩形的边界。参照图7所示，取边界上的几个点，通过最小二乘法直线拟合算法模拟出该边界，如图8所示。其中，最小二乘法直线拟合算法为y计＝a0+a1*x。其中，y计是模拟后的直线，a0、a1是待求参数。根据算法得到a0、a1后，即可得到该边界线的延长线，通过上述方式分别获取四条边界线的延长线，接着确定四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置，也就是中尺寸矩形或者小尺寸矩形的范围，进而确定精确的L1和L2的值。
S35，对所述最大显示图像进行滤噪增强预处理，获得增强图像。
对最大显示图像进行滤噪增强预处理，获得增强图像的原理说明如下：由于采集到的屏幕异常状态图像会受到非均匀光照、屏幕表面不规则间隙和设备等影响，从而使得采集到的图像有噪声，所以需要去除这些噪声，同时能够保持住屏幕异常状态的细节，并尽可能能提高裂缝与背景的对比度，从而有利于后期的分割与识别。从图9的预处理前和图10预处理后的图可以看出，滤噪增强的图噪声干扰少，而且问题比较清晰可见。
本申请利用引导滤波器的方法对图像进行预处理。利用引导滤波器的算法步骤如下所示：以灰度化后的屏幕图像作为输入图像P，同样以灰度化后的屏幕图像作为引导图像I，利用噪音滤波器滤波后的图像为q，利用公式2来增强输入图像P。
P_enhanced＝(I-q)m+q，公式2
其中m为增强系数，可根据实际情况来定，此处的滤波器可以根据实际情况选择，这里我们选择引导滤波器给图像P做滤波处理。
此处的参数m的调整可以根据P_enhanced(增强图像P)的效果进行反馈重新调整。此处我们构想一种迭代求解方案：此方案主要是通过对P_enhanced的效果进行反馈，对表现良好的P_enhanced给予更高的权值，反之给予稍微低的权值。
下述式子的ε是一个满意度比重，enchanted满意度是根据P_enhanced图像效果进行评估的，enhanced的最高满意度我们设置为一个固定值，例如500。
Figure PCTCN2017108740-appb-000002
为了更好调节满意度值，本申请通过下述式子将ε值alpha化，过程如下：
Figure PCTCN2017108740-appb-000003
将迭代第n次记为Tn，则第n+1次可记为Tn+1，经过地n次迭代后m的取值为
Figure PCTCN2017108740-appb-000004
则有：
Figure PCTCN2017108740-appb-000005
迭代的过程如下：首先我们给定一个初始权值m作为
Figure PCTCN2017108740-appb-000006
然后代入公式2求解出 P_enhanced后的图像，然后根据公式3对P_enhanced后的效果图进行满意度评价，将得到的满意度比重ε进行alpha化，然后利用alpha化后的值和公式5对m权值进行更新。然后依次重复执行以上步骤对m进行更新，经过几轮迭代后便可以得到一个比较理想的m权值。因为每一次迭代都会对权值m进行修改，对图像P_enhanced效果表现较差时ε值会降低，从而导致最终m权值也降低，相反会相应使得权值增加。而每一次调整都会驱使图像P_enhanced效果往更好的方向走，将迭代次数设置为固定值(这里设定为10)，迭代n次后将会有一个收敛的状态，也就是图像P_enhanced效果接近最好的状态。将增强后的图像P_enhanced作为输入图像P，灰度化后的屏幕图像作为引导图像I，得到滤波后的图像为最终增强且平滑后的图像。
S36，对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常，所述预设类型的异常包括坏斑、坏线与裂缝。
对所述增强图像进行坏斑检测分析，以判断所述待检测业务设备的显示屏幕是否发生了坏斑。
参照图11所示，为屏幕出现坏线的情况。
在一实施例中，坏线的识别流程如下：获取经过上述步骤预处理后的图像，使用霍夫变换检测直线，当检测到坏线数大于1，则判断存在坏线，否则，判断不存在坏线。
关于霍夫变换，霍夫变换是一种特征提取技术。经典的霍夫变换能够识别出图像中的直线，后来又发展到能够识别出任意形状，但更常见的识别形状是圆和椭圆，这里我们用来识别屏幕环线的状态。
在直角坐标系下，直线被定义为：
y＝mx+b，公式6
其中，m为斜率，b为与y轴的截距，只要确定了m和b，一条直线就可以被唯一地确定下来。如果用ρ0表示原点到该直线的代数距离，θ0表示该直线的正交线与x轴的夹角，则：
Figure PCTCN2017108740-appb-000007
Figure PCTCN2017108740-appb-000008
则该直线又可表示为：
Figure PCTCN2017108740-appb-000009
写成更一般的形式：
ρ＝x cosθ+y sinθ，公式10
很容易想到，(ρ，θ)是极坐标的表示形式。但如果把(ρ，θ)也用直角坐标的形式表示，即把ρ和θ做正交处理，则(ρ，θ)就被称为霍夫空间。
在直角坐标系中的一点，对应于霍夫空间的一条正弦曲线。直线是由无数个点组成的，在霍夫空间就是无数条正弦曲线，但这些正弦曲线会相交于一点(ρ0，θ0)，把该点带入公式7和公式8就得了直线的斜率和截距，这样一条直线就被确定了下来。因此用霍夫变换识别直线时，霍夫空间中的极大值就有可能对应一条直线。
传统的检查坏线的方法是通过人工观察，通过本实施例的方法，机器人通过上述方式对经过预处理后的图像自动检测是否存在坏线。
对所述增强图像进行坏线检测分析，以判断所述待检测业务设备的显示屏幕是否发生 了坏线。参照图12所示，为屏幕出现坏斑的情况。
在一实施例中，在斑点检测上，判断坏斑的流程是，获取预处理后的图片，用SimpleBlobDetector(斑点检测算子)算法检测是否有斑点的存在，当检测到的斑点数大于1时，判断屏幕存在坏斑的情况，否则判断屏幕不存在坏斑的情况。
此外，为了简化对定坏斑的面积范围，二值化阈值等参数的设置，该实施例还引入一种SimpleBlobDetector算法中参数的选取方法。
选取一些屏幕坏斑的图片做统计，并标记哪些图片是存在坏斑。选择不同参数的进行处理，并对处理后的结果进行分析。如下表所示：
表1求解特征k取值
Figure PCTCN2017108740-appb-000010
特征取k值时判断是斑点在实际是斑点中的比例：
Figure PCTCN2017108740-appb-000011
在特征取值为k时，判断是斑点但实际又不是斑点的数量在实际不是斑点的数量中占的比例：
Figure PCTCN2017108740-appb-000012
这里我们需要得出在特征取值为k时的权值系数，我们可以定义为：
Figure PCTCN2017108740-appb-000013
上式的意义在于在特征取值为k时，尽量使得判断是斑点正确的概率越高，同时使得判断是斑点但实际上不是斑点的概率越低时，权值wk越大。选取每个特征参数wk值最大的作为SimpleBlobDetector算法的特征点检测需要确定的参数。从而不用人为判定这个参数的具体数值。
传统的检查坏斑的方法是通过人工观察，通过本实施例的方法，机器人通过上述方式对经过预处理后的图像自动检测是否存在坏斑。
对所述增强图像进行裂缝检测分析，以判断所述待检测业务设备的显示屏幕是否发生了裂缝。参照图13所示，为屏幕出现裂缝的情况。
在一实施例中，按照如下步骤检测屏幕是否出现裂缝：
对图像进行分割，这里我们使用现有成熟方法均值漂移分割算法可以把裂缝分割出来。参照图14所示。然后，对分割后的图像进行裂缝的识别：分割出来后可以看到一些连通区域，连通区域在这里是指裂缝分割出来后组成的图像区域，当检查到连通区域数大于1时，可以判断图像是有裂缝，而裂缝可能是网状或是单条横穿屏幕的裂缝。若只有1个连通区域，我们分别获取裂缝的X轴投影距离H，与y轴的投影距离R，如图14所示。如果H<阈值D并且R<阈值D就表明屏幕没有裂缝。其中，阈值D是一个距离值，当H与R同时小于这个值的时候便可以认为该线条不是裂缝，而是噪音点。假如H与R其中任意一个大于这个阈值就可以判断屏幕上存在横向或纵向裂缝，但是这裂缝不是跨越整个屏幕，如图14所展示的就是横向裂缝的一种形式。
传统的检查裂缝方法是通过人工观察，通过本实施例的方法，机器人通过上述方式对 经过预处理后的图像自动检测是否存在裂缝。
上述实施例中，控制机器人分别移动到无人网点区的各个业务设备的预设区域内，若机器人移动到一个特务设备的预设区域内，则向待检测业务设备发送预设图形码，以根据预设图形码在显示屏幕上显示的内容判断待检测业务设备是否发生了电路故障，若未发生电路故障，则控制待检测业务设备按照预设显示参数进行图形显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出待检测业务设备的显示屏幕是否发生了预设类型的异常，该方案不需要人工参与，通过机器人移动到对应区域对设备的电路故障以及屏幕显示状态进行自动检测。
可选地，在其他的实施例中，屏幕状态自动检测程序还可以被分割为一个或者多个模块，一个或者多个模块被存储于存储器11中，并由一个或多个处理器(本实施例为处理器12)所执行，以完成本申请，本申请所称的模块是指能够完成特定功能的一系列计算机程序指令段。
参照图15所示，为本申请屏幕状态自动检测机器人一实施例中的屏幕状态自动检测程序的程序模块示意图，该实施例中，屏幕状态自动检测程序可以被分割为控制移动模块10、第一检测模块20和第二检测模块30，所述模块10-30被执行所实现的功能或操作步骤与上述实施例大体相同，这里不再详述。示例性地，控制移动模块10用于控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
第一检测模块20用于若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，则向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
第二检测模块30用于若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
此外，本申请还提供一种屏幕状态自动检测方法。参照图16所示，为本申请屏幕状态自动检测方法较佳实施例的流程图。该方法可以由一个装置执行，该机器人可以由软件和/或硬件实现。
在本实施例中，屏幕状态自动检测方法包括：
步骤S10，控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
该实施例中的机器人可以放置于无人网点区，无人网点区的业务设备可以有一个或者多个。可以实时、定时、在处于空闲状态时或者在接收到检测指令时，控制机器人移动到无人网点区，开启对无人网点区内的各个业务设备的屏幕显示状态的检测。其中，当机器人在预设时长内处于无需处理业务时，确定其处于空闲状态。
在无人网点区中的每一台业务设备划分有预设区域，一个业务设备的预设区域指的是距离该业务机器的距离小于或者等于预设距离的区域，机器人在移动过程中，通过对自身位置进行定位确定在无人网点区中的位置坐标，并结合已知的各个业务设备在无人网点区中的位置坐标，计算出当前位置与预先确定的各个业务设备所处位置之间的距离，若当前位置与一个业务设备所处位置之间的距离小于或者等于预设距离，则确定该机器人移动到该业务设备的预设区域。
当无人网点区中设置有多台业务设备时，需要逐个地对各个业务设备进行检测。控制机器人分别移动到无人网点区的各个业务设备的预设区域内的方式可以有多种，以下列举其中两种方式进行说明。
在一实施例中，按照预设移动导航路径控制机器人的移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，控制机器人按照所述移动导航路径继续向未检测的业务设备进移动，直到所有业务设备的显示屏幕均检测完毕后结束移动。
或者，在其他的实施例中，可以控制机器人随机移动，在移动到一个业务设备的预设区域内且对该待检测业务设备的显示屏幕检测完毕后，控制机器人标记该待检测业务设备为障碍物，进行障碍物规避移动，避障移动完毕后，控制机器人继续进行随机移动，移动到另一台业务设备进行检测，直到所有业务设备均被标记为障碍物后结束移动，并将业务设备的障碍物标记清除。
具体地，在机器人随机移动的过程中，障碍物规避移动算法的原理是：当机器人对一台业务设备检测完毕后，在当前的移动方向上向左或者向右偏移预设角度，并判断偏移角度后是否可以不受阻碍地移动，若偏移角度后可以通行，则沿着偏移后的方向继续随机移动，若偏移角度后受到阻碍无法通行，则继续沿同一方向偏移预设角度直至能够不受阻碍的移动，通过这样不断调整偏移角度的算法对阻碍移动的障碍物进行规避。
步骤S20，若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障。
当机器人在进入某一预设区域后，将该预设区域对应的业务设备作为待检测的业务设备，对该预设区域对应的业务设备进行检测。机器人可以调整摄像头15的角度直至能够拍摄到该业务设备当前的显示屏幕的显示画面，然后向该待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障。具体地，控制机器人与待检测的业务设备(即当前所述的预设区域对应的业务设备)进行无线通信，向待检测的业务设备发送显示包含有预设信息的预设图形码的显示指令，业务设备根据接收到的显示指令显示该预设图形码；控制机器人对待检测业务设备的显示屏幕的显示内容进行扫描分析；若从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕没有发生电路故障；若没有从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕发生了电路故障。
在一些实施例中，所述预设图形码可以是带有预设信息的二维码。使用二维码进行电路故障的检测具有以下优点：二维码的容错能力强，并且具有纠错能力，参照图2a至图2d所示，即使显示屏幕上有污迹、斑点、裂缝等干扰情况造成有部分信息被遮挡，二维码仍然可以被识别出来，不会影响通过二维码的方式判断是否能上电，即判断是否发生电路故障；二维码可包含信息，例如屏幕的ID，也就是说即使当机器人同时看到多台业务设备的显示屏幕，也可以根据二维码所包含的屏幕ID中识别出所在预设区域对应的业务设备；由于二维码的外轮廓是正方形，因此，不要求机器人严格正对目标屏幕也可以识别，当在机器人视觉里面产生透视时，机器人很容易对于透视造成的变形进行还原，如图3a至图3c所示。
由图3a和图3b可以得知，机器人在用摄像头对屏幕内容进行处理的时候，由于机器人所处的位置可能会造成一定程度的透视，导致二维码外轮廓显示在机器人的镜头中可能是非矩形的。从图3b可以看出二维码显示的并不是一个正方形，而是四边形，产生了“近大远小”的透视。但是二维码识别可以在产生透视的情况下进行反透视从而能正常识别该二维码，反透视方法是现有成熟方法，此处不做详细解释。参照图3c所示，为将二维码进行反透视后的效果。
进一步地，在一些实施例中，若显示屏幕发生了电路故障，则向预先确定的监控设备发送提示信息，提示所述待检测业务设备的显示屏幕发生了电路故障需要进行处理。例如，向后台服务器发送提示信息，该提示信息包含发生电路故障的业务设备的唯一身份ID，及所述待检测业务设备所处的无人网点区的网点标识，例如，该提示信息格式可以为“网点标识为*****的无人网点区，ID号为*****的业务设备的显示屏幕发生了电路故障需要进行处理”。
步骤S30，若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业 务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
参照图17所示，为本申请屏幕状态自动检测方法较佳实施例中步骤S30的细化示意图，步骤S30包括：
步骤S301，控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，例如，红、绿、蓝三种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域。
步骤S302，在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，即将显示屏幕的除了该预设颜色的其他颜色屏蔽掉，以获取掩码图像。
步骤S303，为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域。
步骤S304，根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像，因为如果屏幕周边范围有干扰区域，例如，屏幕周围有“进水”情况，屏幕边界区域无法正常显示，那么取完掩码获取到的最大图也是不完整的，相当于把边界干扰问题排除在检测之外了，因此需要通过上述方式获取最大显示区域。
参照图4所示，以在显示屏幕设置三个不同尺寸的显示区域为例，其中，小尺寸矩形的屏幕宽度为L1、中尺寸矩形的屏幕宽度为L2、大尺寸矩形的屏幕宽度为L3。机器人在控制屏幕显示的时候可以设定L1、L2和L3的比例关系：L3＝θ1L2＝θ2L1，即θ1和θ2均为已知数。此外，屏幕的长宽之间的比例关系ζ也为已知数据。当根据显示内容获取L1或者L2的值之后，可以根据上述L1、L2和L3的比例关系计算出显示屏幕的宽L3，继而根据屏幕的长宽比例关系L/W＝ζ计算出显示屏幕的长W。
具体地，根据对应较小尺寸的显示区域的预设颜色的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像的原理说明如下：
利用小尺寸矩形区域的宽L1通过L3＝θ1L2＝θ2L1可以求解出大尺寸矩形框的宽，记为L'3，同样，利用中尺寸矩形框的宽L2也可以求解出大尺寸矩形的宽，并记为L”3，本申请为了减少误差，优选地，根据公式1取这两个值的平均值作为大尺寸矩形的宽：
Figure PCTCN2017108740-appb-000014
然后根据屏幕的长宽比例关系L/W＝ζ计算出大尺寸矩形的长W。从而可以得到大尺寸矩形的区域范围，而大尺寸矩形的区域范围就是屏幕最大显示区域，即图6所示的最大尺寸矩形区域的外轮廓曲线。
但是，在分别取不同颜色的掩码图像后，由于屏幕边界本身可能有其他颜色的干扰，例如，屏幕边界可能本身就存在斑点，通过不同颜色分别取掩码后的图可能依然不是完整的图。针对取掩码后的图像存在不完整的现象，本申请采用的解决方案如下：
获取对应较小尺寸的显示区域的预设颜色的掩码图像的每条边界线上的多个点的坐标；
分别基于所述各个边界线上的多个点的坐标，通过最小二乘法直线拟合算法获取所述掩码图像的每条边界线的延长线；
根据所述掩码图像的每条边界线的延长线获取所述四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置，也就是中尺寸或者 小尺寸矩形边框的范围。
将所述四边形进行反透视变换为矩形，根据所述掩码图像矩形的位置获取所述抗干扰图像。
参照图5所示，分别为计算出来的小尺寸矩形的边界和中尺寸矩形的边界。参照图7所示，取边界上的几个点，通过最小二乘法直线拟合算法模拟出该边界，如图8所示。其中，最小二乘法直线拟合算法为y计＝a0+a1*x。其中，y计是模拟后的直线，a0、a1是待求参数。根据算法得到a0、a1后，即可得到该边界线的延长线，通过上述方式分别获取四条边界线的延长线，接着确定四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置，也就是中尺寸矩形或者小尺寸矩形的范围，进而确定精确的L1和L2的值。
步骤S305，对所述最大显示图像进行滤噪增强预处理，获得增强图像。
对最大显示图像进行滤噪增强预处理，获得增强图像的原理说明如下：
由于采集到的屏幕异常状态图像会受到非均匀光照、屏幕表面不规则间隙和设备等影响，从而使得采集到的图像有噪声，所以需要去除这些噪声，同时能够保持住屏幕异常状态的细节，并尽可能能提高裂缝与背景的对比度，从而有利于后期的分割与识别。从图9的预处理前和图10预处理后的图可以看出，滤噪增强的图噪声干扰少，而且问题比较清晰可见。
本申请利用引导滤波器的方法对图像进行预处理。利用引导滤波器的算法步骤如下所示：
以灰度化后的屏幕图像作为输入图像P，同样以灰度化后的屏幕图像作为引导图像I，利用噪音滤波器滤波后的图像为q，利用公式2来增强输入图像P。
P_enhanced＝(I-q)m+q，公式2
其中m为增强系数，可根据实际情况来定，此处的滤波器可以根据实际情况选择，这里我们选择引导滤波器给图像P做滤波处理。
此处的参数m的调整可以根据P_enhanced(增强图像P)的效果进行反馈重新调整。此处我们构想一种迭代求解方案：此方案主要是通过对P_enhanced的效果进行反馈，对表现良好的P_enhanced给予更高的权值，反之给予稍微低的权值。
下述式子的ε是一个满意度比重，enchanted满意度是根据P_enhanced图像效果进行评估的，enhanced的最高满意度我们设置为一个固定值，例如500。
Figure PCTCN2017108740-appb-000015
为了更好调节满意度值，本申请通过下述式子将ε值alpha化，过程如下：
Figure PCTCN2017108740-appb-000016
将迭代第n次记为Tn，则第n+1次可记为Tn+1，经过地n次迭代后m的取值为
Figure PCTCN2017108740-appb-000017
则有：
Figure PCTCN2017108740-appb-000018
迭代的过程如下：首先我们给定一个初始权值m作为
Figure PCTCN2017108740-appb-000019
然后代入公式2求解出P_enhanced后的图像，然后根据公式3对P_enhanced后的效果图进行满意度评价，将得到的满意度比重ε进行alpha化，然后利用alpha化后的值和公式5对m权值进行更新。然后依次重复执行以上步骤对m进行更新，经过几轮迭代后便可以得到一个比较理想的m权值。因为每一次迭代都会对权值m进行修改，对图像P_enhanced效果表现较差时ε值会降低，从而导致最终m权值也降低，相反会相应使得权值增加。而每一次调整都会驱使图 像P_enhanced效果往更好的方向走，将迭代次数设置为固定值(这里设定为10)，迭代n次后将会有一个收敛的状态，也就是图像P_enhanced效果接近最好的状态。将增强后的图像P_enhanced作为输入图像P，灰度化后的屏幕图像作为引导图像I，得到滤波后的图像为最终增强且平滑后的图像。
步骤S306，对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常，所述预设类型的异常包括坏斑、坏线与裂缝。
对所述增强图像进行坏斑检测分析，以判断所述待检测业务设备的显示屏幕是否发生了坏斑。
参照图11所示，为屏幕出现坏线的情况。
在一实施例中，坏线的识别流程如下：获取经过上述步骤预处理后的图像，使用霍夫变换检测直线，当检测到坏线数大于1，则判断存在坏线，否则，判断不存在坏线。
关于霍夫变换，霍夫变换是一种特征提取技术。经典的霍夫变换能够识别出图像中的直线，后来又发展到能够识别出任意形状，但更常见的识别形状是圆和椭圆，这里我们用来识别屏幕环线的状态。
在直角坐标系下，直线被定义为：
y＝mx+b，公式6
其中，m为斜率，b为与y轴的截距，只要确定了m和b，一条直线就可以被唯一地确定下来。如果用ρ0表示原点到该直线的代数距离，θ0表示该直线的正交线与x轴的夹角，则：
Figure PCTCN2017108740-appb-000020
Figure PCTCN2017108740-appb-000021
则该直线又可表示为：
Figure PCTCN2017108740-appb-000022
写成更一般的形式：
ρ＝x cosθ+y sinθ，公式10
很容易想到，(ρ，θ)是极坐标的表示形式。但如果把(ρ，θ)也用直角坐标的形式表示，即把ρ和θ做正交处理，则(ρ，θ)就被称为霍夫空间。
在直角坐标系中的一点，对应于霍夫空间的一条正弦曲线。直线是由无数个点组成的，在霍夫空间就是无数条正弦曲线，但这些正弦曲线会相交于一点(ρ0，θ0)，把该点带入公式7和公式8就得了直线的斜率和截距，这样一条直线就被确定了下来。因此用霍夫变换识别直线时，霍夫空间中的极大值就有可能对应一条直线。
传统的检查坏线的方法是通过人工观察，通过本实施例的方法，机器人通过上述方式对经过预处理后的图像自动检测是否存在坏线。
对所述增强图像进行坏线检测分析，以判断所述待检测业务设备的显示屏幕是否发生了坏线。参照图12所示，为屏幕出现坏斑的情况。
在一实施例中，在斑点检测上，判断坏斑的流程是，获取预处理后的图片，用SimpleBlobDetector(斑点检测算子)算法检测是否有斑点的存在，当检测到的斑点数大于1时，判断屏幕存在坏斑的情况，否则判断屏幕不存在坏斑的情况。
此外，为了简化对定坏斑的面积范围，二值化阈值等参数的设置，该实施例还引入一 种SimpleBlobDetector算法中参数的选取方法。
选取一些屏幕坏斑的图片做统计，并标记哪些图片是存在坏斑。选择不同参数的进行处理，并对处理后的结果进行分析。如表1所示。
特征取k值时判断是斑点在实际是斑点中的比例：
Figure PCTCN2017108740-appb-000023
在特征取值为k时，判断是斑点但实际又不是斑点的数量在实际不是斑点的数量中占的比例：
Figure PCTCN2017108740-appb-000024
这里我们需要得出在特征取值为k时的权值系数，我们可以定义为：
Figure PCTCN2017108740-appb-000025
上式的意义在于在特征取值为k时，尽量使得判断是斑点正确的概率越高，同时使得判断是斑点但实际上不是斑点的概率越低时，权值wk越大。选取每个特征参数wk值最大的作为SimpleBlobDetector算法的特征点检测需要确定的参数。从而不用人为判定这个参数的具体数值。
传统的检查坏斑的方法是通过人工观察，通过本实施例的方法，机器人通过上述方式对经过预处理后的图像自动检测是否存在坏斑。
对所述增强图像进行裂缝检测分析，以判断所述待检测业务设备的显示屏幕是否发生了裂缝。参照图13所示，为屏幕出现裂缝的情况。
在一实施例中，按照如下步骤检测屏幕是否出现裂缝：
对图像进行分割，这里我们使用现有成熟方法均值漂移分割算法可以把裂缝分割出来。参照图14所示。然后，对分割后的图像进行裂缝的识别：分割出来后可以看到一些连通区域，连通区域在这里是指裂缝分割出来后组成的图像区域，当检查到连通区域数大于1时，可以判断图像是有裂缝，而裂缝可能是网状或是单条横穿屏幕的裂缝。若只有1个连通区域，我们分别获取裂缝的X轴投影距离H，与y轴的投影距离R，如图14所示。如果H<阈值D并且R<阈值D就表明屏幕没有裂缝。其中，阈值D是一个距离值，当H与R同时小于这个值的时候便可以认为该线条不是裂缝，而是噪音点。假如H与R其中任意一个大于这个阈值就可以判断屏幕上存在横向或纵向裂缝，但是这裂缝不是跨越整个屏幕，如图14所展示的就是横向裂缝的一种形式。
传统的检查裂缝方法是通过人工观察，通过本实施例的方法，机器人通过上述方式对经过预处理后的图像自动检测是否存在裂缝。
上述实施例提出的屏幕状态自动检测方法中，控制机器人分别移动到无人网点区的各个业务设备的预设区域内，若机器人移动到一个特务设备的预设区域内，则向待检测业务设备发送预设图形码，以根据预设图形码在显示屏幕上显示的内容判断待检测业务设备是否发生了电路故障，若未发生电路故障，则控制待检测业务设备按照预设显示参数进行图形显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出待检测业务设备的显示屏幕是否发生了预设类型的异常，该方案不需要人工参与，通过机器人移动到对应区域对设备的电路故障以及屏幕显示状态进行自动检测。
此外，本申请实施例还提出一种计算机可读存储介质，所述计算机可读存储介质上存储有屏幕状态自动检测程序，所述屏幕状态自动检测程序可被一个或者多个处理器执行，以实现如下操作：
控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
进一步地，所述屏幕状态自动检测程序被处理器执行时还实现如下操作：
若所述待检测业务设备的显示屏幕发生了电路故障，则向预先确定的监控设备发送提示信息，提示所述待检测业务设备的显示屏幕发生了电路故障需要进行处理。
进一步地，所述屏幕状态自动检测程序被处理器执行时还实现如下操作：
按照预设移动导航路径控制所述机器人的移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，控制所述机器人按照所述移动导航路径继续移动，直到所有业务设备的显示屏幕均检测完毕；
或者，控制所述机器人进行随机移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，将所述待检测业务设备标记为障碍物，控制机器人进行规避移动，在规避移动完成后，控制所述机器人继续进行随机移动，直至所有业务设备均被标记为障碍物后结束移动，并将业务设备的障碍物标记清除。
进一步地，所述屏幕状态自动检测程序被处理器执行时还实现如下操作：
控制所述机器人与待检测的业务设备进行无线通信，以向所述待检测业务设备发送显示预设图形码的显示指令，其中，所述预设图像码中包含有预设信息；控制所述机器人对所述待检测业务设备基于所述显示指令在显示屏幕上显示的内容进行扫描分析；若从待检测的业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕没有发生电路故障；若没有从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕发生了电路故障。
进一步地，所述屏幕状态自动检测程序被处理器执行时还实现如下操作：
控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域；在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，以获取掩码图像；为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域；根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像；对所述最大显示图像进行滤噪增强预处理，获得增强图像；对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
进一步地，所述屏幕状态自动检测程序被处理器执行时还实现如下操作：
获取对应较小尺寸的显示区域的预设颜色的掩码图像的每条边界线上的多个点的坐标；分别基于所述各个边界线上的多个点的坐标，通过最小二乘法直线拟合算法获取所述掩码图像的每条边界线的延长线；根据所述掩码图像的每条边界线的延长线获取所述四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置；将所述四边形进行反透视变换为矩形，根据所述掩码图像矩形的位置获取所述抗干扰图像。
本申请计算机可读存储介质具体实施方式与上述屏幕状态自动检测机器人和方法各实施例基本相同，在此不作累述。
需要说明的是，上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。并且本文中的术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、装置、物品或者方法不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、装置、物品或者方法所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、装置、物品或者方法中还存在另外的相同要素。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种屏幕状态自动检测机器人，其特征在于，所述机器人包括存储器和处理器，所述存储器上存储有屏幕状态自动检测程序，所述屏幕状态自动检测程序被所述处理器执行时实现如下操作：
    控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
    若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
    若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  2. 根据权利要求1所述的屏幕状态自动检测机器人，其特征在于，所述处理器还用于执行所述屏幕状态自动检测程序，以在向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障的步骤之后，还实现如下步骤：
    若所述待检测业务设备的显示屏幕发生了电路故障，则向预先确定的监控设备发送提示信息，提示所述待检测业务设备的显示屏幕发生了电路故障需要进行处理。
  3. 根据权利要求1所述的屏幕状态自动检测机器人，其特征在于，所述控制机器人分别移动到无人网点区的各个业务设备的预设区域内包括：
    按照预设移动导航路径控制所述机器人移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，控制所述机器人按照所述移动导航路径继续移动，直到所有业务设备的显示屏幕均检测完毕；
    或者，控制所述机器人进行随机移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，将所述待检测业务设备标记为障碍物，控制机器人进行规避移动，在规避移动完成后，控制所述机器人继续进行随机移动，直至所有业务设备均被标记为障碍物后结束移动，并将业务设备的障碍物标记清除。
  4. 根据权利要求1所述的屏幕状态自动检测机器人，其特征在于，所述向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障的步骤包括：
    控制所述机器人与待检测的业务设备进行无线通信，以向所述待检测业务设备发送显示预设图形码的显示指令，其中，所述预设图像码中包含有预设信息；
    控制所述机器人对所述待检测业务设备基于所述显示指令在显示屏幕上显示的内容进行扫描分析；
    若从待检测的业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕没有发生电路故障；
    若没有从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕发生了电路故障。
  5. 根据权利要求1所述的屏幕状态自动检测机器人，其特征在于，所述控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常包括：
    控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域；
    在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取 掩码，以获取掩码图像；
    为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域；
    根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像；
    对所述最大显示图像进行滤噪增强预处理，获得增强图像；
    对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  6. 根据权利要求2所述的屏幕状态自动检测机器人，其特征在于，所述控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常的步骤包括：
    控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域；
    在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，以获取掩码图像；
    为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域；
    根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像；
    对所述最大显示图像进行滤噪增强预处理，获得增强图像；
    对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  7. 根据权利要求6所述的屏幕状态自动检测机器人，其特征在于，所述为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像的步骤包括：
    获取对应较小尺寸的显示区域的预设颜色的掩码图像的每条边界线上的多个点的坐标；
    分别基于所述各个边界线上的多个点的坐标，通过最小二乘法直线拟合算法获取所述掩码图像的每条边界线的延长线；
    根据所述掩码图像的每条边界线的延长线获取所述四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置；
    将所述四边形进行反透视变换为矩形，根据所述掩码图像矩形的位置获取所述抗干扰图像。
  8. 一种屏幕状态自动检测方法，其特征在于，所述方法包括：
    控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
    若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
    若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  9. 根据权利要求8所述的屏幕状态自动检测方法，其特征在于，向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障的步骤之后，所述方法还包括如下步骤：
    若所述待检测业务设备的显示屏幕发生了电路故障，则向预先确定的监控设备发送提示信息，提示所述待检测业务设备的显示屏幕发生了电路故障需要进行处理。
  10. 根据权利要求8所述的屏幕状态自动检测方法，其特征在于，所述控制机器人分别移动到无人网点区的各个业务设备的预设区域内的步骤包括：
    按照预设移动导航路径控制所述机器人移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，控制所述机器人按照所述移动导航路径继续移动，直到所有业务设备的显示屏幕均检测完毕；
    或者，控制所述机器人进行随机移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，将所述待检测业务设备标记为障碍物，控制机器人进行规避移动，在规避移动完成后，控制所述机器人继续进行随机移动，直至所有业务设备均被标记为障碍物后结束移动，并将业务设备的障碍物标记清除。
  11. 根据权利要求8所述的屏幕状态自动检测方法，其特征在于，所述向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障的步骤包括：
    控制所述机器人与待检测的业务设备进行无线通信，以向所述待检测业务设备发送显示预设图形码的显示指令，其中，所述预设图像码中包含有预设信息；
    控制所述机器人对所述待检测业务设备基于所述显示指令在显示屏幕上显示的内容进行扫描分析；
    若从待检测的业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕没有发生电路故障；
    若没有从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕发生了电路故障。
  12. 根据权利要求8所述的屏幕状态自动检测方法，其特征在于，所述控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常的步骤包括：
    控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域；
    在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，以获取掩码图像；
    为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域；
    根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像；
    对所述最大显示图像进行滤噪增强预处理，获得增强图像；
    对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  13. 根据权利要求9所述的屏幕状态自动检测方法，其特征在于，所述控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像按照预设分析规则 进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常的步骤包括：
    控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域；
    在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，以获取掩码图像；
    为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域；
    根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像；
    对所述最大显示图像进行滤噪增强预处理，获得增强图像；
    对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  14. 根据权利要求13所述的屏幕状态自动检测方法，其特征在于，所述为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像的步骤包括：
    获取对应较小尺寸的显示区域的预设颜色的掩码图像的每条边界线上的多个点的坐标；
    分别基于所述各个边界线上的多个点的坐标，通过最小二乘法直线拟合算法获取所述掩码图像的每条边界线的延长线；
    根据所述掩码图像的每条边界线的延长线获取所述四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置；
    将所述四边形进行反透视变换为矩形，根据所述掩码图像矩形的位置获取所述抗干扰图像。
  15. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有屏幕状态自动检测程序，所述屏幕状态自动检测程序可被一个或者多个处理器执行，以实现如下步骤：
    控制所述机器人分别移动到无人网点区的各个业务设备的预设区域内；
    若所述机器人移动到一个预设区域内，则将该预设区域对应的业务设备作为待检测业务设备，并向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障；
    若所述待检测业务设备的显示屏幕未发生电路故障，则控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  16. 根据权利要求15所述的计算机可读存储介质，其特征在于，所述屏幕状态自动检测程序还可被一个或者多个处理器执行，以在向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障的步骤之后，还实现如下步骤：
    若所述待检测业务设备的显示屏幕发生了电路故障，则向预先确定的监控设备发送提示信息，提示所述待检测业务设备的显示屏幕发生了电路故障需要进行处理。
  17. 根据权利要求15所述的计算机可读存储介质，其特征在于，所述控制机器人分别移动到无人网点区的各个业务设备的预设区域内的步骤包括：
    按照预设移动导航路径控制所述机器人移动，在移动到一个业务设备的预设区域内且 对所述待检测业务设备的显示屏幕检测完毕后，控制所述机器人按照所述移动导航路径继续移动，直到所有业务设备的显示屏幕均检测完毕；
    或者，控制所述机器人进行随机移动，在移动到一个业务设备的预设区域内且对所述待检测业务设备的显示屏幕检测完毕后，将所述待检测业务设备标记为障碍物，控制机器人进行规避移动，在规避移动完成后，控制所述机器人继续进行随机移动，直至所有业务设备均被标记为障碍物后结束移动，并将业务设备的障碍物标记清除。
  18. 根据权利要求15所述的计算机可读存储介质，其特征在于，所述向所述待检测业务设备发送预设图形码，以根据所述预设图形码在显示屏幕上显示的内容判断所述待检测业务设备是否发生了电路故障的步骤包括：
    控制所述机器人与待检测的业务设备进行无线通信，以向所述待检测业务设备发送显示预设图形码的显示指令，其中，所述预设图像码中包含有预设信息；
    控制所述机器人对所述待检测业务设备基于所述显示指令在显示屏幕上显示的内容进行扫描分析；
    若从待检测的业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕没有发生电路故障；
    若没有从待检测业务设备的显示屏幕上扫描分析出预设信息，则确定待检测业务设备的显示屏幕发生了电路故障。
  19. 根据权利要求15所述的计算机可读存储介质，其特征在于，所述控制所述待检测业务设备按照预设显示参数进行图像显示，并对显示屏幕显示的图像按照预设分析规则进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常的步骤包括：
    控制所述待检测业务设备的显示屏幕分别以纯色显示的方式显示多种预设颜色，其中，不同预设颜色对应不同尺寸的显示区域，各个预设颜色对应的显示区域的形状和所述显示屏幕的最大显示区域的形状对应，且其中一个所述预设颜色对应的显示区域为所述显示屏幕的最大显示区域；
    在以纯色显示的方式显示一种预设颜色时，针对该预设颜色对显示屏幕的显示图形取掩码，以获取掩码图像；
    为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像，其中，所述较小尺寸的显示区域为除最大显示区域之外的其它尺寸的显示区域；
    根据获取的抗干扰图像，确定出所述显示屏幕的最大显示区域，并根据确定的最大显示区域，为对应最大显示区域的预设颜色的取掩码图像进行实际显示区域的图像提取，提取出最大显示图像；
    对所述最大显示图像进行滤噪增强预处理，获得增强图像；
    对所述增强图像进行分析，以分析出所述待检测业务设备的显示屏幕是否发生了预设类型的异常。
  20. 根据权利要求19所述的计算机可读存储介质，其特征在于，所述为对应较小尺寸的显示区域的预设颜色的掩码图像进行抗干扰处理以获取抗干扰图像的步骤包括：
    获取对应较小尺寸的显示区域的预设颜色的掩码图像的每条边界线上的多个点的坐标；
    分别基于所述各个边界线上的多个点的坐标，通过最小二乘法直线拟合算法获取所述掩码图像的每条边界线的延长线；
    根据所述掩码图像的每条边界线的延长线获取所述四条边界线的交点，根据所述四条边界线的交点以及所述四条边界线确定出所述掩码图像四边形的位置；
    将所述四边形进行反透视变换为矩形，根据所述掩码图像矩形的位置获取所述抗干扰图像。
PCT/CN2017/108740 2017-08-29 2017-10-31 屏幕状态自动检测机器人、方法及计算机可读存储介质 WO2019041516A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
SG11202001417WA SG11202001417WA (en) 2017-08-29 2017-10-31 Automatic screen state detection robot, method and computer-readable storage medium
US16/632,859 US11361422B2 (en) 2017-08-29 2017-10-31 Automatic screen state detection robot, method and computer-readable storage medium
JP2019556812A JP6911149B2 (ja) 2017-08-29 2017-10-31 スクリーン状態自動検出ロボット、方法及びコンピュータ読み取り可能な記憶媒体

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710754580.7 2017-08-29
CN201710754580.7A CN107688345B (zh) 2017-08-29 2017-08-29 屏幕状态自动检测机器人、方法及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2019041516A1 true WO2019041516A1 (zh) 2019-03-07

Family

ID=61155532

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/108740 WO2019041516A1 (zh) 2017-08-29 2017-10-31 屏幕状态自动检测机器人、方法及计算机可读存储介质

Country Status (5)

Country Link
US (1) US11361422B2 (zh)
JP (1) JP6911149B2 (zh)
CN (1) CN107688345B (zh)
SG (1) SG11202001417WA (zh)
WO (1) WO2019041516A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111339298A (zh) * 2020-02-25 2020-06-26 北京松果电子有限公司 一种分类预测方法、装置及存储介质

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017156046A1 (en) 2016-03-07 2017-09-14 Hyla, Inc. Screen damage detection for devices
US11210777B2 (en) * 2016-04-28 2021-12-28 Blancco Technology Group IP Oy System and method for detection of mobile device fault conditions
US10876914B2 (en) 2016-04-28 2020-12-29 Blancco Technology Group IP Oy Systems and methods for detection of mobile device fault conditions
US12013346B2 (en) 2018-02-19 2024-06-18 Blancco Technology Group IP Oy System and method for detection of mobile device fault conditions
CN110340882A (zh) * 2018-04-08 2019-10-18 苏州宝时得电动工具有限公司 机器人避障方法、装置、计算机设备及存储介质
CN108921843B (zh) * 2018-07-05 2022-08-12 深圳市路畅科技股份有限公司 一种检测车载导航仪性能的系统及方法
WO2021055457A1 (en) 2019-09-16 2021-03-25 Assurant, Inc. System, method, apparatus, and computer program product for utilizing machine learning to process an image of a mobile device to determine a mobile device integrity status
US11580627B2 (en) 2020-01-06 2023-02-14 Assurant, Inc. Systems and methods for automatically grading pre-owned electronic devices
CN112069043A (zh) * 2020-08-04 2020-12-11 北京捷通华声科技股份有限公司 一种终端设备状态检测方法、模型生成方法以及装置
CN113221881B (zh) * 2021-04-30 2022-03-22 浙大城市学院 一种多层级的智能手机屏幕缺陷检测方法
CN113709289A (zh) * 2021-08-31 2021-11-26 展讯通信（上海）有限公司 终端测试的监控系统及方法
CN114973612A (zh) * 2022-03-28 2022-08-30 深圳市揽讯科技有限公司 Led显示屏故障自动告警监控系统及监控方法
CN115375645B (zh) * 2022-08-17 2024-08-16 武汉海微科技股份有限公司 车载显示屏显示异常的测试方法、装置、设备及存储介质
CN117078672B (zh) * 2023-10-13 2024-07-09 南京极景微半导体有限公司 基于计算机视觉的手机屏幕缺陷智能检测方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1765111A (zh) * 2004-01-23 2006-04-26 日本诺贝尔株式会社 设备检查装置、基于其上的设备检查系统以及蜂窝电话保持器
CN1871846A (zh) * 2003-10-21 2006-11-29 皇家飞利浦电子股份有限公司 屏幕图像验证
US20100053604A1 (en) * 2008-08-31 2010-03-04 Paul Rice Broken screen detector
CN101782456A (zh) * 2009-01-19 2010-07-21 深圳市新超亮特种显示设备有限公司 一种远程检测显示器工作状态的方法及系统
CN103838674A (zh) * 2014-03-21 2014-06-04 徐禄勇 基于数字图像的智能测试机器人及其使用方法

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5600574A (en) * 1994-05-13 1997-02-04 Minnesota Mining And Manufacturing Company Automated image quality control
JP2001142512A (ja) * 1999-11-16 2001-05-25 Mitsubishi Electric Corp ロボットの遠隔操作システム
DE10137056C5 (de) * 2001-07-28 2018-09-06 Rieter Ingolstadt Gmbh Verfahren zur Wartung einer Textilmaschine
KR101166843B1 (ko) * 2005-08-30 2012-07-19 엘지디스플레이 주식회사 Ｌｃｄ 검사 장비 및 ｌｃｄ 검사 방법
JP4703480B2 (ja) * 2006-05-17 2011-06-15 株式会社日立製作所 映像内の移動体検出方法、映像システムの異常発生原因分析支援方法及び支援システム
KR101546850B1 (ko) * 2008-06-27 2015-08-24 삼성전자주식회사 디스플레이장치의 화질평가방법 및 화질평가장치
US8643662B2 (en) * 2009-04-22 2014-02-04 Samsung Electronics Co., Ltd. Video entertainment picture quality adjustment
FR3012425B1 (fr) * 2013-10-24 2017-03-24 European Aeronautic Defence & Space Co Eads France Robot collaboratif d'inspection visuelle d'un aeronef
CN105336287B (zh) * 2014-08-06 2019-08-06 珠海格力电器股份有限公司 显示屏的测试方法、装置和系统
CN104318879A (zh) * 2014-10-20 2015-01-28 京东方科技集团股份有限公司 显示装置、显示装置故障分析系统和方法
US9955150B2 (en) * 2015-09-24 2018-04-24 Qualcomm Incorporated Testing of display subsystems
CN106406289A (zh) * 2016-11-17 2017-02-15 北京中科汇联科技股份有限公司 一种机器人故障检修系统及方法
CN106681333A (zh) * 2017-03-02 2017-05-17 刘伟豪 一种提高变电站巡检机器人稳定性的方法及系统
CN106960646A (zh) * 2017-03-30 2017-07-18 上海与德科技有限公司 一种屏幕故障检测方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1871846A (zh) * 2003-10-21 2006-11-29 皇家飞利浦电子股份有限公司 屏幕图像验证
CN1765111A (zh) * 2004-01-23 2006-04-26 日本诺贝尔株式会社 设备检查装置、基于其上的设备检查系统以及蜂窝电话保持器
US20100053604A1 (en) * 2008-08-31 2010-03-04 Paul Rice Broken screen detector
CN101782456A (zh) * 2009-01-19 2010-07-21 深圳市新超亮特种显示设备有限公司 一种远程检测显示器工作状态的方法及系统
CN103838674A (zh) * 2014-03-21 2014-06-04 徐禄勇 基于数字图像的智能测试机器人及其使用方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111339298A (zh) * 2020-02-25 2020-06-26 北京松果电子有限公司 一种分类预测方法、装置及存储介质
CN111339298B (zh) * 2020-02-25 2024-04-09 北京小米松果电子有限公司 一种分类预测方法、装置及存储介质

Also Published As

Publication number Publication date
SG11202001417WA (en) 2020-03-30
CN107688345A (zh) 2018-02-13
US11361422B2 (en) 2022-06-14
US20200167907A1 (en) 2020-05-28
JP6911149B2 (ja) 2021-07-28
CN107688345B (zh) 2019-04-19
JP2020517926A (ja) 2020-06-18

Similar Documents

Publication Publication Date Title
WO2019041516A1 (zh) 屏幕状态自动检测机器人、方法及计算机可读存储介质
CN110097542B (zh) 芯片气泡的检测方法、装置及存储介质
JP5699788B2 (ja) スクリーン領域検知方法及びシステム
CN102750555B (zh) 一种基于即时通讯工具的表情识别装置
CN110199317A (zh) 使用手持设备对木板的自动检测、计数和测量
US9946947B2 (en) System and method for finding saddle point-like structures in an image and determining information from the same
JP6197340B2 (ja) 画像処理装置、画像処理方法、およびプログラム
US11216905B2 (en) Automatic detection, counting, and measurement of lumber boards using a handheld device
CN116883661B (zh) 一种基于目标识别与图像处理的动火作业检测方法
CN101146236A (zh) 用以测试影像撷取装置的方法、相关的测试平台及影像卡
CN111259891A (zh) 一种自然场景下身份证识别方法、装置、设备和介质
CN108154506A (zh) 一种六角螺母自动检测方法
CN108665495A (zh) 图像处理方法及装置、移动终端
EP4171051A1 (en) Collation device, program, and collation method
US11825190B2 (en) Collation device
CN113139399B (zh) 一种图像线框识别方法及服务器
CN110458857A (zh) 中心对称图元检测方法、装置、电子设备及可读存储介质
KR101637977B1 (ko) 레이저 비전 시스템을 이용한 용접 조인트의 특징점 검출 방법
JP2023061880A (ja) 照合装置及びプログラム
CN115424062B (zh) 一种自动识别斜线网的方法及装置、设备及存储介质
JP7613093B2 (ja) 情報処理装置及び情報処理プログラム
CN110769221B (zh) 在拍摄图像中检测投影图像的角点的方法和投影仪
CN118368408B (zh) 图像采集设备的检测方法、设备和存储介质
CN116879292B (zh) 一种光触媒硅藻泥板的质量评估方法及装置
US20240153126A1 (en) Automatic image cropping using a reference feature

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17923772

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019556812

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24/09/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17923772

Country of ref document: EP

Kind code of ref document: A1