CN117392657A - Pointer instrument reading identification method, device, equipment and readable storage medium - Google Patents
Pointer instrument reading identification method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN117392657A CN117392657A CN202311433255.2A CN202311433255A CN117392657A CN 117392657 A CN117392657 A CN 117392657A CN 202311433255 A CN202311433255 A CN 202311433255A CN 117392657 A CN117392657 A CN 117392657A
- Authority
- CN
- China
- Prior art keywords
- pointer
- instrument
- image
- meter
- identified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/15—Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/19007—Matching; Proximity measures
- G06V30/19013—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19127—Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a pointer instrument reading identification method, a device, equipment and a readable storage medium, which are applied to the technical field of computers, and the method comprises the following steps: inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the instrument area image to be identified comprises an instrument pointer and the whole instrument area of the dial plate; determining an instrument pointer in an instrument area image to be identified by using a Hough straight line detection algorithm and based on the space distance; and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template. The method effectively utilizes the advantages of the traditional method and the deep learning algorithm, takes advantage of the advantages and takes the disadvantages, only needs to train one instrument instance to divide a network, and can meet the single pointer instrument identification work requirement in a new scene by simply presetting a template operation.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for identifying a pointer instrument reading.
Background
The usual pointer instrument reading method includes (1) manual reading, but the error is large and the working efficiency is low; (2) Traditional algorithm identification, such as subtraction method, hough transform detection method, template matching method and the like, wherein the traditional image processing methods are poor in robustness and are easily influenced by factors such as illumination, shooting angle, imaging quality and the like; (3) The deep learning algorithm, while robust, requires collection of training data that is sufficiently rich that it is difficult to apply if some meter type is present that has not been touched before.
Therefore, the pointer instrument identification method at the present stage has the defects of higher requirements on application scenes and difficult application.
Disclosure of Invention
In view of the above, the present invention aims to provide a method, a device and a readable storage medium for identifying the reading of a pointer instrument, which solve the problems of high requirements and difficult application of the reading application scene of the pointer instrument in the prior art.
In order to solve the technical problems, the invention provides a pointer instrument reading identification method, which comprises the following steps:
acquiring an image of a pointer instrument to be identified;
inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the to-be-identified instrument area image comprises an instrument pointer and the whole instrument area of the dial plate;
Determining the meter pointer in the meter area image to be identified by using a Hough straight line detection algorithm based on a space distance;
and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template.
Optionally, the determining the meter pointer in the meter area image to be identified by using a Hough straight line detection algorithm and based on a spatial distance includes:
carrying out gray processing on the instrument area image to be identified to obtain an instrument area gray image to be identified;
performing binarization threshold processing on the gray level image of the instrument area to be identified to obtain a binarization image of the instrument area to be identified;
performing linear detection on the binarized image of the instrument area to be identified by using the Hough linear detection algorithm to obtain all candidate instrument pointers;
and calculating weighted values of all candidate meter pointers based on the space distance and the length, and taking the candidate meter pointer corresponding to the maximum weighted value as the meter pointer.
Optionally, the calculating the weighted values of all the candidate meter pointers based on the spatial distance and the length, taking the candidate meter pointer corresponding to the maximum weighted value as the meter pointer includes:
D represents the Euclidean distance of the candidate instrument pointer from the pointer deflection center; (x) P ,y P ) Coordinates of candidate meter pointer centers; l represents the candidate meter pointer length; (x) Q ,y Q ) Representing coordinates of a candidate meter pointer deflection center; (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Representing coordinates of two ends of a candidate instrument pointer; alpha represents a weight coefficient of the distance; beta represents the weight coefficient of the length; l (L) max Representing the longest line; w represents a weight value.
Optionally, the determining, based on a preset pointer instrument standard template and using a SIFT image matching algorithm, a reading result corresponding to the instrument pointer includes:
matching the instrument pointer with the preset pointer instrument standard template by using the SIFT image matching algorithm to obtain a mapping matrix;
determining the coordinates of the instrument pointer in the preset pointer instrument standard template according to the mapping matrix;
and calculating the deflection angle of the instrument pointer according to the coordinates and the central position, the 0 scale position, the minimum scale position, the maximum scale position and the reading range marked on the basis of the standard template of the preset pointer instrument, and obtaining the reading result.
Optionally, the matching the meter pointer with the preset pointer meter standard template by using the SIFT image matching algorithm to obtain a mapping matrix includes:
Calculating key points of the instrument pointer and key points of the preset pointer instrument standard template by using a SIFT image algorithm and a FLANN quick matching algorithm, and performing key point matching;
and mapping the key points of the same position in the instrument pointer and the preset pointer instrument standard template to obtain the mapping matrix.
Optionally, the pre-trained instance segmentation model includes:
acquiring a data set, wherein the data set comprises various pointer instrument images under different instrument styles, different pointer shapes and different background environments; the pointer instrument image is an image marked with an instrument area;
performing size adjustment, data enhancement and normalization processing on various pointer instrument images in the data set to obtain processed images;
training a Yolat model by using the processed image, and determining the trained example segmentation model according to the loss value of the cross loss function.
Optionally, after the obtaining the image of the area of the instrument to be identified, the method further includes:
correcting the instrument area image to be identified by using the preset pointer instrument standard template to obtain a corrected instrument area image to be identified;
Correspondingly, the meter pointer in the meter area image to be identified is determined by using a Hough straight line detection algorithm and based on a space distance, and the method comprises the following steps of;
and determining the meter pointer in the corrected meter area image to be recognized by using a Hough straight line detection algorithm based on the space distance.
The invention also provides a pointer instrument reading identification device, which comprises:
the acquisition module is used for acquiring an image of the pointer instrument to be identified;
the instrument area identification module is used for inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the to-be-identified instrument area image comprises an instrument pointer and the whole instrument area of the dial plate;
the instrument pointer detection module is used for determining the instrument pointer in the instrument area image to be identified by utilizing a Hough straight line detection algorithm and based on a space distance;
and the reading determining module is used for determining a reading result corresponding to the instrument pointer based on a preset pointer instrument standard template and by using a SIFT image matching algorithm.
The invention also provides pointer instrument reading identification equipment, which comprises:
a memory for storing a computer program;
And the processor is used for realizing the pointer instrument reading identification method when executing the computer program.
The invention also provides a readable storage medium, wherein the readable storage medium stores computer executable instructions, and the computer executable instructions realize the pointer instrument reading identification method when being loaded and executed by a processor.
The method comprises the steps of obtaining an image of a pointer instrument to be identified; inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the instrument area image to be identified comprises an instrument pointer and the whole instrument area of the dial plate; determining an instrument pointer in an instrument area image to be identified by using a Hough straight line detection algorithm and based on the space distance; and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template. The method utilizes a deep learning algorithm to divide the instrument, combines a preset pointer instrument template, adopts a traditional digital image processing method to position the pointer of the instrument by utilizing a Hough straight line detection algorithm and a space distance in an instrument area image obtained based on an example division model, and obtains an instrument reading result. The advantages of the traditional method and the deep learning algorithm are effectively utilized, the advantages are overcome, the disadvantages are overcome, only one instrument instance segmentation network is required to be trained, and the single pointer instrument identification work requirement in a new scene can be met by simply presetting template operation. The method can solve the problem that the practical application is difficult due to the high requirement of the traditional algorithm on the image quality, and also can solve the problem that the deep learning algorithm has high requirement on training data when performing end-to-end prediction and is difficult to be rapidly applied to a new scene.
In addition, the invention also provides a pointer instrument reading identification device, equipment and a readable storage medium, which have the same beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a pointer instrument reading identification method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a pointer instrument reading identification device according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a pointer instrument reading identification device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Pointer meters are widely used in various industrial settings, such as in the fields of electric power systems, petrochemical systems, and railway systems. The reading of the value displayed by the instrument is important to judge the safety of the production environment, and currently, the main instrument reading means is by manual reading. However, manual reading is time-consuming and labor-consuming, and is prone to errors, and is difficult to continue to use manually to accomplish in areas where certain hazardous environments or hard to reach are encountered. In recent years, computer image processing and artificial intelligence technology are rapidly developed, and technical support is provided for the realization of intelligent reading of a pointer instrument. Such as: (1) The method based on traditional image processing mainly comprises a subtraction method, a Hough transform detection method, a template matching method and the like, but the methods have some defects, such as the subtraction method not only needs to acquire a zero scale image of an instrument to be identified in advance, but also needs to have no change in a background area to be identified, the Hough transform detection method only can detect a dial plate and a pointer with standard shapes, the template matching method needs to acquire a template image with single size, the template image and the background have higher contrast, the robustness of the traditional image processing method is easily influenced by factors such as illumination, shooting angle and imaging quality, and the application scene is limited. (2) Based on the machine learning method, various types of instruments in different scenes are positioned and identified after a large number of data sets are trained, the method has high robustness, the requirements on application scenes are low, the popularization is high, a large number of data sets and time are needed for training by the current mainstream deep learning algorithm, and the acquisition of the data sets is a relatively difficult task.
In summary, although the conventional method does not require a large amount of data training, the conventional method is easily affected by a complex environment in practical application; the deep learning algorithm is robust, but requires collection of training data that is sufficiently rich that it is difficult to apply if some meter type that has not been touched before occurs. The positions of the scale marks of the pointer instruments 0 of different types may be inconsistent, the display range is also inconsistent, the universality of the identification method is difficult to apply in a new scene, therefore, the invention provides a pointer instrument reading identification method, and particularly referring to fig. 1, fig. 1 is a flow chart of the pointer instrument reading identification method provided by the embodiment of the invention. The method may include:
s101: and acquiring an image of the pointer instrument to be identified.
The embodiment acquires an image of the pointer instrument to be recognized. It will be appreciated that the pointer meter image to be identified in this embodiment may be a conventional meter image; or may also be an unusual meter image; or the image quality may be poor.
S102: inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the meter area image to be identified includes the meter pointer and the entire meter area of the dial.
In the embodiment, the fact that the traditional algorithm is easy to be interfered in a complex scene and the instrument area cannot be accurately segmented from the background is considered, but the example segmentation model based on the deep learning has universality and accuracy, and therefore the instrument area is extracted by the deep learning. The present embodiment is not limited to the example segmentation model, as long as the target region can be identified and segmented. For example, a Yolat (an example segmentation model) model; or may also be a Yolo (an example segmentation model) model.
Further, for accuracy and efficiency of the instrument region extraction, the pre-trained example segmentation model may include the following steps:
step 21: acquiring a data set, wherein the data set comprises various pointer instrument images under different instrument styles, different pointer shapes and different background environments; and the pointer meter image is an image that marks the meter area.
Specifically, training a pointer instrument instance segmentation model, and constructing a pointer instrument instance segmentation data set. A large number of different models and types of pointer meter images are collected, ensuring dial styles, pointer shapes and context including various meters. And labeling each pointer instrument image for training to obtain a corresponding instance segmentation mask, wherein the mask is a binary image with the same size as the original image.
Step 22: and performing size adjustment, data enhancement and normalization processing on various pointer instrument images in the data set to obtain processed images.
Specifically, the image size is unified: all pointer instrument images used for training are adjusted to be of the same size, so that generalization capability of the model under different scales is facilitated, and a specific formula is as follows:
I resized ＝resize(I,(H fixed, W fixed ))；
wherein I represents an original image, I resized Representing resized images, H fixed Representing image height, W fixed Representing the image width.
Data enhancement: the data is enhanced by means of random overturn, rotation, scaling and the like, the data set is expanded, the robustness of the model is improved, and the specific formula is as follows:
wherein I is flipped Representing the data enhanced image; flip-horizontal (I) means performing operations such as horizontal flipping on an image; p represents the probability of performing this operation
Normalization: the pixel values of the images are normalized, the pixel values are mapped into the [0,1] range, and the stable training of the model is facilitated, and the specific formula is as follows:
wherein I is normalized (x, y) represents the image pixel value after normalization operation is performed; i (x, y) represents the original image pixel value; max (I) and min (I) represent the maximum and minimum values, respectively, of an image pixel.
The processed image can effectively improve the performance of the trained model, and the data normalization is beneficial to faster convergence of the model.
Step 23: and training the Yolat model by using the processed image, and determining a trained example segmentation model according to the loss value of the cross loss function.
Specifically, a Yolat model is used to train the pointer instrument instance segmentation network. Wherein for example segmentation tasks, the loss function used is the cross entropy loss at the pixel level. It compares the model predicted mask to the true mask, thereby facilitating model learning of accurate instances.
S103: and determining the meter pointer in the image of the meter area to be identified by using a Hough straight line detection algorithm based on the space distance.
And the traditional Hough (Hough transform) straight line detection algorithm is adopted for straight line detection, and each detected straight line is screened based on the space distance, so that the accuracy of instrument pointer detection is improved.
Further, in order to further improve accuracy of detecting the meter pointer, the determining the meter pointer in the image of the meter area to be identified by using the Hough line detection algorithm and based on the spatial distance may include the following steps:
Step 31: and carrying out gray processing on the instrument area image to be identified to obtain the instrument area gray image to be identified.
Specifically, the image is converted from RBG space to single-channel gray space, and the specific conversion process is as follows:
wherein w represents a weight matrix; w (w) ij Representing the weight from a color channel to a gray channel; i ij (x, y) represents the pixel value of channel i for the image at location (x, y).
Step 32: and carrying out binarization threshold processing on the gray level image of the instrument area to be identified to obtain a binarization image of the instrument area to be identified.
Specifically, binary thresholding is performed on the gray level image, and the binarization threshold of each pixel is determined according to the local image characteristics, so that the segmentation and feature extraction of the image are better realized, and the specific formula is as follows:
wherein I is binary (x, y) represents a binarized pixel value; i gary (x, y) represents the pixel value of the original gray scale image at the position (x, y); t (x, y) represents an adaptive threshold calculated from pixels in the local neighborhood, and if the gray scale pixel value at the position (x, y) is greater than the adaptive threshold T (x, y), the pixel value is set to 255 (representing white); otherwise, the pixel value is set to 0 (representing black). The process realizes the self-adaptive binarization processing of the image according to the gray value of the local pixel, so that the segmentation result of the image can be better adapted to the local illumination and the contrast variation. The adaptive threshold T (x, y) is calculated from the neighborhood of pixels around the pixel (x, y). This process may use gaussian weighted averaging or the like.
Specifically, the adaptive threshold may be calculated using the following formula:
T(x,y)＝mean(I gray (x,y))-constant·stddev(I gray (x,y))；
wherein mean (I gray (x, y)) represents the average gray value of the local neighborhood; stddev (I) gray (x, y)) represents the gray value standard deviation of the local neighborhood; constant is a constant term used to adjust the thresholdIs of a size of (a) and (b).
Step 33: and carrying out linear detection on the binarized image of the instrument area to be identified by using a Hough linear detection algorithm to obtain all candidate instrument pointers.
Specifically, a Hough straight line detection algorithm is applied to the obtained binary image to obtain the coordinates of the starting position and the ending position of each line segment, so that all candidate instrument pointers are obtained.
Step 34: and calculating weighted values of all candidate meter pointers based on the space distance and the length, and taking the candidate meter pointer corresponding to the maximum weighted value as the meter pointer.
Specifically, in this embodiment, a certain error exists in consideration that only using the Hough line detection algorithm will often detect a plurality of pieces of noise data. Therefore, the method and the device screen all candidate instrument pointers obtained based on the Hough line detection algorithm in the two dimensions of the distance and the length, and improve the accuracy of instrument pointer identification.
Further, in order to further improve accuracy of meter pointer identification, the calculating the weighted values of all candidate meter pointers based on the spatial distance and the length, and using the candidate meter pointer corresponding to the maximum weighted value as the meter pointer may include the following steps:
Wherein D represents the Euclidean distance of the candidate instrument pointer from the pointer deflection center; l represents the candidate meter pointer length; (x) P ,y P ) Coordinates of candidate meter pointer centers; (x) Q ,y Q ) Representing coordinates of a candidate meter pointer deflection center;(x 1 ,y 1 ) And (x) 2 ,y 2 ) Representing coordinates of two ends of a candidate instrument pointer; alpha represents a weight coefficient of the distance; beta represents the weight coefficient of the length; l (L) max Representing the desired length; w represents a weight value.
The distance weight coefficient, the length weight coefficient, and the desired length in this embodiment are empirically set, and the closer the candidate meter pointer is to the center, the higher the weight, the longer the candidate meter pointer length, and the higher the weight. For all candidate meter pointers, calculating the weight score of each pointer, taking the candidate meter pointer with the highest weight as a target pointer for calculating meter reading, and meanwhile, the method can also be used for distinguishing the pointer from the scale mark, so that the identification precision of the pointer is improved.
S104: and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template.
In this embodiment, a Scale-invariant feature transform (Scale-invariant feature transform) image matching algorithm is used to match the meter pointer with a preset pointer meter standard template, so as to determine a reading result corresponding to the meter pointer. The present embodiment is not limited to a specific matching process.
Further, in order to improve the accuracy of the reading, the determining, by using a standard template based on a preset pointer instrument and using a SIFT image matching algorithm, the reading result corresponding to the pointer of the instrument may include the following steps:
step 41: matching the instrument pointer with a preset pointer instrument standard template by using a SIFT image matching algorithm to obtain a mapping matrix;
step 42: determining the coordinates of the instrument pointer in a preset pointer instrument standard template according to the mapping matrix;
step 43: and calculating the deflection angle of the instrument pointer according to the coordinates and the central position, the 0 scale position, the minimum scale position, the maximum scale position and the reading range marked on the basis of the standard template of the instrument with the preset pointer, and obtaining a reading result.
In the embodiment, the manually acquired standard instrument image is acquired, the acquisition process ensures that the instrument plane is perpendicular to the shooting angle, and the instrument is a common instrument. And (3) using a trained instrument instance segmentation model to segment instrument areas from the background and filter irrelevant background contents, wherein a specific filtering formula is as follows:
I filtered (x,y)＝I(x,y)·M(x,y)；
wherein M is a binary mask map generated by the deep learning model; m (x, y) =1 means that the pixel (x, y) belongs to the meter region; m (x, y) =0 means that the pixel (x, y) belongs to the background. And I (x, y) represents the original meter image pixel. The purpose of this step is to remove extraneous content in the image, i.e. background content other than the meter area.
Storing the filtered standard image into a system, marking a pointer center point, a 0 scale position, a minimum scale position and a maximum scale position on the image, and inputting the standard image into an instrument display range to obtain a preset pointer instrument standard template.
The standard template of the preset pointer instrument marked with various information is obtained, then a SIFT image matching algorithm is used for matching the standard template of the preset pointer instrument with the pointer image of the instrument to obtain a mapping matrix, and the coordinates of the pointer position in the image to be identified in the standard template of the preset pointer instrument are obtained through the mapping matrix; based on the central position, the 0 scale position, the minimum scale position, the maximum scale position and the reading range of a preset pointer instrument standard template and the position of the pointer in the image to be identified mapped into the template, calculating the deflection angle of the pointer, and obtaining the identification result.
Further, in order to accelerate the matching efficiency, the matching between the instrument pointer and the preset pointer instrument standard template by using the SIFT image matching algorithm to obtain a mapping matrix may include the following steps:
step 51: calculating key points of an instrument pointer and key points of a preset pointer instrument standard template by using a SIFT image algorithm and a FLANN quick matching algorithm, and performing key point matching;
Step 52: and mapping the key points of the same position in the instrument pointer and the preset pointer instrument standard template to obtain a mapping matrix.
In the embodiment, the SIFT algorithm automatically calculates key points in the instrument pointer image and the preset pointer instrument standard template, and in the key point matching, a FLANN algorithm (Fast Library for Approximate Nearest Neighbors, a fast nearest neighbor approximation search function library) is used, and the algorithm is a nearest neighbor search algorithm in a fast mode, and the other image and the nearest key point are searched according to the given key point, so that the fast matching of the key points is completed. Mapping the same position key points in the two images, deriving a mapping matrix, and mapping any position coordinates in the instrument pointer image to coordinates in a preset pointer instrument standard template by means of the mapping matrix. Specifically, the mapping matrix may be obtained using the RANSAC algorithm (Random Sample Consensus ).
Further, in order to further improve accuracy of the instrument area segmentation, after the obtaining of the image of the instrument area to be identified, the method may further include the following steps:
correcting the instrument area image to be identified by using a preset pointer instrument standard template to obtain a corrected instrument area image to be identified;
Correspondingly, determining an instrument pointer in an instrument area image to be identified by using a Hough straight line detection algorithm based on the space distance, wherein the method comprises the steps of;
and determining the meter pointer in the corrected meter area image to be recognized by using a Hough straight line detection algorithm based on the space distance.
According to the embodiment, after the image of the instrument area is extracted through deep learning, the correction can be performed by utilizing the preset pointer instrument standard template based on the image, so that the instrument area is more refined and accurate.
The embodiment can also introduce a processing mechanism based on event triggering, trigger image processing and analysis when necessary, and avoid unnecessary calculation overhead. The real-time and high-efficiency reading identification of the pointer instrument panel is realized under the minimum delay of the system.
By applying the pointer instrument reading identification method provided by the embodiment of the invention, the pointer instrument image to be identified is obtained; inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the instrument area image to be identified comprises an instrument pointer and the whole instrument area of the dial plate; determining an instrument pointer in an instrument area image to be identified by using a Hough straight line detection algorithm and based on the space distance; and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template. The method utilizes a deep learning algorithm to divide the instrument, combines a preset pointer instrument template, adopts a traditional digital image processing method to position the pointer of the instrument by utilizing a Hough straight line detection algorithm and a space distance in an instrument area image obtained based on an example division model, and obtains an instrument reading result. The advantages of the traditional method and the deep learning algorithm are effectively utilized, the advantages are overcome, the disadvantages are overcome, only one instrument instance segmentation network is required to be trained, and the single pointer instrument identification work requirement in a new scene can be met by simply presetting template operation. The method can solve the problem that the practical application is difficult due to the high requirement of the traditional algorithm on the image quality, and also can solve the problem that the deep learning algorithm has high requirement on training data when performing end-to-end prediction and is difficult to be rapidly applied to a new scene. In addition, a large number of training images are preprocessed and pre-labeled, a Yolat instance segmentation model is trained, and accuracy and high efficiency of instrument area extraction are improved; in addition, the binarization threshold processing is carried out by utilizing the self-adaptive threshold method, so that the accuracy of the pointer detection of the instrument is further improved; in addition, candidate meter pointers are screened based on the space distance and the length, a target pointer is determined, and the accuracy of meter pointer identification is improved; the SIFT algorithm is utilized to obtain the coordinates of the instrument pointer in a preset pointer instrument standard template marked with various information; moreover, the FLANN rapid matching algorithm is utilized for matching the two images, so that the matching efficiency is improved; in addition, the instrument image obtained by deep learning is further corrected, and the accuracy of instrument region segmentation is improved; and, the time trigger mechanism is added, so that the cost can be reduced.
The pointer instrument reading identification device provided by the embodiment of the invention is introduced below, and the pointer instrument reading identification device described below and the pointer instrument reading identification method described above can be correspondingly referred to each other.
Referring to fig. 2 specifically, fig. 2 is a schematic structural diagram of a pointer instrument reading identification device according to an embodiment of the present invention, which may include:
an acquisition module 100, configured to acquire an image of a pointer instrument to be identified;
the instrument area identification module 200 is configured to input the pointer instrument image to be identified into a pre-trained example segmentation model, so as to obtain an instrument area image to be identified; the to-be-identified instrument area image comprises an instrument pointer and the whole instrument area of the dial plate;
the meter pointer detection module 300 is configured to determine the meter pointer in the to-be-identified meter area image by using a Hough straight line detection algorithm and based on a spatial distance;
the reading determining module 400 is configured to determine a reading result corresponding to the meter pointer based on a preset pointer meter standard template and by using a SIFT image matching algorithm.
Based on the above embodiment, the meter pointer detection module 300 may include:
the gray processing unit is used for carrying out gray processing on the instrument area image to be identified to obtain an instrument area gray image to be identified;
The binarization unit is used for carrying out binarization threshold processing on the gray level image of the instrument area to be identified to obtain a binarization image of the instrument area to be identified;
the straight line detection unit is used for carrying out straight line detection on the binarized image of the instrument area to be identified by utilizing the Hough straight line detection algorithm to obtain all candidate instrument pointers;
and the meter pointer determining unit is used for calculating weighted values of all candidate meter pointers based on the space distance and the length, and taking the candidate meter pointer corresponding to the maximum weighted value as the meter pointer.
Based on the above embodiment, the meter pointer determining unit may specifically include:
d represents the Euclidean distance of the candidate instrument pointer from the pointer deflection center; (x) P ,y P ) Coordinates of candidate meter pointer centers; l represents the candidate meter pointer length; (x) Q ,y Q ) Representing coordinates of a candidate meter pointer deflection center; (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Representing coordinates of two ends of a candidate instrument pointer; alpha represents a weight coefficient of the distance; beta represents the weight coefficient of the length; l (L) max Representing the longest line; w represents a weight value.
Based on any of the above embodiments, wherein the reading determination module 400 may include:
the matching unit is used for matching the instrument pointer with the preset pointer instrument standard template by using the SIFT image matching algorithm to obtain a mapping matrix;
The coordinate determining unit is used for determining the coordinate of the instrument pointer in the preset pointer instrument standard template according to the mapping matrix;
and the reading result determining unit is used for calculating the deflection angle of the instrument pointer according to the coordinates and the central position, the 0 scale position, the minimum scale position, the maximum scale position and the reading range marked on the basis of the standard template of the preset pointer instrument, and obtaining the reading result.
Based on the above embodiment, the matching unit may include:
the key point calculating subunit is used for calculating the key points of the instrument pointer and the key points of the preset pointer instrument standard template by using a SIFT image algorithm and a FLANN quick matching algorithm and carrying out key point matching;
and the matching subunit is used for mapping the key points of the same position in the instrument pointer and the preset pointer instrument standard template to obtain the mapping matrix.
Based on the above embodiment, the pre-trained instance segmentation model in the meter region identification module 200 may include:
a data set acquisition unit configured to acquire a data set including various pointer meter images in different meter styles, different pointer shapes, and different background environments; the pointer instrument image is an image marked with an instrument area;
The processing unit is used for carrying out size adjustment, data enhancement and normalization on various pointer instrument images in the data set to obtain processed images;
and the training unit is used for training the Yolat model by using the processed image and determining the trained example segmentation model according to the loss value of the cross loss function.
Based on the above embodiment, the pointer meter reading identification device may further include:
the correction module is used for correcting the instrument area image to be identified by utilizing the preset pointer instrument standard template after the instrument area image to be identified is obtained, so as to obtain a corrected instrument area image to be identified;
correspondingly, the meter pointer detection module 300 may specifically include; and determining the meter pointer in the corrected meter area image to be recognized by using a Hough straight line detection algorithm based on the space distance.
It should be noted that the modules and units in the pointer meter reading identification device can be changed in sequence without affecting the logic.
The pointer instrument reading identification device provided by the embodiment of the invention is used for acquiring the pointer instrument image to be identified through the acquisition module 100; the instrument area identification module 200 is used for inputting an instrument image of a pointer to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the instrument area image to be identified comprises an instrument pointer and the whole instrument area of the dial plate; the meter pointer detection module 300 is configured to determine a meter pointer in an image of a meter area to be identified by using a Hough line detection algorithm and based on a spatial distance; the reading determining module 400 is configured to determine a reading result corresponding to the meter pointer based on a preset pointer meter standard template and by using a SIFT image matching algorithm. The device utilizes a deep learning algorithm to divide the instrument, combines a preset pointer instrument template, adopts a traditional digital image processing method to position the pointer of the instrument by utilizing a Hough straight line detection algorithm and a space distance in an instrument area image obtained based on an example division model, and obtains an instrument reading result. The advantages of the traditional method and the deep learning algorithm are effectively utilized, the advantages are overcome, the disadvantages are overcome, only one instrument instance segmentation network is required to be trained, and the single pointer instrument identification work requirement in a new scene can be met by simply presetting template operation. The method can solve the problem that the practical application is difficult due to the high requirement of the traditional algorithm on the image quality, and also can solve the problem that the deep learning algorithm has high requirement on training data when performing end-to-end prediction and is difficult to be rapidly applied to a new scene. In addition, a large number of training images are preprocessed and pre-labeled, a Yolat instance segmentation model is trained, and accuracy and high efficiency of instrument area extraction are improved; in addition, the binarization threshold processing is carried out by utilizing the self-adaptive threshold method, so that the accuracy of the pointer detection of the instrument is further improved; in addition, candidate meter pointers are screened based on the space distance and the length, a target pointer is determined, and the accuracy of meter pointer identification is improved; the coordinates of the instrument pointer on the standard template of the preset pointer instrument marked with various information are obtained by using the SIFT algorithm, so that the accuracy of reading is improved; moreover, the FLANN rapid matching algorithm is utilized for matching the two images, so that the matching efficiency is improved; in addition, the instrument image obtained by deep learning is further corrected, and the accuracy of instrument region segmentation is improved; and, the time trigger mechanism is added, so that the cost can be reduced.
The pointer instrument reading identification device provided by the embodiment of the invention is introduced below, and the pointer instrument reading identification device and the pointer instrument reading identification method described above can be correspondingly referred to each other.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a pointer instrument reading identification device according to an embodiment of the present invention, which may include:
a memory 10 for storing a computer program;
a processor 20 for executing a computer program to implement the pointer meter reading identification method described above.
The memory 10, the processor 20, and the communication interface 31 all communicate with each other via a communication bus 32.
In the embodiment of the present invention, the memory 10 is used for storing one or more programs, the programs may include program codes, the program codes include computer operation instructions, and in the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:
acquiring an image of a pointer instrument to be identified;
inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the instrument area image to be identified comprises an instrument pointer and the whole instrument area of the dial plate;
Determining an instrument pointer in an instrument area image to be identified by using a Hough straight line detection algorithm and based on the space distance;
and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template.
In one possible implementation, the memory 10 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, and at least one application program required for functions, etc.; the storage data area may store data created during use.
In addition, memory 10 may include read only memory and random access memory and provide instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, where the operating instructions may include various operating instructions for performing various operations. The operating system may include various system programs for implementing various basic tasks as well as handling hardware-based tasks.
The processor 20 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, a fpga or other programmable logic device, and the processor 20 may be a microprocessor or any conventional processor. The processor 20 may call a program stored in the memory 10.
The communication interface 31 may be an interface of a communication module for connecting with other devices or systems.
Of course, it should be noted that the configuration shown in fig. 3 is not limiting of the pointer meter reading identification device in the embodiment of the present invention, and the pointer meter reading identification device may include more or less components than those shown in fig. 3, or may be combined with some components in practical applications.
The following describes a readable storage medium provided in an embodiment of the present invention, where the readable storage medium described below and the pointer meter reading identification method described above may be referred to correspondingly.
The invention also provides a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program realizes the steps of the pointer instrument reading identification method when being executed by a processor.
The readable storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Finally, it is further noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The above detailed description of the method, the device, the equipment and the readable storage medium for identifying the pointer instrument readings provided by the invention applies specific examples to illustrate the principle and the implementation of the invention, and the description of the examples is only used for helping to understand the method and the core idea of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (10)
1. A method for identifying readings of a pointer meter, comprising:
acquiring an image of a pointer instrument to be identified;
inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the to-be-identified instrument area image comprises an instrument pointer and the whole instrument area of the dial plate;
determining the meter pointer in the meter area image to be identified by using a Hough straight line detection algorithm based on a space distance;
and determining a reading result corresponding to the instrument pointer by using a SIFT image matching algorithm based on a preset pointer instrument standard template.
2. The method of claim 1, wherein said determining said meter pointer in said meter area image to be identified using a Hough line detection algorithm and based on spatial distance comprises:
carrying out gray processing on the instrument area image to be identified to obtain an instrument area gray image to be identified;
performing binarization threshold processing on the gray level image of the instrument area to be identified to obtain a binarization image of the instrument area to be identified;
performing linear detection on the binarized image of the instrument area to be identified by using the Hough linear detection algorithm to obtain all candidate instrument pointers;
and calculating weighted values of all candidate meter pointers based on the space distance and the length, and taking the candidate meter pointer corresponding to the maximum weighted value as the meter pointer.
3. The method for identifying meter reading of pointer according to claim 2, wherein calculating the weighted values of all the candidate meter pointers based on the spatial distance and the length, and taking the candidate meter pointer corresponding to the maximum weighted value as the meter pointer comprises:
d represents the Euclidean distance of the candidate instrument pointer from the pointer deflection center; (x) P ,y P ) Coordinates of candidate meter pointer centers; l represents the candidate meter pointer length; (x) Q ,y Q ) Representing coordinates of a candidate meter pointer deflection center; (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Representing coordinates of two ends of a candidate instrument pointer; alpha represents a weight coefficient of the distance; beta represents the weight coefficient of the length; l (L) max Representing the longest line; w represents a weight value.
4. A pointer meter reading identification method according to any one of claims 1 to 3, wherein said determining a reading result corresponding to said meter pointer based on a preset pointer meter standard template and using SIFT image matching algorithm comprises:
matching the instrument pointer with the preset pointer instrument standard template by using the SIFT image matching algorithm to obtain a mapping matrix;
determining the coordinates of the instrument pointer in the preset pointer instrument standard template according to the mapping matrix;
and calculating the deflection angle of the instrument pointer according to the coordinates and the central position, the 0 scale position, the minimum scale position, the maximum scale position and the reading range marked on the basis of the standard template of the preset pointer instrument, and obtaining the reading result.
5. The method for identifying a meter reading of a pointer according to claim 4, wherein the matching the meter pointer with the standard template of the meter of the preset pointer by using the SIFT image matching algorithm to obtain a mapping matrix comprises:
Calculating key points of the instrument pointer and key points of the preset pointer instrument standard template by using a SIFT image algorithm and a FLANN quick matching algorithm, and performing key point matching;
and mapping the key points of the same position in the instrument pointer and the preset pointer instrument standard template to obtain the mapping matrix.
6. The pointer meter reading identification method of claim 1 wherein said pre-trained instance segmentation model comprises:
acquiring a data set, wherein the data set comprises various pointer instrument images under different instrument styles, different pointer shapes and different background environments; the pointer instrument image is an image marked with an instrument area;
performing size adjustment, data enhancement and normalization processing on various pointer instrument images in the data set to obtain processed images;
training a Yolat model by using the processed image, and determining the trained example segmentation model according to the loss value of the cross loss function.
7. The pointer meter reading identification method of claim 1 further comprising, after said obtaining the image of the meter area to be identified:
Correcting the instrument area image to be identified by using the preset pointer instrument standard template to obtain a corrected instrument area image to be identified;
correspondingly, the meter pointer in the meter area image to be identified is determined by using a Hough straight line detection algorithm and based on a space distance, and the method comprises the following steps of;
and determining the meter pointer in the corrected meter area image to be recognized by using a Hough straight line detection algorithm based on the space distance.
8. A pointer meter reading identification device, comprising:
the acquisition module is used for acquiring an image of the pointer instrument to be identified;
the instrument area identification module is used for inputting the pointer instrument image to be identified into a pre-trained example segmentation model to obtain an instrument area image to be identified; the to-be-identified instrument area image comprises an instrument pointer and the whole instrument area of the dial plate;
the instrument pointer detection module is used for determining the instrument pointer in the instrument area image to be identified by utilizing a Hough straight line detection algorithm and based on a space distance;
and the reading determining module is used for determining a reading result corresponding to the instrument pointer based on a preset pointer instrument standard template and by using a SIFT image matching algorithm.
9. A pointer meter reading identification device, comprising:
a memory for storing a computer program;
a processor for implementing the pointer meter reading identification method of any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium having stored therein computer executable instructions which when loaded and executed by a processor implement the pointer meter reading identification method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311433255.2A CN117392657A (en) | 2023-10-31 | 2023-10-31 | Pointer instrument reading identification method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311433255.2A CN117392657A (en) | 2023-10-31 | 2023-10-31 | Pointer instrument reading identification method, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117392657A true CN117392657A (en) | 2024-01-12 |
Family
ID=89471721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311433255.2A Pending CN117392657A (en) | 2023-10-31 | 2023-10-31 | Pointer instrument reading identification method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117392657A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118918573A (en) * | 2024-09-13 | 2024-11-08 | 北京尚优力达科技有限公司 | Pointer instrument reading method based on image recognition and inspection equipment |
| CN119625749A (en) * | 2024-10-31 | 2025-03-14 | 广州像素数据技术股份有限公司 | A pointer-type electrical instrument reading method and device for grading physical, chemical and biological experiments |
| CN119992556A (en) * | 2024-12-10 | 2025-05-13 | 北京博维仕科技股份有限公司 | Video image-based instrument intelligent analysis method and system |
-
2023
- 2023-10-31 CN CN202311433255.2A patent/CN117392657A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118918573A (en) * | 2024-09-13 | 2024-11-08 | 北京尚优力达科技有限公司 | Pointer instrument reading method based on image recognition and inspection equipment |
| CN119625749A (en) * | 2024-10-31 | 2025-03-14 | 广州像素数据技术股份有限公司 | A pointer-type electrical instrument reading method and device for grading physical, chemical and biological experiments |
| CN119992556A (en) * | 2024-12-10 | 2025-05-13 | 北京博维仕科技股份有限公司 | Video image-based instrument intelligent analysis method and system |
| CN119992556B (en) * | 2024-12-10 | 2026-01-20 | 北京博维仕科技股份有限公司 | Intelligent Analysis Method and System for Instruments Based on Video Images |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993166B (en) | Automatic identification method of pointer meter reading based on scale search | |
| CN106960208B (en) | Method and system for automatically segmenting and identifying instrument liquid crystal number | |
| CN111950330B (en) | A method for detecting pointer-type instrument numbers based on target detection | |
| CN117392657A (en) | Pointer instrument reading identification method, device, equipment and readable storage medium | |
| CN111488874A (en) | A kind of pointer instrument tilt correction method and system | |
| CN111611643A (en) | Method, device, electronic device and storage medium for obtaining household type vectorized data | |
| CN111476109A (en) | Bill processing method, bill processing apparatus, and computer-readable storage medium | |
| CN108549843A (en) | A kind of VAT invoice recognition methods based on image procossing | |
| CN114399677B (en) | A pointer instrument recognition method based on text area reading | |
| CN113642582B (en) | Meter reading recognition method, device, electronic device and storage medium | |
| CN111353502B (en) | Digital table identification method and device and electronic equipment | |
| CN110569774A (en) | An Automatic Digitization Method of Line Chart Image Based on Image Processing and Pattern Recognition | |
| CN114782940B (en) | Automatic reading method of pointer instrument, electronic equipment and storage medium | |
| CN110569845A (en) | A test paper image correction method and related device | |
| KR101842535B1 (en) | Method for the optical detection of symbols | |
| CN104966047A (en) | Method and device for identifying vehicle license | |
| CN110659637A (en) | Electric energy meter number and label automatic identification method combining deep neural network and SIFT features | |
| CN114005108A (en) | Pointer instrument degree identification method based on coordinate transformation | |
| CN118196097B (en) | Intelligent detection method for crack defect of reinforcing sheet based on image processing | |
| CN114863086A (en) | A multi-dial and multi-pointer identification method for substation based on template matching | |
| CN114092542A (en) | A two-dimensional vision-based bolt measuring method and system | |
| CN109784257B (en) | Transformer thermometer detection and identification method | |
| CN114267044B (en) | Data recognition method and device for digital water meter | |
| CN118097680A (en) | A YOLO-based instrument reading method and system | |
| CN114663681B (en) | Method for reading pointer type instrument and related product thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |