CN116363658A - Digital display signal recognition method, system, device and medium based on deep learning - Google Patents

Abstract

The disclosure relates to a deep learning-based digital display signal recognition method, system, device, and medium. The method includes the following steps: acquiring an image containing a multi-channel non-dispersive infrared methane sensor digital display signal; implementing the image by using a flood filling method Segment and mark the marked area containing the digital display signal; establish the color distribution range of the digital display signal in the marked area based on the H channel of the HSV model, and obtain the numerical area image; input the numerical area image into the prediction model for matching The value is obtained to realize the accurate identification of the digital display signals of multiple non-dispersive infrared methane sensors. Without human participation, it ensures the control of the changes in the sensor's indication value and realizes the whole process of monitoring. Strong adaptability and high stability.

Description

Digital display signal recognition method, system, device and medium based on deep learning

technical field

The present disclosure relates to the field of image signal recognition, in particular to a deep learning-based digital display signal recognition method, system, device and medium.

Background technique

In coal mine environment, gas explosion has always been the biggest threat to the life safety of underground workers. Most of the causes of gas explosions are disasters caused by high gas concentration in underground mines. Therefore, controlling gas concentration is one of the effective measures to avoid gas explosion accidents. In order to accurately detect the methane concentration in the well, the sensor used to detect the methane gas concentration must not only have a high enough accuracy, but also ensure the reliability of the measurement results through the verification and calibration of the instrument and equipment. The verification and calibration of the instrument and equipment are to ensure the measurement results Accurate primary means, so the verification of these sensors has become a basic and essential work.

In traditional methane sensor numerical signal image recognition, only a single methane sensor numerical signal can be identified, and it is impossible to effectively distinguish the methane sensor numerical signal whose quantity and location are unknown. In the traditional value extraction of methane sensor, it is based on RGB color space, and the image is binarized through the threshold value, which has problems such as large noise, low robustness, inaccurate recognition, and susceptibility to interference. In the traditional value matching of methane sensor, first grayscale and binarize the digital tube, change the number to 255, and change the background to 0, and then use the threading method to thread the seven areas of abcdefg in turn to determine whether there is a value of 255 Value, if there is, it means that the area is highlighted, and finally combined with the highlight information of the seven areas, the value is judged comprehensively, and for some digital tubes with low overall brightness, the digital information will be lost after graying.

At present, there are mainly two display methods of digital display instruments: digital tube and liquid crystal. The commonly used methane sensors are mostly non-dispersive infrared methane sensors, which mainly use digital tube display. When the methane sensor automatic verification system collects sensor numerical images, due to the light reflection of the sensor itself and the attachment on the display panel, the quality of the collected numerical images is poor, which makes character recognition difficult and reduces the recognition accuracy. The image recognition algorithm based on deep learning can well solve the problem that traditional image processing methods are sensitive to image noise and have poor robustness.

Contents of the invention

Herein, the present disclosure provides a digital display signal recognition method, system, device and medium based on deep learning, which can solve at least one of the problems mentioned in the background technology. In order to solve the above technical problems, the present disclosure provides the following technical solutions:

S1. Obtain an image containing digital display signals of multiple non-dispersive infrared methane sensors;

S2. Using the flood filling method to realize the segmentation of the image and mark the marked area containing the digital display signal;

Adopting flood filling method to realize the segmentation of the image also includes the following steps:

performing binary processing on the image to obtain a grayscale image, and performing Gaussian filtering on the image to obtain a blurred image;

Acquiring the change of adjacent pixels of the grayscale image and the change of adjacent pixels of the blurred image;

According to the changes of the adjacent pixels of the grayscale image and the changes of the adjacent pixels of the blurred image, the pixel values of the grayscale image and the blurred image are normalized. If the high-frequency component of the high-frequency image has almost no change, it will be judged as a completely blurred image and will not be processed. If the high-frequency component changes significantly, it will be judged as a partially blurred image. If the high-frequency component changes very much, it will be Judged as a clear image. After comparison and analysis, normalization processing is performed; partially blurred images, clear images and completely blurred images are obtained;

For clear images, the segmentation process of the next step will be directly carried out. For some blurred images, it will be re-judged whether the image is a clear image after deblurring processing. If so, the next step of segmentation processing will be performed. For completely blurred images, no processing .

The deblurring process includes: performing pixel grayscale statistics on part of the blurred image to obtain the pixel probability distribution of the grayscale image; obtaining the cumulative distribution function of the image to obtain the transformed deblurred image; degree statistics, calculate the pixel probability distribution of the original image, obtain the cumulative distribution function of the image from the pixel probability distribution, and obtain the transformed defuzzified image according to the mapping function, and the mapping function is as follows:

k= 0,1,2,3,···L-1

k= 0,1,2,3,···L-1

是当前灰度级的像素个数，L是图像中可能的灰度级总数。where n is the sum of pixels in the image,

is the number of pixels in the current gray level, and L is the total number of possible gray levels in the image.

The segmentation processing of the next link includes:

Use the mask matrix to mark the flood filled area;

Use a single-channel image whose size is 2 pixels wider and higher than the input image as a mask matrix, and mark the flood-filled area by filling the pixel values of the pixels in the mask matrix (the flood-filled area is more The display area of the digital display signal of the methane sensor on the image)

Obtain the upper and lower bound values of the seed point area conditions, where the upper and lower bound values include: upper bound value and lower bound value;

Determine the coordinate value of the seed point;

With the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower limit value, the pixel is added to the area where the seed point is located; When the difference of the pixel value of the point is less than the upper limit value, the pixel point is added to the area where the seed point is located;

If all pixels have not been added, mark the position; use the newly added pixel as a new seed point, and return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point When it is greater than the lower bound value, the pixel point is added to the area where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper bound value, the pixel point is added to the area where the seed point is located area; until all pixels are added;

If all the pixels are added; each area is separated and marked for output;

S3. Based on the H channel of the HSV model, the color distribution range of the digital display signal in the marked area is established, and an image of the numerical value area is obtained;

In HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or excessive brightness. The H channel will be used as the main support in the color extraction of this system, and then the color distribution range of the numerical signal will be established, a mask mask will be generated, and finally the numbers in the marked area will be accurately extracted.

S4. Inputting the image of the numerical value area into a prediction model for matching to obtain a value, the prediction model is obtained by training the neural network through a training data set, and the training data set includes non-dispersive infrared methane sensors under different environments, positions or illumination Digital signal image.

Before the neural network is trained by the training data set to obtain the prediction model, the following steps are also included: collecting the non-dispersive infrared methane sensor digital display signal image under different environments, positions, and illuminations as a test data set; inputting the test data set The prediction model is tested; the training of the prediction model is stopped after the test result meets the requirement of the preset recognition accuracy.

The prediction model obtained by training the neural network through the training data set also includes the following steps:

Collect digital display signal images of non-dispersive infrared methane sensors under different environments, locations or illumination to obtain sample libraries;

Construct the training data set and test data set of the neural network according to the sample library;

According to the constructed training data set, the neural network is trained to obtain the prediction model of the digital display signal.

Inputting the numerical value area image into the prediction model for matching to obtain the numerical value also includes the following steps:

Obtain the image of the numerical region and perform size scaling to obtain the numerical region vector;

Substitute the value area vector into the prediction model for matching to obtain the value;

Sorting from left to right according to the pixel coordinates to get the numerical sequence concatenated as a string output.

Collect digital display images of methane sensors in different environments, locations, and lighting conditions, and establish a sample library for training and testing data sets for neural networks. According to the constructed training data set, a deep neural network is used to increase the number of neuron connection weights, thresholds and other parameters by increasing the number of hidden layers to increase the number of neurons and the number of nested layers of the activation function. The parameters are grouped by pre-training and fine-tuning methods, and the local optimal settings are first found for each group, and the global optimization is jointly performed based on the locally optimal results. It effectively saves training overhead while utilizing the degree of freedom provided by a large number of parameters of the model. Calculate the loss and update the model parameters through continuous iteration of the trained model. Finally, input the digital display image of the methane sensor to be identified into the model for matching, and obtain the specific value of the digital display image.

Through multiple acquisitions of digital display images in the field environment, build a test data set and a training data set, build a data set to train the network, complete the model required for recognition, and scale the size of the extracted value area to the model size And substitute into the model for numerical matching and output. Complete the identification of digital display signals of multiple methane sensors.

As another aspect of the embodiments of the present disclosure, a digital display signal recognition system based on deep learning is provided, including:

The image acquisition module acquires images containing digital display signals of multi-channel non-dispersive infrared methane sensors;

The image segmentation module implements the segmentation of the image and marks the marked area containing the digital display signal by using the flood filling method;

The image segmentation module also includes an image preprocessing module: performing binarization processing on the image to obtain a grayscale image, and performing Gaussian filtering on the image to obtain a blurred image;

Acquiring the change of adjacent pixels of the grayscale image and the change of adjacent pixels of the blurred image;

According to the changes of the adjacent pixels of the grayscale image and the changes of the adjacent pixels of the blurred image, the pixel values of the grayscale image and the blurred image are normalized. If the high-frequency component of the high-frequency image has almost no change, it will be judged as a completely blurred image and will not be processed. If the high-frequency component changes significantly, it will be judged as a partially blurred image. If the high-frequency component changes very much, it will be Judged as a clear image. After comparison and analysis, normalization processing is performed; partially blurred images, clear images and completely blurred images are obtained;

For clear images, the segmentation process of the next step will be directly carried out. For some blurred images, it will be re-judged whether the image is a clear image after deblurring processing. If so, the next step of segmentation processing will be performed. For completely blurred images, no processing .

The image segmentation module also includes a deblurring processing module: perform pixel grayscale statistics on a part of the blurred image to obtain the pixel probability distribution of the grayscale image; obtain the cumulative distribution function of the image to obtain the transformed deblurred image; The image is first subjected to pixel grayscale statistics, the pixel probability distribution of the original image is calculated, the cumulative distribution function of the image is obtained from the pixel probability distribution, and the transformed deblurred image is obtained according to the mapping function. The mapping function is as follows:

k= 0,1,2,3,···L-1

k= 0,1,2,3,···L-1

是当前灰度级的像素个数，L是图像中可能的灰度级总数。where n is the sum of pixels in the image,

is the number of pixels in the current gray level, and L is the total number of possible gray levels in the image.

The image segmentation module also includes the segmentation processing module of the next link, which is used for:

Use the mask matrix to mark the flood filled area;

Use a single-channel image whose size is 2 pixels wider and higher than the input image as a mask matrix, and mark the flood-filled area by filling the pixel values of the pixels in the mask matrix (the flood-filled area is more The display area of the digital display signal of the methane sensor on the image)

Obtain the upper and lower bound values of the seed point area conditions, where the upper and lower bound values include: upper bound value and lower bound value;

Determine the coordinate value of the seed point;

With the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower limit value, the pixel is added to the area where the seed point is located; When the difference of the pixel value of the point is less than the upper limit value, the pixel point is added to the area where the seed point is located;

If all pixels have not been added, mark the position; use the newly added pixel as a new seed point, and return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point When it is greater than the lower bound value, the pixel point is added to the area where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper bound value, the pixel point is added to the area where the seed point is located area; until all pixels are added;

If all the pixels are added; each area is separated and marked for output;

The extraction module establishes the color distribution range of the digital display signal in the marked area based on the H channel of the HSV model, and obtains a numerical area image;

In HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or excessive brightness. The H channel will be used as the main support in the color extraction of this system, and then the color distribution range of the numerical signal will be established, a mask mask will be generated, and finally the numbers in the marked area will be accurately extracted.

For the extraction of the value in the area to be identified, based on the HSV model, the distribution range of the value color is found by analyzing the distribution histogram of the H component. The extraction of regional values is completed based on the H channel.

A matching module, inputting the image of the value region into a prediction model for matching to obtain a value, the prediction model is obtained by training the neural network through a training data set, and the training data set includes non-dispersive infrared methane under different environments, positions or illumination Sensor digital display signal image.

The matching module also includes a training module: collection includes non-dispersive infrared methane sensor digital display signal images under different environments, positions and illuminations as a test data set; the test data set is input into the prediction model for testing; when the test results meet the predetermined Stop training the prediction model after setting the recognition accuracy requirement.

The matching module also includes the model prediction module:

Collect digital display signal images of non-dispersive infrared methane sensors under different environments, locations or illumination to obtain sample libraries;

Construct the training data set and test data set of the neural network according to the sample library;

According to the constructed training data set, the neural network is trained to obtain the prediction model of the digital display signal.

The matching module also includes the result output module:

Obtain the image of the numerical region and perform size scaling to obtain the numerical region vector;

Substitute the value area vector into the prediction model for matching to obtain the value;

Sorting from left to right according to the pixel coordinates to get the numerical sequence concatenated as a string output.

Collect digital display images of methane sensors in different environments, locations, and lighting conditions, and establish a sample library for training and testing data sets for neural networks. According to the constructed training data set, a deep neural network is used to increase the number of neuron connection weights, thresholds and other parameters by increasing the number of hidden layers to increase the number of neurons and the number of nested layers of the activation function. The parameters are grouped by pre-training and fine-tuning methods, and the local optimal settings are first found for each group, and the global optimization is jointly performed based on the locally optimal results. It effectively saves training overhead while utilizing the degree of freedom provided by a large number of parameters of the model. Calculate the loss and update the model parameters through continuous iteration of the trained model. Finally, input the digital display image of the methane sensor to be identified into the model for matching, and obtain the specific value of the digital display image.

The present disclosure provides a method for segmenting, collecting and accurately identifying images of digital display signals of multiple methane sensors. Through the multi-channel methane sensor recognition system based on deep learning, the accurate recognition of the digital display signals of multiple non-dispersive infrared methane sensors is realized, and the control of the change of the sensor indication value is guaranteed without human participation, and the realization Full monitoring. Strong adaptability and high stability.

Description of drawings

FIG. 1 is a flowchart of a deep learning-based digital display signal recognition method in Embodiment 1 of the present disclosure;

FIG. 2 is a flow chart of image segmentation steps in Embodiment 1 of the present disclosure;

Fig. 3 is a diagram of a digital display signal recognition system based on deep learning in Embodiment 2 of the present disclosure.

Fig. 4 is a block diagram of a digital display signal recognition system based on deep learning in Embodiment 2 of the present disclosure.

Implementation

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, components and circuits that are well known to those skilled in the art have not been described in detail so as to obscure the gist of the present disclosure.

It can be understood that the above-mentioned method embodiments mentioned in this disclosure can all be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, this disclosure will not repeat them.

In addition, the present disclosure also provides a deep learning-based digital display signal recognition method system, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any deep learning-based digital display signal provided by the present disclosure. The identification method, corresponding technical solutions and descriptions refer to the corresponding records in the method section, and will not be repeated here.

A digital display signal recognition method based on deep learning may be executed by a computer or other device capable of implementing the method. For example, the method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user device (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc. In some possible implementation manners, the method may be implemented by a processor invoking computer-readable instructions stored in a memory.

Example

As an aspect of the embodiments of the present disclosure, a deep learning-based digital display signal recognition method is provided, as shown in FIG. 1 , including the following steps:

S1. Obtain an image containing digital display signals of multiple non-dispersive infrared methane sensors;

S2. Using the flood filling method to realize the segmentation of the image and mark the marked area containing the digital display signal;

Adopting flood filling method to realize the segmentation of the image also includes the following steps:

performing binary processing on the image to obtain a grayscale image, and performing Gaussian filtering on the image to obtain a blurred image;

Acquiring the change of adjacent pixels of the grayscale image and the change of adjacent pixels of the blurred image;

According to the changes of the adjacent pixels of the grayscale image and the changes of the adjacent pixels of the blurred image, the pixel values of the grayscale image and the blurred image are normalized. If the high-frequency component of the high-frequency image has almost no change, it will be judged as a completely blurred image and will not be processed. If the high-frequency component changes significantly, it will be judged as a partially blurred image. If the high-frequency component changes very much, it will be judged as a clear image;

For clear images, the segmentation process of the next step will be directly carried out. For some blurred images, it will be re-judged whether the image is a clear image after deblurring processing. If so, the next step of segmentation processing will be performed. For completely blurred images, no processing .

The deblurring process includes: performing pixel grayscale statistics on part of the blurred image to obtain the pixel probability distribution of the grayscale image; obtaining the cumulative distribution function of the image to obtain the transformed deblurred image; degree statistics, calculate the pixel probability distribution of the original image, obtain the cumulative distribution function of the image from the pixel probability distribution, and obtain the transformed defuzzified image according to the mapping function, and the mapping function is as follows:

k= 0,1,2,3,···L-1

k= 0,1,2,3,···L-1

是当前灰度级的像素个数，L是图像中可能的灰度级总数。where n is the sum of pixels in the image,

is the number of pixels in the current gray level, and L is the total number of possible gray levels in the image.

As shown in Figure 2, it is a flowchart of image segmentation steps in this embodiment, and the segmentation processing of the next link includes:

Use the mask matrix to mark the flood filling area; use a single-channel image whose size is 2 pixels wider and higher than the input image as a mask matrix, and mark the flood by filling the pixel values of the pixels in the mask matrix Water-filled area (the area filled with water is the display area of digital display signals of multiple methane sensors on the image)

Obtain the upper and lower bound values of the seed point area conditions, where the upper and lower bound values include: upper bound value and lower bound value;

Determine the coordinate value of the seed point;

With the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower limit value, the pixel is added to the area where the seed point is located; When the difference of the pixel value of the point is less than the upper limit value, the pixel point is added to the area where the seed point is located;

If all pixels have not been added, mark the position; use the newly added pixel as a new seed point, and return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point When it is greater than the lower bound value, the pixel point is added to the area where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper bound value, the pixel point is added to the area where the seed point is located Area; until all pixel points are added; for example, with the seed point as the center, and with the seed point as the center, judge the difference between the pixel value in the 4-field and the pixel value of the seed point, and the difference is less than the upper bound and greater than the lower bound Value, add this pixel point to the area where the seed point is located; and use the newly added pixel point as a new seed point.

If all the pixels are added; each area is separated and marked for output;

The flood filling method is used to segment the image, and for the image frame containing the digital display signal of multiple methane sensors, the area block where the value is located is segmented. Find the same area according to the difference between the pixel gray values, understand the gray value of the pixel as the height of the pixel, regard a frame of image as a rugged mountain, inject a certain amount of water into a low-lying area on the ground, The water surface will cover the area below a certain height. Based on this principle, a water injection pixel is selected in the image, which is the seed point. The seed points will continue to spread outward according to certain rules to form independent regions with similar characteristics, and then realize image segmentation and frame selection and mark out the regions that need to be identified.

Send specified instructions to the processor through the host computer, the camera will continuously and intermittently take multiple digital display images of the methane sensor, and perform the first sharpness recognition processing on the original image, and directly perform the segmentation processing on the next link for the clear image, For partially blurred images, it will be re-judged whether the image is qualified after deblurring processing. For completely blurred images, there is no processing meaning and will not be processed. After completing the screening of the image, the image is segmented based on the distribution of the value, and the area to be recognized is found and marked.

S3. Based on the H channel of the HSV model, the color distribution range of the digital display signal in the marked area is established, and an image of the numerical value area is obtained;

In some embodiments, in the HSV color space, the value range of the hue H is [0, 360]. There are 256 gray levels that can be represented by each pixel in an 8-bit image, so when representing an HSV image in an 8-bit image, the angle of the hue must be mapped to the range [0,255]. After determining the value range, you can directly find the corresponding value in the H channel of the image to find a specific color. In the saturation S, the components of R, G, and B contained in the grayscale color are equivalent, which is equivalent to an extremely unsaturated color. So, the saturation of a grayscale color is 0. When displayed as a grayscale image, lighter areas correspond to colors with higher saturation. If the color is low in saturation, then it will not be able to reliably calculate the hue. The brightness V range is consistent with the saturation range. The larger the brightness value, the brighter the image; the lower the brightness value, the darker the image.

In some embodiments, in HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or excessive brightness. The H channel will be used as the main support in the color extraction of this system, and then the color distribution range of the numerical signal will be established, a mask mask will be generated, and finally the numbers in the marked area will be accurately extracted.

In some embodiments, for the extraction of numerical values in the area to be identified, based on the HSV model, the distribution range of the numerical color is found by analyzing the distribution histogram of the H component. The extraction of regional values is completed based on the H channel.

S4. Inputting the image of the numerical value area into a prediction model for matching to obtain a value, the prediction model is obtained by training the neural network through a training data set, and the training data set includes non-dispersive infrared methane sensors under different environments, positions or illumination Digital signal image.

Before the neural network is trained by the training data set to obtain the prediction model, the following steps are also included: collecting the non-dispersive infrared methane sensor digital display signal image under different environments, positions, and illuminations as a test data set; inputting the test data set The prediction model is tested; the training of the prediction model is stopped after the test result meets the requirement of the preset recognition accuracy.

The prediction model obtained by training the neural network through the training data set also includes the following steps:

Collect digital display signal images of non-dispersive infrared methane sensors under different environments, locations or illumination to obtain sample libraries;

Construct the training data set and test data set of the neural network according to the sample library;

According to the constructed training data set, the neural network is trained to obtain the prediction model of the digital display signal.

Inputting the numerical value area image into the prediction model for matching to obtain the numerical value also includes the following steps:

Obtain the image of the numerical region and perform size scaling to obtain the numerical region vector;

Substitute the value area vector into the prediction model for matching to obtain the value;

Sorting from left to right according to the pixel coordinates to get the numerical sequence concatenated as a string output.

In some embodiments, the digital display images of the methane sensor under different environments, locations, and illuminations are collected, and a sample library is established, which is used to construct training data sets and test data sets of the neural network. According to the constructed training data set, a deep neural network is used to increase the number of neuron connection weights, thresholds and other parameters by increasing the number of hidden layers to increase the number of neurons and the number of nested layers of the activation function. The parameters are grouped by pre-training and fine-tuning methods, and the local optimal settings are first found for each group, and the global optimization is jointly performed based on the locally optimal results. It effectively saves training overhead while utilizing the degree of freedom provided by a large number of parameters of the model. Calculate the loss and update the model parameters through continuous iteration of the trained model. Finally, input the digital display image of the methane sensor to be identified into the model for matching, and obtain the specific value of the digital display image.

The numerical matching process includes:

S301. Collect digital display signal diagrams of methane sensors in different environments

S302. Construct training and testing data sets of the neural network according to the sample library

S303. Train the neural network according to the constructed training set to obtain a methane sensor digital display signal prediction model

S304. Substituting the image to be recognized into the prediction model

S305. Acquire the value area and perform size scaling processing into vector form

S306. Substituting the numerical region vector into the model for matching to obtain a numerical value

S307. Sorting from left to right according to the pixel coordinates to obtain a numerical sequence and connect it as a string output

Through multiple acquisitions of digital display images in the field environment, build a test data set and a training data set, build a data set to train the network, complete the model required for recognition, and scale the size of the extracted value area to the model size And substitute into the model for numerical matching and output. Complete the identification of digital display signals of multiple methane sensors.

The test of this method is completed in the laboratory of the metrological verification test center, the ambient temperature is kept between 15 and 35°C, the relative humidity is not more than 85%, and the pressure is standard atmospheric pressure. During the test, the sensors with inspection are hung in the verification cabinet designed by the system. In order to ensure the verification of twelve sensors at the same time, four rows are fixed in the cabinet, and the sensors to be verified are hung by hooks. Place the wide-angle camera used for detection in front of the 12-way sensor, and connect it to the processor through a signal line.

Example

As another aspect of the embodiments of the present disclosure, a multi-channel methane sensor digital display signal recognition system based on deep learning is provided, as shown in FIG. 3 , including:

The image acquisition module acquires images containing digital display signals of multi-channel non-dispersive infrared methane sensors;

The image segmentation module implements the segmentation of the image and marks the marked area containing the digital display signal by using the flood filling method;

For the image frame containing the digital display signal of multiple methane sensors, the area block where the value is located is segmented. Find the same area according to the difference between the pixel gray values, understand the gray value of the pixel as the height of the pixel, regard a frame of image as a rugged mountain, inject a certain amount of water into a low-lying area on the ground, The water surface will cover the area below a certain height. Based on this principle, a water injection pixel is selected in the image, which is the seed point. The seed points will continue to spread outward according to certain rules to form independent regions with similar characteristics, and then realize image segmentation and frame selection and mark out the regions that need to be identified.

Send specified instructions to the processor through the host computer, the camera will continuously and intermittently take multiple digital display images of the methane sensor, and perform the first sharpness recognition processing on the original image, and directly perform the segmentation processing on the next link for the clear image, For partially blurred images, it will be re-judged whether the image is qualified after deblurring processing. For completely blurred images, there is no processing meaning and will not be processed. After completing the screening of the image, the image is segmented based on the distribution of the value, and the area to be recognized is found and marked.

The image segmentation module also includes an image preprocessing module: performing binarization processing on the image to obtain a grayscale image, and performing Gaussian filtering on the image to obtain a blurred image;

Acquiring the change of adjacent pixels of the grayscale image and the change of adjacent pixels of the blurred image;

According to the changes of the adjacent pixels of the grayscale image and the changes of the adjacent pixels of the blurred image, the pixel values of the grayscale image and the blurred image are normalized. If the high-frequency component of the high-frequency image has almost no change, it will be judged as a completely blurred image and will not be processed. If the high-frequency component changes significantly, it will be judged as a partially blurred image. If the high-frequency component changes very much, it will be judged as a clear image;

For clear images, the segmentation process of the next step will be directly carried out. For some blurred images, it will be re-judged whether the image is a clear image after deblurring processing. If so, the next step of segmentation processing will be performed. For completely blurred images, no processing .

The image segmentation module also includes a defuzzification processing module including: performing pixel gray level statistics on part of the blurred image to obtain the pixel probability distribution of the gray level image; obtaining an image cumulative distribution function to obtain a transformed image.

The image segmentation module also includes the segmentation processing module of the next link, which is used for:

Use the mask matrix to mark the flood filled area;

Use a single-channel image whose size is 2 pixels wider and higher than the input image as a mask matrix, and mark the flood-filled area by filling the pixel values of the pixels in the mask matrix (the flood-filled area is more The display area of the digital display signal of the methane sensor on the image)

Obtain the upper and lower bound values of the seed point area conditions, where the upper and lower bound values include: upper bound value and lower bound value;

Determine the coordinate value of the seed point;

With the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower limit value, the pixel is added to the area where the seed point is located; When the difference of the pixel value of the point is less than the upper limit value, the pixel point is added to the area where the seed point is located;

If all pixels have not been added, mark the position; use the newly added pixel as a new seed point, and return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point When it is greater than the lower bound value, the pixel point is added to the area where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper bound value, the pixel point is added to the area where the seed point is located Area; until all pixel points are added; for example, with the seed point as the center, and with the seed point as the center, judge the difference between the pixel value in the 4-field and the pixel value of the seed point, and the difference is less than the upper bound and greater than the lower bound Value, add this pixel point to the area where the seed point is located; and use the newly added pixel point as a new seed point.

If all the pixels are added; each area is separated and marked for output;

The extraction module establishes the color distribution range of the digital display signal in the marked area based on the H channel of the HSV model, and obtains a numerical area image;

In HSV, the V channel is most affected by light, and the H channel is basically not affected by shadows or excessive brightness. The H channel will be used as the main support in the color extraction of this system, and then the color distribution range of the numerical signal will be established, a mask mask will be generated, and finally the numbers in the marked area will be accurately extracted.

For the extraction of the value in the area to be identified, based on the HSV model, the distribution range of the value color is found by analyzing the distribution histogram of the H component. The extraction of regional values is completed based on the H channel.

In some embodiments, in the HSV color space, the value range of the hue H is [0, 360]. There are 256 gray levels that can be represented by each pixel in an 8-bit image, so when representing an HSV image in an 8-bit image, the angle of the hue must be mapped to the range [0,255]. After determining the value range, you can directly find the corresponding value in the H channel of the image to find a specific color. In the saturation S, the components of R, G, and B contained in the grayscale color are equivalent, which is equivalent to an extremely unsaturated color. So, the saturation of a grayscale color is 0. When displayed as a grayscale image, lighter areas correspond to colors with higher saturation. If the color is low in saturation, then it will not be able to reliably calculate the hue. The brightness V range is consistent with the saturation range. The larger the brightness value, the brighter the image; the lower the brightness value, the darker the image.

A matching module, inputting the image of the value region into a prediction model for matching to obtain a value, the prediction model is obtained by training the neural network through a training data set, and the training data set includes non-dispersive infrared methane under different environments, positions or illumination Sensor digital display signal image.

The matching module also includes a training module: collection includes non-dispersive infrared methane sensor digital display signal images under different environments, positions and illuminations as a test data set; the test data set is input into the predictive model for testing; The training of the predictive model is stopped after the requirement of the preset recognition accuracy is met.

The matching module also includes the model prediction module:

Collect digital display signal images of non-dispersive infrared methane sensors under different environments, locations or illumination to obtain sample libraries;

Construct the training data set and test data set of the neural network according to the sample library;

According to the constructed training data set, the neural network is trained to obtain the prediction model of the digital display signal.

The matching module also includes the result output module:

Obtain the image of the numerical region and perform size scaling to obtain the numerical region vector;

Substitute the value area vector into the prediction model for matching to obtain the value;

Sorting from left to right according to the pixel coordinates to get the numerical sequence concatenated as a string output.

In some embodiments, the digital display images of the methane sensor under different environments, locations, and illuminations are collected, and a sample library is established, which is used to construct training data sets and test data sets of the neural network. According to the constructed training data set, a deep neural network is used to increase the number of neuron connection weights, thresholds and other parameters by increasing the number of hidden layers to increase the number of neurons and the number of nested layers of the activation function. The parameters are grouped by pre-training and fine-tuning methods, and the local optimal settings are first found for each group, and the global optimization is jointly performed based on the locally optimal results. It effectively saves training overhead while utilizing the degree of freedom provided by a large number of parameters of the model. Calculate the loss and update the model parameters through continuous iteration of the trained model. Finally, input the digital display image of the methane sensor to be identified into the model for matching, and obtain the specific value of the digital display image.

Through multiple acquisitions of digital display images in the field environment, build a test data set and a training data set, build a data set to train the network, complete the model required for recognition, and scale the size of the extracted value area to the model size And substitute into the model for numerical matching and output. Complete the identification of digital display signals of multiple methane sensors. As shown in Figure 4, it is a structural diagram of the system.

Example

An electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, when the processor executes the computer program, a digital display based on deep learning in Embodiment 1 is realized Signal recognition method.

Embodiment 3 of the present disclosure is only an example, and should not impose any limitation on the functions and application ranges of the embodiments of the present disclosure.

An electronic device may take the form of a general computing device, which may be a server device, for example. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, a bus connecting different system components (including memory and processor).

The bus includes data bus, address bus and control bus.

The memory may include volatile memory, such as random access memory (RAM) and/or cache memory, and may further include read only memory (ROM).

The memory may also include program means having a set (at least one) of program modules including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of these examples or Implementations of network environments may be included in some combination.

The processor executes various functional applications and data processing by running computer programs stored in the memory.

Electronic devices may also communicate with one or more external devices (eg, keyboards, pointing devices, etc.). This communication can take place through an input/output (I/O) interface. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through the bus. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (array of disks) systems , tape drives, and data backup storage systems.

It should be noted that although several units/modules or subunits/modules of an electronic device are mentioned in the above detailed description, such division is only exemplary and not mandatory. Actually, according to the embodiment of the present application, the features and functions of two or more units/modules described above may be embodied in one unit/module. Conversely, the features and functions of one unit/module described above can be further divided to be embodied by a plurality of units/modules.

Example

A computer-readable storage medium, the readable storage medium stores a computer program, and when the program is executed by a processor, the steps of the deep learning-based digital display signal recognition method in Embodiment 1 are implemented.

Wherein, the readable storage medium may more specifically include but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device or any of the above-mentioned the right combination.

In a possible implementation manner, the present disclosure may also be implemented in the form of a program product, which includes program code, and when the program product is run on a terminal device, the program code is used to make the terminal device execute The steps of the digital display signal recognition method based on deep learning described in Embodiment 1.

Wherein, the program code for executing the present disclosure may be written in any combination of one or more programming languages, and the program code may be completely executed on the user equipment, partially executed on the user equipment, or used as an independent The package executes, partly on the user device and partly on the remote device, or entirely on the remote device.

Claims (10)

1. The digital display signal recognition method based on deep learning, is characterized in that, comprises the steps: Obtain images containing digital display signals of multi-channel non-dispersive infrared methane sensors; Using the flood filling method to realize the segmentation of the image and mark the marked area containing the digital display signal; Based on the H channel of the HSV model, the color distribution range of the digital display signal in the marked area is established, and an image of the numerical value area is obtained; Inputting the image of the numerical value region into a prediction model for matching to obtain a value, the prediction model is obtained by training the neural network through a training data set, and the training data set includes digital display of non-dispersive infrared methane sensors under different environments, positions or illumination Signal image. 2. The digital display signal recognition method based on deep learning as claimed in claim 1, characterized in that, before the neural network is trained to obtain the prediction model through the training data set, the following steps are also included: the acquisition includes different environments, positions, illumination The non-dispersive infrared methane sensor digital display signal image below is used as a test data set; the test data set is input into the prediction model for testing; the training of the prediction model is stopped after the test results meet the requirements of the preset recognition accuracy. 3. the digital display signal recognition method based on deep learning as claimed in claim 1, is characterized in that, adopts flood filling method to realize the segmentation of described image and also comprises the steps: performing binary processing on the image to obtain a grayscale image, and performing Gaussian filtering on the image to obtain a blurred image; Acquiring the change of adjacent pixels of the grayscale image and the change of adjacent pixels of the blurred image; According to the changes of the adjacent pixels of the grayscale image and the changes of the adjacent pixels of the blurred image, the pixel values of the grayscale image and the blurred image are normalized. If the high-frequency component of the high-frequency image has almost no change, it will be judged as a completely blurred image and will not be processed. If the high-frequency component changes significantly, it will be judged as a partially blurred image. If the high-frequency component changes very much, it will be judged as a clear image; For clear images, the segmentation process of the next step will be directly carried out. For some blurred images, it will be re-judged whether the image is a clear image after deblurring processing. If so, the next step of segmentation processing will be performed. For completely blurred images, no processing . 4. The digital display signal recognition method based on deep learning as claimed in claim 3, wherein the deblurring process comprises the steps of: performing pixel grayscale statistics on a part of the blurred image to obtain the grayscale of the grayscale image Pixel probability distribution; obtain the cumulative distribution function of the image, and obtain the transformed deblurred image; first perform pixel grayscale statistics on the partial blurred image, calculate the pixel probability distribution of the original image, and obtain the cumulative distribution function of the image from the pixel probability distribution, The transformed deblurred image is obtained according to the mapping function, and the mapping function is as follows:

k= 0,1,2,3,···L-1 where n is the sum of pixels in the image,

is the number of pixels in the current gray level, and L is the total number of possible gray levels in the image. 5. the digital display signal recognition method based on deep learning as claimed in claim 3, is characterized in that, the segmentation processing of described next link comprises: Use a single-channel image whose size is 2 pixels wider and higher than the input image as a mask matrix, and mark the flooded area by filling the pixel values of the pixels in the mask matrix. Use the mask matrix to mark the flooded area filled area; Obtain the upper and lower bound values of the seed point area conditions, where the upper and lower bound values include: upper bound value and lower bound value; Determine the coordinate value of the seed point; With the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point is greater than the lower limit value, the pixel is added to the area where the seed point is located; When the difference of the pixel value of the point is less than the upper limit value, the pixel point is added to the area where the seed point is located; If all pixels have not been added, mark the position; use the newly added pixel as a new seed point, and return to the previous step: take the seed point as the center, when the difference between the pixel value of a pixel in the neighborhood and the pixel value of the seed point When it is greater than the lower bound value, the pixel point is added to the area where the seed point is located; when the difference between the pixel value of the seed point and the pixel value of a pixel in the neighborhood is less than the upper bound value, the pixel point is added to the area where the seed point is located area; until all pixels are added; If all the pixels are added, each area is separated and marked for output. 6. the digital display signal recognition method based on deep learning as claimed in claim 1, is characterized in that, described prediction model obtains by training data set to neural network and also comprises the steps: Collect digital display signal images of non-dispersive infrared methane sensors under different environments, locations or illumination to obtain sample libraries; Construct the training data set and test data set of the neural network according to the sample library; According to the constructed training data set, the neural network is trained to obtain the prediction model of the digital display signal. 7. The digital display signal recognition method based on deep learning as claimed in claim 1 or 6, wherein the input prediction model of the numerical value region image is matched to obtain a numerical value, further comprising the steps of: Obtain the image of the numerical region and perform size scaling to obtain the numerical region vector; Substitute the value area vector into the prediction model for matching to obtain the value; Sorting from left to right according to the pixel coordinates to get the numerical sequence concatenated as a string output. 8. A digital display signal recognition system based on deep learning, characterized in that, comprising: The image acquisition module acquires images containing digital display signals of multi-channel non-dispersive infrared methane sensors; The image segmentation module implements the segmentation of the image and marks the marked area containing the digital display signal by using the flood filling method; The extraction module establishes the color distribution range of the digital display signal in the marked area based on the H channel of the HSV model, and obtains a numerical area image; A matching module, inputting the image of the value region into a prediction model for matching to obtain a value, the prediction model is obtained by training the neural network through a training data set, and the training data set includes non-dispersive infrared methane under different environments, positions or illumination Sensor digital display signal image. 9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements any one of claims 1 to 7 when executing the computer program. The digital display signal recognition method based on deep learning described in the item. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the deep learning-based digital display signal recognition method according to any one of claims 1 to 7 is implemented .

Images