Disclosure of Invention
The present invention is directed to a method and a system for recovering an optical fiber image, so as to solve the problems in the background art.
In order to achieve the purpose, the invention adopts the following technical scheme: a fiber optic image restoration method comprising the steps of:
A. storing the captured video stream into an external SDRAM, reading an image from the SDRAM, displaying the image on a display through a display driving module, and finally adjusting an optical fiber bundle formed by the array optical fibers to a clear focusing position and fixing the optical fiber bundle by checking the image of the display to realize calibration;
B. high-intensity parallel light is irradiated on the optical fiber bundle, Y brightness components in the extracted video stream are subjected to binarization processing through a binarization module, then the binary image is subjected to corrosion expansion processing through a binary corrosion module and a binary expansion module to obtain an optical fiber image dead zone template, and a FLASH control module stores the optical fiber image dead zone template into an external FLASH to realize dead zone acquisition;
C. the dead zone template cache module reads an optical fiber image dead zone template from an external FLASH and stores the optical fiber image dead zone template in an internal BRAM of the FPGA, the video format conversion module converts the format of a video stream, and the dead zone recovery module respectively performs sliding processing on images of R, G, B three channels and the optical fiber dead zone image from left to right and from top to bottom by adopting the templates to obtain an image with recovered dead zones;
D. the 3D noise reduction algorithm divides the image into a plurality of blocks, each optical fiber corresponds to a corresponding block, and the SAD of the effective area of the optical fiber is adopted to calculate the motion intensity, so that the noise reduction is realized;
E. calculating the SAD of the optical fiber effective block between the front frame and the rear frame, and determining to adopt multi-frame noise reduction or 2D filtering to synthesize an image, wherein the calculation formula is as follows;
F. the edge sharpening module and the contrast enhancement module carry out edge sharpening and contrast enhancement processing on the image, the obtained target image is stored in an external SDRAM through the SDRAM control module, and finally the image is read out from the SDRAM and displayed on a display through the display driving module.
Further, in step E, BK-1,Bk,Bk+1Respectively a previous frame, a current frame and a next frame, and w is the weight of image operation.
And further, performing time-domain filtering when the motion intensity is smaller than T, and otherwise, performing spatial filtering.
Further, before step B, a camera parameter configuration module is required to configure a parameter register of the camera, a video capture module captures a picture, and a video stream of 1280 × 720@60Hz YUV422 is output through the camera.
Further, the template in step C is specifically a 5 × 5 template.
Further, the step C includes two cases after the sliding process.
Further, in the first case where the template center position is the non-dead-zone position, its corresponding R, G, B luminance value is directly taken as the target luminance value.
In the second case, when the center position of the template is the dead zone position, R, G, B pixels corresponding to the non-dead zone position in the template are averaged to obtain the target luminance.
Further, an optical fiber image recovery system is characterized in that: the camera is respectively connected with the camera parameter configuration module and the video capture module, one path of the video capture module is connected to the binarization module, the binary corrosion module, the binary expansion module, the FLASH control module and the dead zone template cache module, the other path of the video capture module is connected to the video format conversion module, the dead zone recovery module, the FLASH control module and the dead zone template cache module, and the other path of the video capture module is connected to the video format conversion module, the dead zone recovery module, the edge sharpening module, the contrast enhancement module, the SDRAM control module and the display drive module
Further, the dead zone template cache module is connected with the dead zone recovery module.
Compared with the prior art, the invention has the beneficial effects that:
the invention adopts high-intensity parallel light to irradiate the optical fiber bundle, carries out binarization processing and corrosion expansion processing on the gray level image of the image to obtain an optical fiber image dead zone template, then utilizes the optical fiber image dead zone template and 5 × 5 neighborhood to recover the image of a dead zone part, and carries out noise processing on the image by matching with space-time joint 3D noise reduction, thereby effectively improving the signal-to-noise ratio of optical fiber imaging.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
In the description of the present invention, it is to be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention.
Referring to fig. 1 to 4, the present invention provides a technical solution: a fiber optic image restoration method comprising the steps of:
the optical fiber image recovery and display system has three processing flows: a calibration mode, a dead band acquisition mode, and a dead band recovery mode. After the system is started, a camera parameter configuration module configures a register related to a camera, so that the camera outputs a 1280 × 720@60Hz YUV422 video stream, and the video stream is captured by a video capture module;
firstly, in a calibration mode, a captured video stream is stored in an external SDRAM, then an image is read from the SDRAM, the image is displayed on a display through a display driving module, and finally, an optical fiber bundle is adjusted to a clear focusing position and fixed by watching the image of the display.
Secondly, in a dead zone acquisition mode, high-intensity parallel light is applied to the optical fiber bundle, Y brightness components in the video stream are extracted for binarization processing, then the binary image is subjected to corrosion expansion processing to obtain an optical fiber image dead zone template, and the optical fiber image dead zone template is stored in an external FLASH through a FLASH control module.
Then, under a dead zone recovery mode, reading an optical fiber image dead zone template from an external FLASH and storing the optical fiber image dead zone template in an internal BRAM of the FPGA, converting a video stream YUV422 into RGB888, then respectively carrying out left-to-right and top-to-bottom sliding processing on R, G, B images of three channels and the optical fiber dead zone image by using a 5-by-5 template, directly taking a R, G, B brightness value corresponding to the template as a target brightness value if the central position of the template is a non-dead zone position, and obtaining an average value of R, G, B pixels corresponding to the non-dead zone position in the template as the target brightness if the central position of the template is the dead zone position, thus obtaining an image after dead zone recovery, then carrying out edge sharpening and contrast enhancement processing on the image, storing the obtained target image into an external SDRAM, finally reading the image from the SDRAM, and displaying the image on a display through a display driving module.
In this embodiment, an optical fiber image recovery system includes a camera parameter configuration module, a video capture module, a binarization module, a binary erosion module, a binary expansion module, a FLASH control module, a dead zone template cache module, a video format conversion module, a dead zone recovery module, an edge sharpening module, a contrast enhancement module, an SDRAM control module, and a display driving module,
the camera parameter configuration module configures various parameter registers of the camera through an IIC interface;
the video capturing module collects a video stream from the camera;
the binarization module converts the gray level image into a binary image;
the binary corrosion module removes noise, but the target image is compressed;
the binary expansion module performs expansion processing on the corroded image, so that not only is noise removed, but also the target image is kept original, and an optical fiber dead zone template is obtained;
the FLASH control module stores the optical fiber dead zone template into FLASH or reads the optical fiber dead zone template out of the FLASH;
the dead zone template cache module is used for storing the optical fiber dead zone template read out from the FLASH into the BRAM;
the video format conversion module converts YUV422 into RGB 888;
the edge sharpening module sharpens the edge of the image;
the contrast enhancement module can improve the contrast of the image;
SDRAM control module: storing the image into SDRAM or SDRAM to read out;
the display driving module generates related video time sequence to drive the display.
The camera is respectively connected with a camera parameter configuration module and a video capture module, one path of the video capture module is connected to a binarization module, a binary corrosion module, a binary expansion module, a FLASH control module and a dead zone template cache module, the other path of the video capture module is connected to a video format conversion module, a dead zone recovery module, an edge sharpening module, a contrast enhancement module, an SDRAM control module and a display driving module, and the dead zone template cache module is connected with the dead zone recovery module.
The working principle and the using process of the invention are as follows:
and (5) carrying out image binarization processing and converting the gray level image into a binary image. If the target pixel brightness value p1If the value is more than or equal to T, the corresponding binary image pixel value p is added2Setting as 1; if the target pixel brightness value p1If the value is less than T, the corresponding binary image pixel value p is2Is set to 0. Wherein T is a binarization threshold value and can be set through a key;
performing image erosion processing, centering on a target pixel, taking a binary image 3-by-3 region (P1-P9), and if the luminance value of the region is 0, namely P1 & P2 & P3 & P4 & P5 & P6 & P7 & P8 & P9, the luminance value of the target pixel is 0; if all brightness values in the field are 1, namely P1 & P2 & P3 & P4 & P5 & P6 & P7 & P8 & P9, the brightness value of the target pixel is 1;
performing image expansion processing, taking a binary image 3-by-3 field (P1-P9) with a target pixel as a center, and setting the brightness value of the target pixel to 1 if the field has a brightness value of 1, namely, P1| P2| P3| P4| P5| P6| P7| P8| P9, respectively; if all luminance values in the field are 0, i.e., P1| P2| P3| P4| P5| P6| P7| P8| P9 ═ 0, then the luminance value of the target pixel is set to 0;
converting YUV422 into RGB888, converting YUV422 into YUV444, and then converting YUV 888 into RGB888, wherein the formula of converting YUV444 into RGB888 is as follows:
R=1.1644*Y+1.6019*V-223.5521
G=1.1644*Y-0.3928*U-0.8163*V+136.1381
B=1.1644*Y+2.0253*U-278.0291
when the formula conversion is realized by logic, the formula is subjected to fixed point processing, and the corresponding formula is as follows:
R=(1192*Y+1640*V-228917)>>10
G=(1192*Y-402*u-836*V+139405)>>10
B=(1192*Y+2074*U-284702)>>10
and (3) recovering dead zones of the image, wherein the non-dead zone positions in the image are represented by original pixel brightness, and the dead zone positions are represented by pixel brightness mean values of the non-dead zone positions in a 5-by-5 field of the dead zone positions. In this embodiment, the pixels in the 5 × 5 neighborhood are P1 to P9, the dead zone position in the 5 × 5 neighborhood is represented by 0, the non-dead zone position is represented by 1, and the image restoration calculation process is shown in fig. 2;
the spatial-temporal joint 3D noise reduction algorithm is used for processing, 1280 × 720 imaging is adopted, and 100 × 100 array fibers are adopted, each fiber is approximately just an 8 × 8 block, the SAD of the effective area of the fiber is used for calculating the motion intensity, the interference of dead zones can be effectively reduced, and SAD calculation errors caused by noise influence under low illumination can be filtered out, the traditional 3D noise reduction divides the image into blocks such as 8 × 8 and 16 × 16, the motion intensity is judged through the SAD (the square sum of difference) between adjacent frames, and due to the specificity of light imaging, the image is naturally divided into prismatic blocks, and the fiber gaps are arranged among the blocks, as shown in fig. 3;
finally, whether multi-frame noise reduction or 2D filtering is adopted is determined through calculation of an SAD (sum of absolute differences) of the optical fiber effective blocks between the front frame and the rear frame, and a related formula is shown as follows;
BK-1,Bk,Bk+1the weights of the w image operation are respectively the previous frame, the current frame and the next frame. And performing time domain filtering when the motion intensity is smaller than T, and otherwise, performing spatial filtering. Assuming that w is 0.5, when motion is small, the operation is performed on the pixels of the matching block, and then the weight of 0.25 of the previous frame, the weight of 0.5 of the current frame, and the weight of 0.25 of the next frame are taken. When the motion intensity is high, the spatial filtering is not carried out;
then, processing the image by adopting a Laplace sharpening algorithm, as shown in FIG. 4;
then, image contrast enhancement processing is carried out to improve the contrast of the image, namely, the image is made to have darker and brighter brightness, and a corresponding formula is shown as follows;
and realizing circuit design by adopting a table look-up mode, namely calculating y corresponding to x of 0-255 in advance and storing the y in a ROM.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.