CN104063847A - FPGA based guide filter and achieving method thereof - Google Patents

Abstract

The invention discloses an FPGA based guide filter and an achieving method thereof. The FPGA based guide filter comprises a signal controller, an average filtering module, a parameter calculating module, an addressing dereferencing module, a state judging device, a data cache and a comprehensive operation module. The achieving method of the FPGA based guide filter is combined with an FPGA chip, fully utilizes a characteristic that large-scale parallel operation of the FPGA is easy to achieve and improves the operation speed and reduces the complexity level of the guide filter hardware design while ensuring the filter effect. A filtering treatment parallel structure is given wholly, and the filtering speed is improved. The binary division operation is improved in an addressing dereferencing mode, hardware achievement is facilitated, and the system running time is shortened. A binary division algorithm is improved through the judging device, and the system reliability is improved.

Description

A Guided Filter Based on FPGA and Its Realization Method

technical field

The invention belongs to the technical field of guided filters, in particular to an FPGA-based guided filter and an implementation method thereof.

Background technique

The guided filter is a kind of edge smoothing filter, which can realize functions such as smoothing of image edges, enhancement of details, and image fusion and denoising. It is a powerful filter. Its principle is to filter the input image through a guide image, and the output image can fully obtain the change details of the guide image while retaining the overall characteristics of the input image. Guided filters have achieved good results in image denoising, image detail smoothing, enhancement, matting, feathering, etc., and are widely used.

However, the guided filtering algorithm is more complicated and the calculation amount is relatively large, which leads to the poor real-time performance of the guided filter, which hinders its practical application. Considering the independence of parallel modules in the algorithm, hardware implementation is one of the effective ways to improve the real-time performance of filtering. At present, the vast majority of guided filtering literature is about its theoretical research and algorithm simulation in software, but very little about its hardware implementation.

FPGA is a programmable device based on a look-up table structure. It has a large-capacity BlockRAM resource, and can change the logic function through reconfiguration in the system, which provides the possibility for the hardware design and implementation of the guided filter. How to use FPGA to realize the hardware design of guided filter to achieve the goal of fast processing of guided filter, so as to realize the real-time application of guided filter in image processing and other fields, is of great significance.

Contents of the invention

The purpose of the embodiments of the present invention is to provide an FPGA-based guided filter and its implementation method, aiming to solve the problem that the guided filter algorithm is relatively complex and the amount of calculation is relatively large, resulting in poor real-time performance of the guided filter.

The embodiment of the present invention is realized in this way, a kind of guiding filter based on FPGA, this guiding filter based on FPGA comprises: signal controller, average value filtering module, parameter calculation module, addressing value module, state discriminator, data Registers, integrated computing modules;

The signal controller is composed of row and column counters and comparators, which are used for real-time monitoring of data, analyzing data status through input system control signals, marking and sending row signal and field signal work instructions in time, to ensure that data can be processed correctly according to the instructions;

The average value filtering module is connected with the signal controller and is used to perform average value filtering processing on the data. It is composed of a local window generation module and an average value calculator module. In order to output the serial data in parallel, the average value calculation module is composed of an adder and a multiplier for data processing. On the basis of the improved algorithm of binary division, it calculates the sum of the data input in parallel to the average value calculation module, and then enters to the multiplier to calculate the mean of the parallel input data;

The parameter calculation module is connected with the mean value filter module and consists of multiple adders, subtractors, multipliers, comparators and registers, used to collect data and complete preliminary data processing. During the calculation process, it calls each other with the addressing value module. Obtain the variance value and local linear coefficient value through calculation;

The addressing value module is connected with the mean value filter module and parameter calculation module, which is used to improve the binary division algorithm, and fully utilizes the FPGA's large-capacity BIOCKRAM resource, which is used to store a large amount of data and realize the characteristics of a fast lookup table function. The division operation is changed to a register addressing process and a multiplication operation process, and the addressing and value-taking module is generated by improving to provide the required data for the parameter calculation module and the mean filtering module;

The state discriminator is connected with the parameter calculation module, and is composed of a comparator, an adder, a subtractor and a register. It is set based on the fact that binary numbers with symbols in the improved calculation process are not easy to distinguish and process. It is used for input The data comparison generates a state enable signal, and the drive parameter calculation module and the comprehensive operation module are intelligently processed according to the enable state to reduce the computational complexity of the system;

The data buffer, connected to the comprehensive operation module, is composed of FIFO and shift register quickly generated by the system, which is used to coordinate the data timing and ensure that the data can be processed synchronously;

The comprehensive operation module is connected with the mean value filter module, the state discriminator and the data buffer, and is used for data comprehensive processing, combined with the data comprehensive processing of the data buffer and the parameter calculation module, and finally outputs the image data information filtered by the guided filter.

Another object of the embodiments of the present invention is to provide a method for implementing a guided filter based on FPGA, the method for implementing a guided filter based on FPGA includes the following steps:

Step 1: Signal controller:

According to the pipeline design method, after the local window is generated, it will gradually move to the right with the continuous input of data, and an adder is used to form a counter, which marks, judges and sends status information to the system input control signal under the control of the synchronous clock;

Step 2: Mean filtering:

The local generation window in the mean value filter module is a 3*3 module, the input image is P_in, and the guide image is I_in, which are input to the local generation window module row by row in the form of data strings under the excitation of a synchronous clock. After passing through two FIFOs and 6 After registers, output data group 1: P1, P2...P9 and data group 2: I1, I2...I9, use the multiplier to obtain data group 3: IP1, IP2...IP9 and data group 4: II1, II2... ...II9, send the four sets of data to four mean calculation modules respectively, and obtain the mean values of the data sets through calculation: ave_P, ave_I, ave_IP and ave_II;

Step 3: Variable calculation:

The first step, parameter calculation:

After obtaining the four sets of data of ave_P, ave_I, ave_IP and ave_II, through analysis and sorting, calculate (a _k , b _k );

The second step, addressing value: Based on the FPGA design of the improved binary division algorithm, the dividend is multiplied by the reciprocal of the divisor, and the division is converted into multiplication to simplify the calculation process;

The third step, state discriminator:

The calculation process of steps 3 and 6 involves binary subtraction operations, and negative numbers will be generated during the calculation process, which is based on the processing of the state discriminator for signed binary numbers;

Step 4: Mean filtering:

It is used to perform mean value filtering on (a _k , b _k ) obtained in step 3, see step 2 for the specific method, local window generation modules C and D respectively generate data groups 5: a _k1 , a _k2 ... a _k9 and data Group 6: b _k1 , b _k2 ... b _k9 to the mean value calculation module, and output the mean values ave_a and ave_b of the data group;

Step 5: Data buffer:

Step 2 and Step 4 call the local window generation module respectively, so 4 FIFOs are needed in the line buffer design, and the FIFO depth is consistent with the FIFO depth used in the local generation window. During the design period, some formula calculations will cause delays. Therefore, register banks are also needed in the design of the data buffer, and I_in outputs I_in_delay after being cached by the data buffer;

Step 6: Comprehensive calculation:

such as formula As shown, the data ave_a and ave_b obtained in step 4 are combined with the I_in_delay output in step 5 and then output. The obtained data is the final image data output after being processed by the guiding filter. During the design process, the state discriminator is used to select the output.

Further, the specific steps in step two are as follows:

In the first step, the window is generated:

Under the action of the FIFO controller, the P_in and I_in data are written into and read from the FIFO, and the data in the (i-2) row is first stored in FIFO1 in sequence under the action of the synchronous clock, and then stored according to the time when it is full. Store in FIFO2 sequentially, and store the data in row (i-1) in FIFO1 at the same time. When FIFO1 and FIFO2 are full of data and the i-th row of data arrives, using the principle of pipeline design, two registers are set for each row of the window to cache the same For the data of column coordinates, wait for the window to be filled and output 9 data in parallel at the same time, that is, get data group 1: P1, P2...P9 and data group 2: I1, I2...I9, and get data group 3: IP1, IP2 ...IP9 and data group 4: II1, II2...II9, the obtained four sets of data are sent to the mean calculation module;

The second step, mean calculation:

The mean value calculation is divided into two processes, that is, the summation process and the division operation process, as shown in formulas (7) and (8):

ave=Sum/N (7)

$\mathrm{<span\; class="notranslate">\; Sum</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Among them, ave is the calculated average value, Sum is the sum of all pixel values in the sliding template, N is the total number of pixels in the template, and N is 9;

a, summation, according to the formula (8), sum the data groups 1-4 respectively to obtain Sum1, Sum2, Sum3, Sum4;

b. Calculating the mean value. The process of calculating the mean value is the process of FPGA processing the binary division operation. After calculating the mean value, the mean values of the 4 data groups are obtained, which are respectively ave_P, ave_I, ave_IP and ave_II.

Further, in the first step of step three, the calculation process of (a _k , b _k ) is as follows:

1) According to the formula The calculation process of a _k is decomposed and calculated in parallel as follows:

a. Molecule That is, ave_IP-ave_I*ave_P is directly obtained through the multiplier and subtractor, and there is a binary subtraction operation in the numerator, and the state discriminator is used for optimization;

b. The denominator σ _k ² +ε, where ε is a fixed value, and σ _k ² is the variance of the guide image in the local window. Using the definition of variance in mathematical concepts, we can know the relationship between variance and mean value as shown in formula (9):

σ _k ² =E(x ² )-E ² (X) (9)

That is, ave_II-ave_I*ave_I, a subtractor, a multiplier and an adder are required in the calculation process;

c. Utilize the principle integration formula in 2 addressing values to obtain the a _k value in the local window;

2) by the formula It can be seen that the calculation of b _k depends on a _k , there is a calculation delay in the calculation process of a _k , and three registers are required to cache the data timing, and a multiplier and a subtractor are also used in the calculation process of b _k .

Further, in the second step of step 3, the specific method of addressing and fetching values is as follows:

When the divisor is given, its reciprocal is obtained by calculation. Since the reciprocal of the divisor is in decimal form and the value is not greater than 1, the high 16 bits after binarization of the decimal part are taken, together with the last 1 bit of the integer part, to form a bit Binary data with a width of 17 is recorded as the reciprocal of the divisor, and the division function is realized through multiplication.

Further, in the third step of step three, the specific method based on the state discriminator for signed binary numbers is as follows:

Calculate n=A-B and m=n+k, wherein k is a constant, and A and B are input values;

1) Numerical calculation and state discrimination, compare and judge the data A and B that need to be subtracted, if A is greater than or equal to B, record it as state S1, otherwise record it as S2, and calculate n1=A-B and n2=B-A at the same time ;n=n1 under the state S1 situation, n=n2 under the state S2 situation;

2) The state is reserved, and subsequent processing is performed to calculate m1=n+k and m2=k-n. If the state is S1, m=m1; otherwise, m=m2.

The guiding filter based on FPGA and its implementation method provided by the present invention, by combining FPGA chip, make full use of the characteristics that FPGA is easy to realize large-scale parallel computing, improve the computing speed while ensuring the filtering effect, and reduce the hardware design of guiding filter. of complexity. The present invention generally provides a parallel structure of filtering processing, which increases the speed of filtering; improves the binary division operation through addressing and taking values, facilitates hardware implementation, and improves the system running time; Subtraction algorithm improves system reliability. In addition, the present invention realizes the guided filter based on FPGA hardware, and can be applied to fields such as denoising, enhancement, and detail smoothing of image processing.

Description of drawings

Fig. 1 is the structure schematic diagram of guiding filter based on FPGA that the embodiment of the present invention provides;

Fig. 2 is the flow chart of the realization method of guiding filter based on FPGA provided by the embodiment of the present invention;

In the figure: 1. Signal controller; 2. Mean value filter module; 3. Parameter calculation module; 4. Addressing value module; 5. State discriminator; 6. Data register; 7. Comprehensive operation module;

FIG. 3 is a schematic diagram of a local window generation method provided by an embodiment of the present invention;

FIG. 4 is a schematic diagram of an addressing and value-taking method provided by an embodiment of the present invention;

Fig. 5 is a working flow chart of the state discriminator provided by the embodiment of the present invention;

Fig. 6 is a working principle diagram of the data buffer provided by the embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The application principle of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

As shown in Figure 1, the guiding filter based on FPGA of the embodiment of the present invention is mainly composed of: signal controller 1, mean value filtering module 2, parameter calculation module 3, addressing value module 4, state discriminator 5, data register 6. Composed of 7 comprehensive computing modules;

Signal controller 1, composed of row and column counters and comparators, is used to monitor data in real time, analyze data status through input system control signals, mark and send work instructions such as row signals and field signals in time, and ensure that data can be processed correctly according to instructions ;

The average value filter module 2 is connected with the signal controller 1, and is used to perform average value filter processing on the data. It is mainly composed of a local window generation module and an average value calculator module. The local window generation module is a FIFO and a shift register quickly generated by the system It is used to parallelize the output of serial data. The average value calculation module is composed of an adder and a multiplier for data processing. On the basis of the improved algorithm of binary division, the sum value is calculated for the data input in parallel to the average value calculation module. , and then enter the multiplier to calculate the mean value of the parallel input data;

The parameter calculation module 3 is connected with the mean value filtering module 2, and is composed of multiple adders, subtractors, multipliers, comparators and registers, and is used to collect data and complete preliminary data processing. During the calculation process, it is connected with the addressing value module 4 Call each other to obtain the variance value and local linear coefficient value through calculation;

The addressing value module 4 is connected with the mean value filter module 2 and the parameter calculation module 3, and is used to improve the binary division algorithm, improve the operating speed of the system, and make full use of the large-capacity BIOCKRAM resource of the FPGA, which can be used to store a large amount of data and implement The characteristics of the fast lookup table function change the division operation into a register addressing process and a multiplication operation process, and provide the required data for the parameter calculation module 3 and the mean filtering module 2 by improving the generated addressing value module 4;

The state discriminator 5 is connected with the parameter calculation module 3 and is composed of a comparator, an adder, a subtractor and a register. It is based on the fact that the binary numbers with symbolic forms in the improved calculation process are not easy to distinguish and handle. It is used for A state enabling signal is generated by comparing the input data, and the driving parameter calculation module 3 and the comprehensive operation module 7 are intelligently processed according to the enabling state to reduce the computational complexity of the system;

The data buffer 6 is connected with the comprehensive operation module 7, and is composed of FIFO and shift register quickly generated by the system, and is used to coordinate the data timing and ensure that the data can be processed synchronously;

The comprehensive operation module 7 is connected with the mean value filter module 2, the state discriminator 5 and the data buffer 6, and is used for data comprehensive processing, combined with the data comprehensive processing of the data buffer 6 and the parameter calculation module 3, and the final output is filtered by a guided filter After the image data information,.

As shown in Figure 2, the FPGA-based guided filter implementation method of the embodiment of the present invention includes the following steps:

S201: According to the pipeline design method, after the local window is generated, it will gradually move to the right with the continuous input of data, and an adder is used to form a counter, and the system input control signal is marked, judged and sent status information under the control of the synchronous clock;

S202: The input image and the guide image are respectively input to the local generation window module row by row in the form of a data string under the excitation of the synchronous clock, and after passing through two FIFOs and 6 registers, the output data group 1: P1, P2...P9 and Data group 2: I1, I2...I9, use the multiplier to obtain data group 3: IP1, IP2...IP9 and data group 4: II1, II2...II9, and send the four groups of data to 4 mean calculation modules respectively, The mean value of the data group is obtained by calculation: ave_P, ave_I, ave_IP and ave_II;

S203: The four sets of data of ave_P, ave_I, ave_IP and ave_II will be obtained, and through analysis and sorting, calculation (a _k , b _k ), the addressing value is multiplied by the dividend and the reciprocal of the divisor, and the division is converted into multiplication to simplify the calculation process, Negative numbers will be generated during the calculation process, which is processed by a state discriminator;

S204: mean value filtering, used to perform mean value filtering on the obtained (a _k , b _k ), respectively, local window generation modules C and D respectively generate data group 5: a _k1 , a _k2 ... a _k9 and data group 6: b _k1 , b _k2 ...b _k9 to the mean value calculation module, and output the mean values ave_a and ave_b of the data set;

S205: output I_in_delay after the guide image I_in is cached by the data buffer;

S206: Comprehensive calculation: use the I_in_delay of the obtained data ave_a and ave_b to jointly output and output after comprehensive calculation, and the obtained data is the final image data output after being processed by the guiding filter.

Step S202 specifically includes:

Step 1, window generation:

Under the action of the FIFO controller, the data of the input image and the guide image are written into and read out from the FIFO, and the data of the (i-2) line is first stored in FIFO1 in sequence under the action of the synchronous clock, and the data is full. After that, it is stored in FIFO2 in chronological order, and at the same time, the data of line (i-1) is stored in FIFO1. When FIFO1 and FIFO2 are full of data and the i-th line of data arrives, using the principle of pipeline design, two windows are set for each line. The register is used to cache the data of the same column coordinates, so that after waiting for the window to fill up, 9 data are output in parallel at the same time, that is, data group 1: P1, P2...P9 and data group 2: I1, I2...I9 can be obtained, according to For the requirements of step 2, it is also necessary to design a multiplier to obtain data group 3: IP1, IP2...IP9 and data group 4: II1, II2...II9, and the obtained four groups of data are sent to the mean calculation module;

Step 2, mean calculation:

The mean value calculation is divided into two processes, that is, the summation process and the division operation process, as shown in formulas (7) and (8):

ave=Sum/N (7)

$\mathrm{<span\; class="notranslate">\; Sum</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Wherein, ave is the calculated mean value, Sum is the sum of all pixel values in the sliding template, and N is the total number of pixels in the template, and N is 9 in the present invention;

a, summation, according to the formula (8), sum the data groups 1-4 respectively to obtain Sum1, Sum2, Sum3, Sum4;

b. Calculating the mean value. The process of calculating the mean value is the process of FPGA processing the binary division operation. After calculating the mean value, the mean values of the 4 data groups are obtained, which are respectively ave_P, ave_I, ave_IP and ave_II.

In step S203, the calculation process for (a _k , b _k ) is as follows:

1) According to formula (3), the calculation process of a _k is decomposed and calculated in parallel as follows:

a. Molecule That is, ave_IP-ave_I*ave_P can be obtained directly through multipliers and subtractors. There is a binary subtraction operation in the numerator, which is optimized by a state discriminator. For details, see step 8;

b. The denominator σ _k ² +ε, where ε is a fixed value, and σ _k ² is the variance of the guide image in the local window. Using the definition of variance in mathematical concepts, we can know the relationship between variance and mean value as shown in formula (9):

σ _k ² =E(x ² )-E ² (X) (9)

That is, ave_II-ave_I*ave_I, a subtractor, a multiplier and an adder are required in the calculation process;

c. Utilize the principle integration formula in 2 addressing values to obtain the a _k value in the local window;

2) From the formula (3), it can be known that the calculation of b _k depends on a _k , and there is a calculation delay in the calculation process of a _k , three registers are needed to cache the data timing, and one more register is used in the calculation process of b _k multiplier and a subtractor.

In step S203, the addressing value is multiplied by the dividend by the reciprocal of the divisor, and the division is converted into multiplication to simplify the calculation process. The specific method is as follows:

When the divisor is given, its reciprocal can be obtained by calculation. Since the reciprocal of the divisor is in decimal form and its value is not greater than 1, the upper 16 bits after binarization of the decimal part are taken, together with the last 1 bit of the integer part, Form a binary data with a bit width of 17, which is recorded as the reciprocal of the divisor, so that the division function can be realized through multiplication.

In step S203, a state discriminator is designed based on the improvement of the signed binary number, and the specific method is as follows:

For example, calculate "n=A-B" and "m=n+k", where k is a constant, and A and B are input values;

1) Numerical calculation and state discrimination, compare and judge the data A and B that need to be involved in subtraction, if A is greater than or equal to B, record it as state S1, otherwise record it as S2, and at the same time calculate "n1=A-B" and " n2=B-A"; "n=n1" under the state S1 situation, "n=n2" under the state S2 situation;

2) State preservation, follow-up processing, calculation "m1=n+k" and "m2=k-n", if state S1 is "m=m1", otherwise "m=m2".

Concrete of the present invention includes as follows:

Step 1: Signal controller:

According to the pipeline design method, after the local window is generated, it will gradually move to the right with the continuous input of data. In the present invention, an adder is used to form a counter, and the system input control signal is marked, judged and sent status information under the control of the synchronous clock. ;

Step 2: Mean filtering:

In the present invention, the local generation window in the mean value filter module is a 3*3 module, the input image is P_in, and the guide image is I_in. They are respectively input to the local generation window module by row under the synchronous clock excitation in the form of data strings. After two After 1 FIFO and 6 registers, output data group 1: P1, P2...P9 and data group 2: I1, I2...I9, use the multiplier to obtain data group 3: IP1, IP2...IP9 and data group 4: II1, II2...II9, send the four sets of data to the four mean value calculation modules respectively, and obtain the mean values of the data groups through calculation: ave_P, ave_I, ave_IP and ave_II. The specific decomposition steps are as follows:

1 window generation:

As shown in Figure 3, the design of the 3*3 local window generation module in the present invention needs to use 2 FIFOs and 6 registers, because the FIFO needs to store one line of data, so the FIFO depth in the present invention is 1024, and the data bit width is 8 bits, with the characteristics of first-in-first-out. Under the action of the FIFO controller, P_in and I_in data are written into and read from the FIFO, and the data of the (i-2) row is first sequentially Store it in FIFO1, and store it in FIFO2 in chronological order when it is full. At the same time, the data of the (i-1) row is stored in FIFO1. When FIFO1 and FIFO2 are full of data and the i-th row of data arrives, use the pipeline design principle , set two registers in each row of the window to cache the data of the same column coordinates, so that after waiting for the window to fill up, 9 data will be output in parallel at the same time, that is, data group 1: P1, P2...P9 and data group 2: I1 can be obtained , I2...I9, according to the requirements of step 2, a multiplier needs to be designed to obtain data group 3: IP1, IP2...IP9 and data group 4: II1, II2...II9, and the obtained four groups of data are sent to the mean value In the computing module;

2 mean calculation:

The mean value calculation is divided into two processes, that is, the summation process and the division operation process, as shown in formulas (7) and (8):

ave=Sum/N (7)

$\mathrm{<span\; class="notranslate">\; Sum</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Wherein, ave is the calculated mean value, Sum is the sum of all pixel values in the sliding template, and N is the total number of pixels in the template, and N is 9 in the present invention;

a, summation, according to the formula (8), sum the data groups 1-4 respectively to obtain Sum1, Sum2, Sum3, Sum4;

B, seeking mean value, seeking mean value is the process that FPGA handles binary division operation, proposes improved algorithm for binary division operation in the present invention, specifically in step 3 in 2 addressing values in detail, obtains 4 data after calculating by mean value Group means, respectively ave_P, ave_I, ave_IP and ave_II;

Step 3: Variable calculation module:

1 parameter calculation:

In step 2, the four sets of data of ave_P, ave_I, ave_IP and ave_II are obtained, and after analysis and sorting, the calculation process of (a _k , b _k ) is as follows:

1) According to formula (3), the calculation process of a _k is decomposed and calculated in parallel as follows:

a. Molecule That is, ave_IP-ave_I*ave_P can be obtained directly through multipliers and subtractors. There is a binary subtraction operation in the numerator, which is optimized by a state discriminator. For details, see step 8;

b. The denominator σ _k ² +ε, where ε is a fixed value, and σ _k ² is the variance of the guide image in the local window. Using the definition of variance in mathematical concepts, we can know the relationship between variance and mean value as shown in formula (9):

σ _k ² =E(x ² )-E ² (X) (9)

That is, ave_II-ave_I*ave_I, a subtractor, a multiplier and an adder are required in the calculation process;

c. Utilize the principle integration formula in 2 addressing values to obtain the a _k value in the local window;

2) From the formula (3), it can be known that the calculation of b _k depends on a _k , and there is a calculation delay in the calculation process of a _k , three registers are needed to cache the data timing, and one more register is used in the calculation process of b _k multiplier and a subtractor;

2 addressing value:

As shown in Figure 4, the addressing value is based on FPGA’s design of an improved binary division algorithm. The dividend is multiplied by the reciprocal of the divisor, and the division is converted into multiplication to simplify the calculation process. The specific design ideas are as follows:

When the divisor is given, its reciprocal can be obtained by calculation. Since the reciprocal of the divisor is in decimal form and its value is not greater than 1, the upper 16 bits after binarization of the decimal part are taken, together with the last 1 bit of the integer part, Form a binary data with a bit width of 17, which is recorded as the reciprocal of the divisor, so that the division function can be realized through multiplication;

In the design and implementation of the FPGA-based guided filter hardware system of the present invention, a large number of division operations are involved, which increases the complexity of the design process. Considering that the divisor in the denominator in the design is an indefinite value, its numerical range is always It is kept between 0 and 255, so it is feasible to realize the division function by multiplying the reciprocal of the divisor and the dividend. This definition can be made: the divisor is used as the register address in the FPGA, and its reciprocal is used as the content of the register, so calculate the divisor The reciprocal process can be considered as a register addressing process. The principle process is shown in Figure 4. During the design process, a storage space with a bit width of 17 bits and a depth of 256 is required to store the reciprocal of the divisor;

3 state discriminator:

The calculation process of steps 3 and 6 involves binary subtraction, and negative numbers will be generated during the calculation process. Based on the improvement of signed binary numbers, a state discriminator is designed, as shown in Figure 5. The specific design ideas are as follows:

For example, calculate "n=A-B" and "m=n+k", where k is a constant, and A and B are input values;

1) Numerical calculation and state discrimination, compare and judge the data A and B that need to be involved in subtraction, if A is greater than or equal to B, record it as state S1, otherwise record it as S2, and at the same time calculate "n1=A-B" and " n2=B-A"; "n=n1" under the state S1 situation, "n=n2" under the state S2 situation;

2) State reservation, follow-up processing, calculation "m1=n+k" and "m2=k-n", if state S1 "m=m1", otherwise "m=m2";

Step 4: Mean filtering:

It is used to perform mean value filtering on (a _k , b _k ) obtained in step 3. For the specific design method, see step 2. Local window generation modules C and D respectively generate data groups 5: a _k1 , a _k2 ... a _k9 and Data group 6: b _k1 , b _k2 ... b _k9 to the mean value calculation module, and output the mean values ave_a and ave_b of the data group;

Step 5: Data buffer:

Because there is a line delay in the partial window generation stage, step 2 and step 4 respectively call the local window generation module, so four FIFOs are needed in the line buffer design, and the FIFO depth is consistent with the FIFO depth used in the partial generation window. During the design period, due to the delay caused by some formula calculations, the register set is also needed in the design of the data buffer. The design principle is shown in Figure 6. I_in outputs I_in_delay after being cached by the data buffer;

Step 6: Comprehensive operation:

As shown in formula (6), the data ave_a and ave_b obtained in step 4 are combined with the I_in_delay output in step 5 and then output. The obtained data is the final image data output after being processed by the guiding filter. During the design process, the state is used The discriminator selects the output. For the selection method, refer to Step 3 3.3 State Discriminator.

Steps 1 to 6 have introduced in detail the design idea and design process of the FPGA-based guide filter hardware design and implementation of the present invention. The specific examples can be improved according to the different needs of the user for the processing results. After the design is completed, through QuartusII11.0 The tool synthesizes and implements the system, and generates a downloadable flow file for configuring the FPGA.

FPGA-based guide filter design and realization of the present invention, used FPGA chip is CycloneIVEP4CE115F29 of ALTER company, has 114480 logical units, 3888Kbits embedded memory position, 4 phase-locked loops, register in the system, multiplier, The adder, subtractor, and buffer are all system IP cores;

The present invention is based on the algorithm principle of the FPGA-based guide filter hardware system design and implementation: the key assumption of the guide filter is the local linear model of the guide image I and the filter output q, which is assumed to be within a square window w k centered on pixel _k , q is the linear transformation of I as shown in formula (1):

${\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ,</span>{<span\; class="notranslate">\; \&ForAll;</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In a local square window w _k with radius r, (a _k , b _k ) is a constant linear coefficient, and there can be ▽q=a▽I, in order to minimize the difference between the output image q and the input image p, Use the cost function in the minimized square window, as shown in formula (2):

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; \&epsiv;\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Here ε is a regular parameter to prevent a _k from being too large, and the solution of formula (2) can be expressed in the form of linear regression:

${\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\frac{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; |</span>}\underset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}}{{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Here u _k and σ _k ² are the mean and variance of I within the square window, |w| is the number of all pixels in the square window, is the mean value of p within the range of the square window, the linear model is applied to all local windows of the entire image, and finally the output is improved by a formula (5);

${\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; |</span>}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; :</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

here $\stackrel{\&OverBar;}{a_i}=\underset{k\&Element;w_k}{\mathrm{\&Sigma;}}{a}_k,\stackrel{\&OverBar;}{b_i}=\underset{k\&Element;w_k}{\mathrm{\&Sigma;}}{b}_k;$

Due to the linear coefficient are distributed in different spaces, so the modified ▽q and ▽I are not multiplicative, on the other hand is the mean value of the filtered output, and its gradient is much smaller than the gradient of the guiding image I near the strong edge, so, That is, sudden changes in pixel brightness in the guide image I are mostly maintained in the output image q.

The present invention generally provides a parallel structure of filtering processing, which increases the speed of filtering; improves the binary division operation through addressing and taking values, facilitates hardware implementation, and improves the system running time; The subtraction algorithm improves the reliability of the system; the guided filter is implemented based on FPGA hardware, which can be applied to the fields of image processing such as denoising, enhancement, and detail smoothing.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (6)

1. the guiding wave filter based on FPGA, is characterized in that, should comprise by the guiding wave filter based on FPGA: signal controller, mean filter module, parameter calculating module, addressing value module, condition identifier, data buffer, comprehensive computing module;

Signal controller, consists of line count device and comparer, and the time monitoring factually for logarithm is analyzed data mode by the system control signal of input, and tense marker concurrent see off signal and field signal work order, guarantees according to the correct deal with data of instruction;

Mean filter module, be connected with signal controller, for data are carried out to equalization filtering processing, by local window generation module and mean value computation device module composition, FIFO and shift register that wherein local window generation module is generated fast by system form, for serial data parallelization is exported, mean value computation module is comprised of totalizer and multiplier, for data processing, in binary division, improve on the basis of algorithm, the parallel data that are input to mean value computation module are asked for to total value, then enter into multiplier, calculate the average of parallel input data;

Parameter calculating module, be connected with mean filter module, by a plurality of totalizers, subtracter, multiplier, comparer and register, formed, be used for collecting data, completing preliminary data processes, in computation process, call mutually with addressing value module, obtain as calculated variance yields and local linear coefficient value;

Addressing value module, be connected with parameter calculating module with mean filter module, for the improvement to binary division algorithm, make full use of FPGA and there is jumbo BIOCKRAM resource, for storing mass data and the feature that realizes fast finding table function, change division arithmetic into register addressing process and multiplication operation process, by improvement, generate addressing value module, for parameter calculating module and mean filter module provide desired data;

Condition identifier, be connected with parameter calculating module, by comparer, totalizer, subtracter and register, formed, that to be not easy to the situation differentiating and process set based on improving in computation process binary number with sign format, for input Data Comparison is produced to state enable signal, drive parameter calculating module and comprehensive computing module to process according to enabled state intelligent data, reduce system-computed complexity;

Data buffer, is connected with comprehensive computing module, and the FIFO and the shift register that by system, are generated fast form, and for coordination data sequential, guarantees that data can synchronously process;

Comprehensive computing module, be connected with mean filter module, condition identifier and data buffer, for aggregation of data, process, in conjunction with the aggregation of data of data buffer and parameter calculating module, process, final output is by the image data information after the filtering of Steerable filter device.

2. the guiding wave filter implementation method based on FPGA, is characterized in that, should the guiding wave filter implementation method based on FPGA comprise the following steps:

Step 1: signal controller:

According to the pipeline design method, after generating, local window can progressively move to right along with the continuous input of data, and utilize totalizer to form a counter, under the control of synchronous clock, system input control signal is carried out mark, judgement and sends status information;

Step 2: mean filter:

In mean filter module, local generating window is 3*3 module, input picture is P_in, navigational figure is I_in, form with serial data is input in row and separately local generating window module under synchronous clock excitation, after two FIFO and 6 registers, output data groups 1:P1, P2 ... P9 and data group 2:I1, I2 ... I9, utilize multiplier to obtain data group 3:IP1, IP2 ... IP9 and data group 4:II1, II2 ... II9, four groups of data are delivered to respectively to 4 mean value computation modules, by calculating, obtain respectively data class mean: ave_P, ave_I, ave_IP and ave_II,

Step 3: variable calculates:

The first step, calculation of parameter:

Obtaining ave_P, ave_I, ave_IP and tetra-groups of data of ave_II, by analysis and arrangement, to (a _k, b _k) calculate;

Second step, addressing value: based on FPGA, binary division is improved the design of algorithm, be multiplied by the inverse of divisor with dividend, transfer division to multiplication and simplify computation process;

The 3rd step, condition identifier:

In the computation process of step 3 and step 6, relate to binary subtraction computing, in computation process, can produce negative, the condition identifier based on to signed binary number is processed;

Step 4: mean filter:

For (a that step 3 is obtained _k, b _k) carrying out respectively mean filter, concrete grammar is shown in step 2, local window generation module C, D be generated data group 5:a respectively _k1, a _k2a _k9with data group 6:b _k1, b _k2b _k9to mean value computation module, output data groups average ave_a and ave_b;

Step 5: data buffer:

Step 2 and step 4 have been called respectively local window generation module, so need to use 4 FIFO in row cache design, the FIFO degree of depth is consistent with the FIFO degree of depth of using in local generating window, during the design is because the calculating of part formula can cause time delay, so also need to use register group in the design of data buffer, I_in is exporting I_in_delay after data buffer buffer memory;

Step 6: comprehensive computing:

As formula shown in, utilize step 4 to obtain exporting after the comprehensive computing of I_in_delay of the collaborative step 5 output of data ave_a and ave_b, resulting data are the final image data of exporting after guiding filter process, utilize condition identifier to select output in design process.

3. the guiding wave filter implementation method based on FPGA as claimed in claim 2, is characterized in that, the concrete steps in step 2 are as follows:

The first step, window generates:

Under the effect of fifo controller, P_in and I_in data are written into FIFO and read from FIFO, (i-2) row data are under synchronous clock effect, first deposit according to the order of sequence in FIFO1, after being filled with, deposit in chronological order in FIFO2, (i-1) row data deposit in FIFO1 simultaneously, when FIFO1 and FIFO2 are full of data and the capable data of i and arrive, utilize the pipeline design principle, the every row of window arranges two registers for the data of buffer memory same column coordinate, wait for that window fills up rear simultaneously by 9 data parallel outputs, obtain data group 1:P1, P2 ... P9 and data group 2:I1, I2 ... I9, obtain data group 3:IP1, IP2 ... IP9 and data group 4:II1, II2 ... II9, four groups of resulting data are delivered in mean value computation module,

Second step, mean value computation:

Mean value computation is divided into two processes, i.e. summation process and division arithmetic process, as shown in formula (7), (8):

ave＝Sum/N (7)

$\mathrm{Sum}=\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{I}_i---\left(8\right)$

Wherein, ave is the average calculating, and Sum is all pixel value summations in sleiding form, and N is number of pixels sum in template, and N is 9;

A, summation, according to formula (8), to data group 1-4 summation, obtain Sum1, Sum2, Sum3, Sum4 respectively;

B, average, averaging is the process that FPGA processes binary division computing, by obtaining the average of 4 data groups after mean value computation, is respectively ave_P, ave_I, ave_IP and ave_II.

4. the guiding wave filter implementation method based on FPGA as claimed in claim 2, is characterized in that, in the first step of step 3, to (a _k, b _k) computation process as follows:

1) according to formula by a _kthe parallel decomposition computation of computation process as follows:

A, molecule be ave_IP-ave_I*ave_P, directly by multiplier and subtracter, obtain, in molecule, have binary subtraction computing, take condition identifier to be optimized;

B, denominator σ _k ²+ ε, wherein ε is definite value, σ _k ²for the variance of navigational figure in local window, utilize the definition of mathematical concept to variance, can know variance and average relation as shown in formula (9):

σ _k ²＝E(x ²)-E ²(X) (9)

Be ave_II-ave_I*ave_I, in computation process, need to use 1 subtracter, 1 multiplier and 1 totalizer;

C, utilize in 2 addressing values principle to integrate formula, acquire a in local window _kvalue;

2) by formula known, b _kcalculating depend on a _k, a _kin computation process, there is calculation delay, need to use 3 register pair data time sequences and carry out buffer memory, b _kcomputation process also will be used 1 multiplier and 1 subtracter.

5. the guiding wave filter implementation method based on FPGA as claimed in claim 2, is characterized in that, in the second step of step 3, addressing value concrete grammar is as follows:

When given divisor, just by calculating its inverse, because the inverse of divisor is decimal form, and value is not more than 1, therefore get fraction part and carry out high 16 after binaryzation, together with last 1 of integral part, form a binary data that bit wide is 17, note is done the inverse of divisor, by multiplying, realizes division function.

6. the guiding wave filter implementation method based on FPGA as claimed in claim 2, is characterized in that, in the 3rd step of step 3, the condition identifier concrete grammar based on to signed binary number is as follows:

Calculate n=A-B and m=n+k, wherein k is constant, A, and B is input value;

1) numerical evaluation and condition discrimination, compare differentiation to relating to data A and the B of subtraction, remember and be state S1, otherwise note is S2 if A is more than or equal to B, meanwhile does and calculate n1=A-B and n2=B-A; N=n1 in state S1 situation, n=n2 in state S2 situation;

2) state retains, and subsequent treatment is done and calculated m1=n+k and m2=k-n, if m=m1 in state S1 situation, otherwise m=m2.