Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. Measuring Chip Seal Surface Texture with Digital Imagery Douglas D. Gransberg, PhD, PE, University of Oklahoma Bryan D. Pidwerbesky, PhD, FIPENZ Fulton Hogan Ltd. Roman Stemprok, PhD, PE, University of North Texas Jeff Waters Fulton Hogan Ltd. ABSTRACT This paper details the results of analysis of chip seal surfaces in New Zealand and the USA using digital imaging techniques. Information theory permits the derivation of an objective metric of a digital image. A two-dimensional Fourier transform of an image allows computation of the volume of information contained in that image. The information content is governed by the amount of texture present in a chip seal. Thus, a road that is badly flushed will have measurably different information content than one that is in satisfactory condition. At the present time New Zealand utilizes the sand circle test to quantify existing surface texture. Even with experienced, skilled operators, the test takes some time to perform, and is normally done in live traffic conditions with varying levels of traffic control. Correlating the image information content with a sand circle measurement taken at the same spot promises to improve the method for evaluating the condition of chip sealed surfaces by enhancing the reproducibility of the test as well as greatly reducing the time that technicians are exposed to live traffic during data collection. Skid resistance is highly dependent on macrotexture. This paper proposes that the results of this research be extended beyond mere texture measurement to the characterization of skid resistance. Keywords: Chip Seal, Digital Imagery, Surface Texture, Fourier Transform. 1 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. 1. BACKGROUND The New Zealand seal design algorithm requires texture depth of the existing surface as a key input. This texture has been measured using a volumetric technique called the sand circle test, which consists of spreading, with a straightedge in circular motion, a known volume of uniform-sized sand on the road surface, measuring the diameter of the circular area covered by the sand, and dividing the volume by the area to obtain an average texture depth. Even with experienced, skilled operators, the test takes some time to perform, and is normally done in live traffic conditions with varying levels of traffic control. Even though the reproducibility (40%) of the sand circle test is poor, it is the most common means to measure texture (Patrick et al, 2000). With the development of laser technology, numerous attempts have been made to use lasers to measure texture (such as the Mini-Texture Meter and High speed vehicle mounted lasers used in New Zealand), but as these do not generate a volumetrics-based texture, laser-measured textures cannot be used for seal design. Multiple scanning lasers could feasibly generate volumetric texture, but this would be a very expensive and costly procedure. Transit NZ has a stationary laser profilometer, which is a precise tool for measuring texture, but this device cannot be used for routine measurements of texture because of the substantial time and effort involved in setting up the device at each test site. The purpose of the research was to evaluate whether a practical method of road surface texture measurement using digital image processing, incorporating information theory and fast Fourier transform (FFT) analysis can be developed. The objectives of the research were: • To develop an accurate, repeatable method of measuring texture to replace the sand circle method, and • To develop a fast safe method of measuring texture to reduce the hazard of road surface texture measurement and minimise disruption to traffic. Similar research has been undertaken in Texas, USA, but the focus of the American research was to correlate a qualitative performance rating of the chip sealed surface pavement with a quantitative measure of texture derived from digital imagery (Gransberg et al, 2002). When a proposal was submitted to conduct experiments to correlate chip seal image FFT numbers to the measured skid resistance, the Texas highway agency was not interested in developing the concept any further. Thus, this research aims to apply the concept for measuring chip seal texture depth, for seal design purposes, in order to replace the present sand circle method of measuring texture in use in New Zealand. Road users are rapidly becoming less tolerant of travel delays caused by road works, so the research will benefit road users by substantially reducing the time involved in measuring the texture of existing surfaces. Also, society in general is placing more emphasis on worker safety, and one of the potentially most dangerous activities on the road is the current manual measurement of surface texture using the sand circle test; the proposed research aims to significantly reduce the exposure of consultants and contractors to the risk of injury and death while measuring surface texture. 2. DIGITAL IMAGING THEORY As previously mentioned, the technique used in this research project was discovered on a chip seal research project funded by the Texas Department of Transportation (TxDOT). In that project, the researchers conducted site surveys of representative chip seal sections in each of the twenty-five TxDOT Districts in conjunction with a state-wide chip seal 2 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. constructability review (Gransberg et al, 1998). District personnel were asked to pick site survey sections that typified the overall quality of the chip seals in their districts. During each of these site surveys, the condition of the roadway was recorded by taking digital camera images to document the quality of pavement condition on each section. These images not only showed the overall condition of the roadway but also showed close-up views of the shoulder, wheel path and the area between the wheel-paths. A standardized camera setup was used where the camera angle, zoom and height were kept constant in each of the images. Three of these images (shoulder, wheel path and between wheel-paths) were used to find an objective parameter that would quantify the quality level of the chip seal surface. The parameter selected was the information content of each image as calculated by a mathematical transform to be discussed later in this paper. In essence, each image has a finite amount of information contained in its boundaries. This information can be measured by determining the relative change in luminance intensity between adjoining pixels in the image. This relative difference in luminance is called the spatial frequency. For example, if the luminance intensity of one pixel is high and the intensity of the next pixel is low, the difference between the pixels is a large number, and the two pixels would be said to have a high contrast and a correspondingly high spatial frequency. On the other hand, if two adjoining pixels have luminance intensities that are nearly equal, they would have low contrast and low spatial frequencies. High contrast occurs at the boundaries between two different objects in an image (Ellis, 1976). The relative visibility of an object against its background is a function of the amount of contrast (Cuvalci, et al, 1999). Thus, in the chip seal image, the contrast is formed by the amount of light reflected off the exposed aggregate against the amount of light reflected off the background formed by the asphaltic binder (Christie, 1954). The study found that TxDOT maintenance personnel could easily discern between a satisfactory chip seal surface and an unsatisfactory one by merely looking at it (Gransberg, et al, 1998). It was also obvious to the naked eye that the difference between chip seal performance success and failure had to do with the relationship between the aggregate and the surrounding binder. Therefore, it was postulated that one could measure the surface condition by correlating the information content of a digital image and the qualitative rating of the human expert. Such an objective metric would significantly facilitate the decision-making process of allocating funds among several chip seal candidate sections on a basis of a quantitative comparison rather than qualitative comparison. The Image Processing Toolbox of MATLAB ® software (MATLAB, 2000; Tang, 1999) was utilized to process the digital images of chip seal test sections in Texas. The processing of the chip seal images consisted of filtering the information content found in the images and quantifying this filtered information. One way to filter information in such an image is detecting the edges of the aggregate particles (i.e. focusing on the boundary between the aggregate and the surrounding binder). As will be seen later, the edge patterns of flushed, stripped and satisfactory pavement surfaces exhibit a significant difference. This difference in edge patterns constituted the main analysis tool to differentiate a flushed or shelled surface from a satisfactory pavement. When a sufficiently large population is imaged and its qualitative performance rating is associated with the product of the fast Fourier transform (FFT) image processing output, a distinct difference can be seen between chip seal surfaces with satisfactory texture and those that have failed either by flushing or shelling. Figure 1 comes from the previously mentioned article that reported the proof of this concept (Gransberg et al, 2002). One can easily see the potential for associating a quantitative rather than qualitative texture rating and being able to regress the relationship between the physical texture measurement and its associated image processing output to derive a 3 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. formula that would allow the engineer to compute the texture measurement from the image output. Thus, the literature and mathematical justification for this proposed methodology must be reviewed and explained to give the reader the necessary background before moving on to the details of the current research. 0 .0 0 0 2 5 Standard Deviation 0 .0 0 0 2 0 .0 0 0 1 5 F lu sh e d & S h e lle d S a tisfa cto ry 0 .0 0 0 1 0 .0 0 0 0 5 0 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 M a x im u m fft va lu e Figure 1: 3. Normal Distribution of Maximum FFT Values for Different Textures. (Gransberg, et al, 2002) FAST FOURIER TRANSFORMS As previously stated the mathematical process that will be used in this research is called the fast Fourier transform (FFT). This approach can easily be used to quantify the information content of a digital image using a very straightforward application of information theory. The proposed approach is quite elegant in that it seeks to measure the information content of an image and then use that quantitative measure to statistically correlate with a physical texture measurement taken at the same location as the image. Thus, it quite intuitively seeks to differentiate surface texture on a basis of visual information content. As a result, the process contains a built-in check on image processing output: the ability to qualitatively confirm that images of like visual texture (i.e. satisfactory, flushed, etc.) are also yielding similar FFT numbers as well as similar sand circle measurements. Fourier analysis was initially developed by the physicist Joseph Fourier to study heat transfer problems (Goodman, 1968; Wilson, 1995; Hecht, 1975) where it recognized that any function whose graph displays a periodicity can be considered to be an infinite sum of sinusoidal functions. The Fourier series may be represented as the sum of a series of sine functions, cosine functions, complex exponential functions or any of several other sinusoidal representations (Wilson, 1995). The Fourier transform decomposes a waveform (or function) into sinusoids of different frequencies that sum to the original waveform. It 4 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. identifies or distinguishes the different frequency sinusoids and their respective amplitudes (Brigham,1988). Physical laws suggest that any conceivable object that can yield an image may always be represented by a series or by a simple or multiple Fourier integral. The amplitudes of the terms of the series or the integrand of the integral usually can be regarded as describing the spatial frequencies, which leads to a complete representation of the same object in a different domain rather than the spatial. The image obtained from chargecoupled device (CCD) cameras furnish the input for the FFT analysis and the opportunity to relate the analysis of visual output processed using the FFT with a physical measurement taken at the same location as the image. The algorithm used to obtain this spatial information from a digital image operates as follows. • The image is acquired using a CCD camera. The CCD camera is chosen for the unique properties it provides. • The acquired image is converted into a black and white image, which contains the standard range of 256 grey levels. • The image is processed and the FFT of the image is computed. • The frequency components in the FFT are segregated as shown in Figures 2. This segregation is achieved by separating the FFT into bands (regions). The frequency components start with the zeroth component in the centre pixel (Stemprok et al, 2000). • The sum of the FFT’s of the pixels local to each ring is calculated. • The sums are plotted against the frequency band using any graphing software package. • The data is ready for analysis. Figure 2: Rectangular Frequency Bands Standardized image acquisition method was carefully designed and carried out. To do so, the experiment must maintain fixed focal length, and fixed tilt of camera. When these constraints are met, constant lighting is not needed due to a camera self-adjustment (Stemprok et al, 2000). 4. IMAGE COLLECTION AND PROCESSING RESULTS A series of limited experiments were run using the imaging processing software and protocol on digital images collected on Oklahoma chip seals in September 2004 and on NZ chip seals in October 2004. The image processing output in Oklahoma was correlated with qualitative ratings of chip seal texture to ensure that the new software and hardware could replicate the process published in (Gransberg et al, 2002). The trials were successful and the researchers concluded that the experimental design would be adequate to move to the next stage of the research. 5 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. Figure 3 illustrate the outcome of these limited trials. One can see that with the exception of image condition number 6, that the FFT number computed using the ring 10 output from each of the eleven images exhibit the behaviour that was predicted by the journal article. The images of flushed chip seals seem to have the highest FFT numbers. The satisfactory textures show the lowest FFT numbers, and the seals that are experiencing a loss of aggregate fall somewhere in the middle. Even image 6 can be logically explained. It was qualitatively rated as “slightly flushed.” Thus, the amount of binder that is exposed would be greater than the amount exposed in a satisfactory texture but less than the amount that would be exposed in a fully flushed surface. Mathematically, this appears to be approximately in the same range as those images where stripping is evident and as a result, there would be patches of exposed binder and other patches of satisfactorily imbedded aggregate. Therefore, it can be confidently concluded that the experimental design is yielding results that are consistent with those predicted in the literature. Oklahoma Trial Images Ring 10 FFT Flushing 6000000 FFT # 5000000 Stripping 4000000 Satisfactory 3000000 2000000 1000000 0 0 2 4 6 8 10 12 Condition # Figure 3: Oklahoma Qualitative Image/Condition Correlation in Ring 10. There is one significant, though completely inconsequential, difference. The literature states that a satisfactory texture would yield a high FFT when compared to a flushed texture. In this experiment, that relationship is exactly reversed. This reversal is due to the use of a more robust and sensitive version of the commercial software and the graphing of the inverse of what was graphed in the literature. Therefore, the relationship has not changed. Only the directional magnitudes have been reversed. As the purpose of this research is to differentiate between chip seal textures, it does not matter which surface condition has the higher or lower FFT magnitude. What matters is that there is a consistent mathematical differential that can be measured and correlated against the physical measurement of texture. Another series of limited experiments were run using the imaging processing software and protocol on digital images collected on New Zealand chip seals in October 2004. The output from the image processing in New Zealand was then correlated with the sand circle texture 6 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. measurements taken at the same time and in the same locations as the images. Both linear and nonlinear regression models were developed and the classic statistical measurement of correlation, the coefficient of determination (also called the R-squared value) was computed. 4.1 Proof of Concept Figures 4, 5 and 6 illustrate the image processing output derived from the digital images collected in New Zealand along side the corresponding image. One can see that in the region of ring 10 there is a pronounced difference in FFT values. This graphically illustrates the importance of applying this type of analysis to the problem of chip seal texture measurement with a digital camera. Each ring exhibits somewhat different behaviour and the research team will exploit this new knowledge to enhance the ultimate accuracy of the measurement technique. Figure 4: Satisfactory texture; Grade 3 Single Chip; 175 mm Sand Circle. Figure 5: Major aggregate loss; Grade 2 and Grade 5 Multiple Chip; 120 mm Sand Circle. 7 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. Figure 6: Very heavy flushing; Grade 3 Single Chip; 300 mm Sand Circle. The Ring 10 phenomenon in the previous needs further explanation. Figures 4 and 6 are of the same size chip and were taken using the same camera at the same focal length. One can easily observe that the FFT value for satisfactory chip seal texture (Figure 4) is around 6.1 million whereas when the surface becomes heavily flushed that it drops to around 1.2 million. One can see that the sand circle measurement nearly doubles between the two images. It is also interesting to note that in Figure 5 (the image portraying aggregate loss) that the Ring 10 FFT is something less that 5.0 million. While this is a different chip seal design, thus making the comparison indirect, the concept that the FFT and hence the information content should reduce as the amount of visible aggregate-binder edge boundaries decreases is validated. 4.2 Proof of Principle Table 1 contains the information on the image/sand circle tests that were taken in New Zealand. One can see that a decent cross-section of typical New Zealand chip seals has been included in the population. Additionally, typical chip seal distresses were also included. Initially it was hoped that there would be no need to sort images out by design type as the work in Texas was not diminished by the inclusion of images that contained not only two different chip gradations, but also a combination of precoated and non-precoated chips in the sample. However, the correlations made in the Texas study were between a qualitative condition rating and the quantitative output of the image analysis. When the same approach was applied to correlating two quantitative measures for the entire sample population (i.e. the sand circle and the FFT value), the effort was less successful. Figure 7 is a graph that shows the highest correlation. This effort was only able to achieve a coefficient of determination (R2) of 0.4237, which means that the FFT accounts for only 42 % of the variation in the sand circle measurement. This is unacceptable. Though interestingly, this would put the use of the camera to measure chip seal in approximately the same range of variability as the sand circle test. 8 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. Table 1: New Zealand Trial Image Sample Population Image Sequence Number #2 #3 #5 #6 #7 #8 #9 #10 #11 #12 Design Texture Sand Circle (mm) 2-coat Grade 2 & Grade 4 West coast Grade 5 variegated colour chip 2-coat Grade 2 & Grade 5 2-coat Grade 2 & Grade 5 Single Grade 3 Single Grade 3 Single Grade 3 2-coat Grade 3 & Grade 5 Greywacke Single Grade 2 Single Grade 2 Satisfactory 145 mm Satisfactory 185 mm Minor Aggregate loss Major aggregate loss Very heavy flushing Heavy flushing Satisfactory Over chipped 150 mm 300 mm 285 mm 175 mm 160 mm Slight Flushing Satisfactory 180 mm 155 mm 120 mm Ring 3 Sand Circle vs FFT Complete Population y = -1E-11x 2 + 8E-05x + 120.48 R2 = 0.4237 SC (MM) 350 300 250 200 150 100 50 0 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 Ring 3 FFT Figure 7: Attempt to Correlate Sand Circle Measurement and FFT Value for Complete Population. Next the research team sought to explain the seeming failure of the regression analysis. The explanation must lie in the visual variety that was presented in the sample population. As the FFT value is a function of the quantity of edge boundaries present in the image, the team then tried sorting the double-chip seals from the single-chip seal. The justification being that the double chip seal, which incidentally is not used in the US, creates an image with a much higher degree of edge boundaries and may require to be correlated as a separate group. 9 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. This produced the correlation shown in Figure 8. This increased the coefficient of determination (R2) to 0.6316. Thus, the statistical correlation was improved. The population of single-chip seals included one seal that used Grade 2 chips that are larger than the Grade 3 chips used in the remainder of the population. This data point was removed and the results are shown in Figure 9. With a coefficient of determination (R2) of 0.9387, this furnished a satisfactory result and demonstrates the potential for a strong improvement in variability over the sand circle test. Ring 3 Sand Circle vs FFT y = -2E-11x 2 + 0.0001x + 93.448 R2 = 0.6316 SC (MM) 350 300 250 200 150 100 50 0 0 1000000 2000000 3000000 4000000 5000000 6000000 Ring 3 FFT Figure 8: Attempt to Correlate Sand Circle Measurement and FFT Value for Single Seals Only. Ring 3 Sand Circle vs FFT y = -3E-11x 2 + 0.0002x + 66.157 R2 = 0.9387 SC (MM) 350 300 250 200 150 100 50 0 0 1000000 2000000 3000000 4000000 5000000 6000000 Ring 3 FFT Figure 9: Correlation Sand Circle Measurement and FFT Value for Grade 3 Single Seals Only. 10 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. 5. CONCLUSIONS A number of interesting conclusions can be drawn from the research at this point. First, the literature supports the need for an accurate, reproducible method to measure chip seal surface texture. Texture is the single most important physical characteristic when it comes to pavement management involving chip seals. The seals texture is directly related to its skid resistance and additionally, this measurement is important to the design of reseals to achieve the proper aggregate gradation and bitumen amount that not only achieves the desired pavement preservation objectives but also improves the physical characteristics of the newly sealed surface. Thus, developing a method that allows Transit New Zealand and its contractors enhance the accuracy, reproducibility, and speed of the texture measurement task will accrue the benefits cited in the proposal for this project. The literature also supports the use of digital imaging and image processing using the FFT as a means to quantify physical characteristics contained in an image. The Texas study proved that this approach could be used to correlate surface condition and information content. Quantifying information content as the primary metric makes sense in that as chip seal surface deteriorate through either flushing or aggregate loss, the number of edge boundaries between the chips and the binder decrease which intuitively decreases the amount of information in the image. Correlating information content using the FFT value with a physical texture measurement is the next logical step in developing this technology for use in public roadway pavement management information systems. The results of the trials in Oklahoma and New Zealand clearly demonstrated that the merger of digital image processing and physical texture measurements is possible and the potential to successfully replace the sand circle test with a digital camera is high. The researchers were able to validate both the concept and the principle through these experiments. They allowed the team to standardize the experimental set-up and calibrate the software and hardware necessary to achieve strong correlation using non-linear regression analysis with a sorted sample population. It appears that separate models will be required for each standard chip seal design. This is due to the fact that each design creates a different average quantity of edge-boundaries between the chips and the binder. The team has also found that this phenomenon will make it more difficult to develop strong relationships for double-chip seals and those that are losing aggregate. The major issue with aggregate loss is not in the imaging technology but rather in the sand circle test where it becomes extremely difficult to accurately apply the test if the “hole” in which the aggregate is missing is relatively large. This is because the standard volume of sand can literally fail to fill “hole” and as a result no accurate area of sand can be measured. Nevertheless, the technology’s ability to accurately measure and correlate the difference been satisfactory texture and texture that is flushed is excellent. Thus, it appears that the chip seal fail condition that corresponds to a pavement surface condition that is of greatest danger to the travelling public can directly be covered by the proposed technology. The next logical stage in this research is to correlate chip seal image output against standard measurements of surface friction. This would then allow this potentially powerful technology to give engineers a fast, inexpensive means to measure surface friction and use that information to maintain and ensure the safety of public roads. 11 Measuring Chip Seal Surface Texture with Digital Imagery D. Gransberg, B. Pidwerbesky, R.Stemprok, , and J. Waters. 7. ACKNOWLEDGEMENT The authors gratefully acknowledge the research funding from Land Transport New Zealand and Fulton Hogan Ltd that enabled this research to proceed. 8. BIBLIOGRAPHY Brigham, E.O., (1974). Jersey. Fast Fourier Transform, Prentice Hall, Englewood Cliffs, New Christie, A.W.,(1954). “Reflection Characteristics of Pavement Surfaces,” Highway Research Board Bulletin No. 89,21-37. Cuvalci, O. D. D. Gransberg, B. Green and C. Nuhrat, (1999). "Visibility Measurement Technique Using Photometric Images," I999 Annual ASEE Conference Proceeding, Session: 3659, Charlotte, North Carolina, <http://www.asee.org>. Ellis, King L., (1976). “Measurement of Directional Reflectance of Pavement Surfaces and Development of Computer Techniques for Calculating Luminance,” Journal of the IES, 5(2), 118-126. Goodman, J.W.(1968) Introduction to Fourier Optics, McGraw-Hill, New York. Gransberg, D.D., S. Senadheera, and I. Karaca, (1998). Analysis of Statewide Seal Coat Constructability Review, Texas Department of Transportation Research Report TX- 98/01787-1R, Texas Tech University, Lubbock, Texas. Gransberg, D.D., I. Karaca, and, W.R. Burkett, (2002), “Quantifying Seal Coat Surface Condition Using Digital Image Processing Based On Information Theory,” International Journal of Pavement Engineering, Vol. 3 (4), pp 197-205. Hecht, E. (1975), Schaum’s Outline Series Theory and Problems of Optics, McGraw-Hill, New York. “MATLAB ® Image Massachusetts. Processing Toolbox” (1997). The MathWorks, Inc., Natick Patrick, J.E., Cenek, P.D., and Owen, M. (2000), Comparison of Methods to Measure Macrotexture. Proc. 1st Int’l Conf. World of Asphalt Pavements on CD, Australian Asphalt Pavement Association, Sydney Stemprok, R., Green, B. and Tang, Z (2000) “Calculated Roadway lighting errors and Measurement of Visibility with CCD devices,” Proceedings of the Symposium of Commission Internationale De L’Eclaire, (CIE), Division 4 and 5 meeting, Toronto, Canada. Tang, Z. (1999). “Digital Image Processing and Spatial Frequency Analysis of Texas Roadway Environments,” Master’s Thesis, Texas Tech University, Lubbock, Texas. Wilson, R.G.(1995). Fourier series and Optical Transform Techniques in Contemporary Optics, an Introduction, John Wiley & Sons, Inc.; New York. 12