Hindawi

Journal of Advanced Transportation

Volume 2018, Article ID 8647607, 10 pages
https://doi.org/10.1155/2018/8647607

Research Article
Characterization of Road Condition with Data Mining Based on
Measured Kinematic Vehicle Parameters

Johannes Masino ,1 Jakob Thumm,1 Guillaume Levasseur,2 Michael Frey,1

Frank Gauterin,1 Ralf Mikut,3 and Markus Reischl3
1
Institute of Vehicle System Technology, Karlsruhe Institute of Technology, Germany
2
Artificial Intelligence Research Laboratory, Université Libre de Bruxelles, Belgium
3
Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology, Germany

Correspondence should be addressed to Johannes Masino; johannes.masino@kit.edu

Received 31 July 2018; Accepted 30 September 2018; Published 22 October 2018

Guest Editor: Petru Andrei

Copyright © 2018 Johannes Masino et al. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.

This work aims at classifying the road condition with data mining methods using simple acceleration sensors and gyroscopes
installed in vehicles. Two classifiers are developed with a support vector machine (SVM) to distinguish between different types of
road surfaces, such as asphalt and concrete, and obstacles, such as potholes or railway crossings. From the sensor signals, frequency-
based features are extracted, evaluated automatically with MANOVA. The selected features and their meaning to predict the classes
are discussed. The best features are used for designing the classifiers. Finally, the methods, which are developed and applied in this
work, are implemented in a Matlab toolbox with a graphical user interface. The toolbox visualizes the classification results on
maps, thus enabling manual verification of the results. The accuracy of the cross-validation of classifying obstacles yields 81.0% on
average and of classifying road material 96.1% on average. The results are discussed on a comprehensive exemplary data set.

1. Motivation accelerations of unsprung masses and accelerations and

rotations of the vehicle body [5]. To monitor road sections,
In 2006, bad conditions of road infrastructure were one of a vehicle can be used as a mobile sensor platform that records
the causes of 50% of fatal accidents in France [1]. In 2016, four both vehicle dynamics and the environment, such as the
accidents in Germany were caused exclusively by road surface road state. The road state can be estimated using cameras
damage [2]. Road traffic authorities are aimed to improve and or inertial measurement units to record rotation speeds
automate monitoring the road state to detect and repair road and accelerations of the vehicle. Such sensors are already
damages to enhance the safety of road traffic. Based on the integrated into modern vehicles having an active or adaptive
detection results, specific and cost-optimized maintenance of body control or new lighting systems. Inertial sensors are
roads can be ensured. Furthermore, suppliers of navigation even part of the standard equipment of new vehicles and data
systems can profit from the available information of the road can be fused with GPS data for more accurate positioning,
state, because roads in bad condition may be neglected in an example being the new Audi A7. Previous studies revealed
route planning [3]. Automotive manufacturers can use the that measurements made by these sensors allow for the
collected data to control adaptive vehicle suspensions and to derivation of road features, such as potholes or mends of
display warnings in real time [4]. asphalt roads [6–8].
Contrary to physical modeling, data-based estimation The inertial sensor is inexpensive and part of the standard
of the road state does not require any comprehensive equipment of many vehicles. Its data include information of
system characterization, such as vehicle, road, sensor, and minor unevenness of road surfaces that causes the vehicle
environment. Moreover, modeling of a full vehicle requires to vibrate. Inertial sensors, however, only provide data on
five acceleration sensors or gyroscopes to measure vertical the road section just crossed. Cameras, by contrast, record
2 Journal of Advanced Transportation

the complete road section in front of the vehicle, including Time Evalua-
the neighboring lane. However, they are integrated into high- Classification Visualization
series tion
class vehicles only. Cameras currently used in vehicles are of
limited accuracy and can detect potholes with a minimum Parameter
depth of about 3 cm only.
Data Mining
Presently, the state of motorways is measured automat-
ically using expensive and complex measurement vehicles,
while that of roads in urban and rural areas is determined Ground truth
manually [9]. These methods are associated with a high Road Time Reference Evalua-
Visualization
expenditure. Due to manual evaluation, it takes a long time measurement series data set tion
until the road network quality is updated. Safety-relevant
damage may be detected too late. This may have severe Figure 1: Overview of method.
consequences, such as traffic accidents or cost-intensive and
complete renewal of the road.
For road maintenance, some countries determine the 2. Methods
stochastic road profile depth or international roughness
index (IRI), as outlined in [10]. However, the latter is often 2.1. Design. Figure 1 presents an overview of the method to
calculated for 100 m intervals only. As a result, certain evaluate the road state [15]. In a first step, the road state is to
obstacles, such as potholes, are not detected. In countries be measured by suitable sensors. For measurement, acoustic
pursuing a systematic road maintenance scheme, not only the sensors, such as the sensors described in [16], acceleration
IRI but also individual obstacles are measured. This also is the sensors and gyroscopes, cameras, and similar devices, can
objective of the present study. be used. As a result, several synchronized time series will be
Approaches to automatic road state monitoring using obtained. To obtain a representative reference data set, sensor
inertial sensors exist, e.g., [6, 11–13]. They only concentrate data have to cover a maximum of framework conditions, e.g.,
on single road features (such as potholes), do not have any variations of external temperature, driver, and speed. Every
representative dataset, or are based on data measured under point of time/road section has to be assigned a label, e.g. type
restricted conditions, e.g., in speed limit areas or on certain of road surface, simultaneously or afterwards. In this way, a
sections only. Moreover, the validation phase only covers data set with correct allocations of sensor data to labels is
checks as to whether the road damage detected actually is obtained (ground truth). By means of data mining, models
damage or not (true or false positives), but not whether road can be designed (offline) for retrospective evaluation (offline)
damage was overseen (false negatives). or classification during driving operation (online). The results
Road construction offices also need to know the material of the classification models then have to be visualized and
(road surface), as repairs on different surfaces produce evaluated on the basis of map material. To estimate the
different results and may cause different types of damage [14]. information on the road surface and event or damage that
It is also important to distinguish between safety-relevant is of relevance to road construction offices, two separate
damage that has to be repaired within 24 hours and damage classification routines have to be developed.
that is not relevant to safety and the repair of which can be
planned and postponed. 2.2. Data Acquisition. The data measured by the sensors
The main contribution of this paper is to evaluate the installed in the vehicle, e.g., GPS and inertial sensors, are
principle feasibility of automatic road surface and road encoded on the CAN bus and cannot be read without
damage measurement with an inertial sensor in the vehicle the communication matrix that is available to the control
body. Therefore, this work is aimed at system developer and automotive manufacturer only. Hence,
an inexpensive measurement system similar to the inertial
(i) designing a processing chain to evaluate road data sensor incorporated in the vehicle is proposed for the easy
based on measurements of inertial sensors, measurement and readout of data. Measurements cover the
position and dynamics of the vehicle, in particular vertical
(ii) automatically recording an adequate dataset, dynamics caused by unevenness [17]. In addition, the data
(iii) developing and evaluating a method to estimate road may be assigned labels during measurement already. The
surfaces and damage, and at measurement system (Figure 2) mainly consists of a GPS
receiver (Adafruit ultimate GPS Hat) and a MEMS inertial
(iv) integrating the algorithms developed into a graphic sensor (LSM9DS1) measuring accelerations and rotation rates
user interface for evaluation of datasets with alterna- of the vehicle along all three axes. The sensor data are
tive parameterizations by nonexperts as well. acquired using a Raspberry Pi and stored as a csv-table in
fused form. As soon as the engine of the vehicle is turned off,
The methodology will be presented in Section 2. Section 3 the UPS is activated and data can be transmitted via WiFi to a
will outline the implementation derived, while Section 4 central data base, if the Raspberry Pi is connected to a known
will explain the results based on a first dataset. The result, WiFi network.
its applicability, and open problems will be discussed in The GPS receiver has a sample rate of 10 Hz, a position
Section 5. resolution of 3 m, and a speed resolution of 0.1 m/s. As the
Journal of Advanced Transportation 3

measurement system is installed near the static center of mass

of a BMW 116d in the console between the driver’s seat and
passenger’s seat. Orientation of the sensor axes corresponds
to the vehicle axes according to ISO 8855:2011. For the method
to be generally applicable, measurement data are recorded on
randomly selected roads in the region of Karlsruhe, Germany.
The speed, road condition, and environmental conditions
(e.g., measurement drives in good and rainy weather) are
varied strongly. In total, reference data are recorded for a
period of three months on a distance of more than 200 km.
The data are recorded on eight days (three times a whole
Figure 2: Measurement unit, consisting of Raspberry Pi, inertial day) by three different drivers. Acquisition of reference data
sensor, Adafruit ultimate GPS Hat, UPS Hat, and two buttons for is a time-consuming process, as the materials and events
labeling the data. In service, the inertial sensor is fixed under the have to be crossed under variable environmental conditions
Raspberry Pi on the bottom of the case. The case is cut open for
and at variable speeds. In particular, individual events, such
visualization purposes.
as potholes, of various types have to be found in the road
network and crossed several times with variable approach
Table 1: Code and corresponding name of the classes for material
angles and vehicle tracks.
and event. Lacking GPS data due to variable scanning rates are
reconstructed by linear interpolation. As the measurement
𝑚-Material 𝑒-Event series are not recorded at a constant sample rate, resampling
0- Unknown 0- Unknown is required. By resampling, the data are converted from the
1- Smooth surface 1- Good time domain (s−1 ) to the space domain (m−1 ). In [18, 19], it
2- Damaged asphalt 2- Light damages was shown that the response of the vehicle to the excitation of
3- Damaged concrete 3- Pothole the road depends on speed and that presentation in the space
4- Cobblestone 4- Manhole cover domain reduces this effect. All-time series are resampled
5- Railway crossing with a (spatial) frequency of 100 m−1 . The section driven is
6- Speed bump
calculated from the time stamp and speed with the help of
the implicit Euler method. Calculation via GPS would also
be possible but lacks precision.
The classes of materials 𝑚 and events 𝑒 are encoded by
inertial sensor is a low-cost MEMS sensor, the sample rate is natural numbers (Table 1). Light damages are general types of
not uniform. This has to be compensated by a filter in data unevenness, which are not safety-relevant and include minor
processing. The sample rate is about 220 Hz. The accuracy faults and repairs. Manhole cover, railway crossing, and speed
of the acceleration sensor is 0.05 m/s2 and of the gyroscope bump are construction obstacles. A speed bump is defined as
0.003 degree/s. Without data transfer, a 32 GB memory card an elevated construction transverse to the driving direction.
can record data for up to 1000 h. The pothole represents a fault of at least 2 cm in depth. The
For allocating labels to data, different approaches are latter event is safety-relevant and should be repaired within a
presented in literature. For example, a microphone records maximum of 24 h. For every sample point 𝑘 of the reference
the (road) damage report of the passenger [13]. This method, data set, two labels are annotated for the material 𝑚[𝑘] and
however, is subject to several deficiencies. Among others, the event 𝑒[𝑘].
labels are recorded much later than the actual road damage
and the soundtrack is not synchronized with the sensor data. 2.3. Signal Processing
Reference [6] uses “loosely labeled” training data. Here, only
the number of classes but not the exact position is recorded 2.3.1. Overview. To derive information on the road surface
for large road segments. or material and event/damage from the sensor data recorded,
The measurement system developed for this study is the data streams first have to be transferred to a feature
based on two buttons. A pressed button annotates the damage space. Feature extraction calculates representative and useful
class (event) or the change of road surface (material). For individual features from complete or partial measurement
every measurement drive, data with a certain material (e.g., series. Without knowing the physical model of effects of
if asphalt 1, otherwise 0) and an event (e.g., if pothole 1, asphalt changes or road damage on the sensor, it is recom-
otherwise 0) are recorded. After the measurement drives, mended to calculate a large set of features and to check their
binary coding of the data of the respective files is transformed suitability for the classification problem based on data with
into the coding given in Table 1. the corresponding labels (ground truth). Efficient feature
The unix time 𝑡, ID for the sensor, speed V, position calculation is needed for calculation on mobile devices (e.g.,
and time stamp of the GPS 𝑙𝑎𝑡, 𝑙𝑜𝑛, and 𝑡𝐺𝑃𝑆 , accelerations microcontrollers).
𝑎 and rotation rates 𝜔 along all three axes, and the two
labels for the event 𝑒 and material 𝑚 are recorded and stored 2.3.2. Generation of New Time Series. To describe the road
in a csv-table on the measurement system (Figure 2). The state, acceleration in vertical direction (𝑎𝑧 ) and rotation
4 Journal of Advanced Transportation

speeds in longitudinal and transverse direction (𝜔𝑥 and 𝜔𝑦 ) Of the GPS latitude 𝑙𝑎𝑡 and longitude 𝑙𝑜𝑛 time series, the
are very important [6]. Furthermore, the roll and pitch medians in every window are used for later visualization.
acceleration (𝜔̇ 𝑥 and 𝜔̇ 𝑦 ) as well as the jerk (𝑎𝑧̇ ) of the vehicle
are done using the derivation of the vertical acceleration 2.3.4. Classification. Based on the extracted individual fea-
in time domain. The space series data of the vertical roll tures and the corresponding labels, two classifiers are
and pitch acceleration is transformed into frequency domain designed for material and event. For the design and appli-
with the short-time Fourier transform, which contains the cation of classification, a combination of feature selection,
short-term distance-localized frequency content of the signal. feature aggregation, and classifier is chosen.
Hereby, features based on specific frequency bands can For the surface classification, the five best individual
be investigated. Hence, the three distance series data are features each are determined using the multivariate analysis
extended by the following data streams, which lead us to 7 of variances (MANOVA) method, for event classification the
data streams in total: ten best features are selected. For visualization purposes,
(i) vertical acceleration, the selected individual features are then aggregated to two
features using linear discriminant analysis (DA), which can
(ii) roll acceleration, also minimize the calculation expenditure. A support vector
(iii) pitch acceleration, machine (SVM) classifier with polynomial kernel function
(iv) deviation of vertical acceleration, with order 2 is used. Validation is carried out with the help
of cross-validation with 5-folds.
(v) short-time Fourier transformed vertical acceleration,
(vi) short-time Fourier transformed pitch acceleration 2.3.5. Performance Measures. From the correct and false
and predicted instances, we can calculate a confusion matrix 𝑀 =
(vii) short-time Fourier transformed roll acceleration. (𝑚𝑖𝑗 ) ∈ N𝑘×𝑘 for classes 𝐾𝑖 , 𝑖 = 1, . . . , 𝑘. In the confusion
matrix, 𝑚𝑖𝑖 presents the true positives for class 𝑖. The other
2.3.3. Feature Extraction. The features are calculated for elements in column 𝑗 are called false negatives, in row 𝑖 false
windows with a specific length in distance domain and a positives and in the diagonal true negatives.
specific overlap. A window 𝑈𝑖 denotes all indexes [𝑛𝑖 −𝐿/2, 𝑛𝑖 + From the confusion matrix, one can calculate multiple
𝐿/2 − 1] with the running index 𝑛, window index 𝑖, and performance measures to evaluate the model, such as recall
window length 𝐿. The window overlap 𝑟 corresponds to those with 𝑚𝑖𝑖 / ∑𝑛𝑗=1 𝑚𝑗𝑖 for class 𝐾𝑖 , the overall accuracy of
values from the window 𝑈𝑖 that are contained in the previous the classifier with ∑𝑛𝑖=1 𝑚𝑖𝑖 / ∑𝑛𝑖=1 ∑𝑛𝑗=1 𝑚𝑖𝑗 , or the precision
window 𝑈𝑖−1 , i.e., 𝑛𝑖+1 = 𝑛𝑖 + 𝐿(1 − 𝑟). If a longer distance is 𝑚𝑖𝑗 / ∑𝑛𝑗=1 𝑚𝑖𝑗 = 𝜋𝑖𝑗 . The precision presents the fraction of
chosen, short amplitudes, for example, due to potholes, have
retrieved instances that are relevant and can be seen as the
a weaker impact on the value of features, which incorporate
probability 𝜋𝑖𝑗 of the classifier to predict class 𝑖 as class 𝑗 for
the overall signal, such as the standard deviation. These short
𝑖, 𝑗 = 1, . . . , 𝑙. An overview for performance measures for
amplitudes can be captured by shortening the window size or
different calculation problems can be found in [20].
using features, which calculate extrema.
For the feature extraction for material and events we use
window sized of 50 m or 5 m, respectively, and an overlap of 3. Implementation
20%.
From the distance series data, we calculate the standard To facilitate operation by non-experts, the methods are
deviation as well as peak-to-peak. The root mean square implemented in a graphical user interface called Vehicle
value or effective value for specific frequencies and the Learner Toolbox, which is available in [21]. It is based on
spectral centroid is extracted from the short-time Fourier Matlab and implements several machine learning opera-
transformed data streams for the following spatial frequency tions of the freely available toolbox SciXMiner [22] (formerly,
bands (1 m−1 ): Gait-CAD [23]). The Vehicle Learner Toolbox provides the
possibility to
[0.1, 0.5] [0.5, 15] [15, 20] [0.1, 25] [0.1, 50] (1)
(i) import vehicle sensor data in different file formats,
The vehicle velocity has a strong sensitivity on the vehicle (ii) compress the imported data and automatically extract
vibration. Previous research suggests performing a linear various features,
regression with each feature as the dependent variable and
the velocity as the independent variable [12]. The velocity (iii) train a classifier model with a wide-ranging set of
dependency is then removed by subtracting the estimated options,
linear equation from the corresponding feature. However, the (iv) test the trained classifier with a test set,
vehicle vibration and the extracted features are not linear (v) visualize the results with the help of plots and maps.
dependent on the velocity. The dependent parameters are
incorporated and the mean velocity is calculated for each A project folder can be selected and sensor data can
window as additional feature. To allow nonlinear relation- be imported in the corresponding frame Data (Figure 3).
ships a kernel function of higher order can be applied for the There is the option to assign the sensor data to specific
classification. vehicles, since they vary in suspensions, damping, and other
Journal of Advanced Transportation 5

Figure 3: The import data frame of the Vehicle Learner Toolbox. Figure 4: The train classifier frame of the Vehicle Learner Toolbox.
Different vehicles can be chosen and the data type can be set. Multiple classifiers can be trained with a wide range of options.

parameters, which have an impact on the vibration behaviour.

Furthermore, the parameters for the window profile,
Therefore, in the following data processing, feature selection
and classification can be performed for data from specific such as length of road segments and overlapping factor of
vehicles. The import allows .csv and .xlsx file format with the these windows, can be determined, as well as the resampling
frequency. The standard window profiles are material with a
following column headers:
window length of 50 m and event with a window length of
(i) timestamp (unix timestamp) 5 m.
After the import and preprocess of the data, new time
(ii) x-, y-, z-accel (the acceleration values in each direc-
series data are calculated and features are automatically
tion)
extracted, as proposed in Sections 2.3.2 and 2.3.3. The code
(iii) x-, y-, z-gyro (the gyroscope values in each direction) to calculate new data series or features can be easily added in
(iv) gps-timestamp (format: YYYY-MM-DDThh:mm:ss, the corresponding Matlab function.
000Z) The proposed data mining methods (Section 2.3.4) can
be applied in the toolbox under the menu Supervised Learn-
(v) lat, lon (position in latitude and longitude) ing (Figure 4). In the first step, a training data set must
(vi) speed (in m/s) be generated. There are two different ways to accomplish
(vii) m, e (material 𝑚 and event 𝑒 labeling, if the data is not this. Either an external data set containing features can be
labeled, these columns should only contain zeros). imported or the imported data within the toolbox can be
used and modified by choosing the time interval of the data
Since the GPS data is acquired with a lower sample rate acquisitions or the area. Furthermore, data annotated with
compared to the inertial sensor, these data are automatic specific labels can be excluded from the classification. For
interpolated. Furthermore, the sensor signals are subject to our example, all data with labels 0-unknown were deleted
noise [24] and are automatic smoothed during the import (Table 1). Another option is to thin out classes with significant
process with the following filters. Despite the noise of a more data points than other classes to allow an approximately
MEMS gyroscope visible as spikes in the signal, it is well uniform distribution of data points among the classes to pre-
known for its good accuracy in short term [25]. A suitable vent over-fitting of specific classes. Furthermore, systematic
filter for this purpose is the median filter, which is robust errors during labeling the data can be removed; e.g., if the
against outliers and removes noise while preserving high fre- trigger to annotate the data was activated too early or too
quency content. Since the data from the MEMS accelerometer late the annotation can be moved or data points with the
do not show such spikes but contains more noise in the short wrong annotation can be excluded. After the generation of
term [25], a Savitzky-Golay FIR smoothing filter is applied. the data set to be processed, the settings for the classifier can
It fits a polynomial of a specified degree to frames of noisy be determined under the tab Train Classifier (Figure 4).
data and minimizes the least-squares error [26]. Therefore, In the first step the vehicle and the training data set
the filter outperforms standard averaging FIR filters, which must be set. Afterwards a new classifier model can be created
might remove high frequency content with the noise. or an existing model can be selected. The next section
There is also the possibility to import tire cavity sound contains the settings of feature selection (e.g,. MANOVA)
data along the inertial sensor data for road roughness esti- and aggregation (e.g., discriminant analysis), as proposed
mation, as presented in [16, 27], but is not substance in this in Section 2.3.4. Reducing the amount of features highly
paper. Moreover, the imported data set can be categorized as influences the classification result by reducing the chances for
training, testing or unlabeled data. overfitting. It is possible to cross-validate the training process
6 Journal of Advanced Transportation

1 1

0.8 0.8

aggregated feature 2
aggregated feature 2

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

aggregated feature 1 aggregated feature 1

smooth damaged asphalt potholes manholes

damaged concrete cobblestone railway crossing speed bump
light damages good
(a) Classification of material 𝑚 (b) Classification of events 𝑒

Figure 5: Classification results with two aggregated features and borders of the classifier in black.

by setting the k-folded-cross-validation value to higher than (i) peak-to-peak of pitch acceleration
1. The last section offers a variety of settings for the classifier, (ii) peak-to-peak of roll acceleration
e.g. for a SVM, including the kernel function and penalty
term. Afterwards, the classifier can be trained and data can (iii) maximum of jerk in vertical direction
be plotted on open street maps. Furthermore, the confusion (iv) root mean square (RMS) of the vertical acceleration
matrix and the total loss is shown in the Matlab console. (v) speed
For testing new data, a data set with modifications in time
range and area to be analyzed can be generated as described By comparing each class with each other, it emerges that
for training, and a trained classifier must be selected. If the the peak-to-peak value of pitch and roll acceleration are
test data set is labeled, the output of the prediction is again mainly responsible to separate events, which occur on
a confusion matrix and the classification error. Moreover, the
results can be visualized and plotted on open street maps, as it (i) both vehicle lanes (railway crossing, speed bump),
will be presented in Section 4. The trajectories will be cut into (ii) on only one side of the vehicle (manhole cover,
segments of different color referring to the corresponding pothole),
classes, which are predicted. (iii) or have only little impact on the vehicle vibration
(light damages, road segments in good condition).

4. Results In addition, the average RMS of the vertical acceleration

is important to separate light damages and road segments in
4.1. Event Classification. The accuracy of the cross-validation good condition. Furthermore, potholes and manhole covers
of classifying events yields 81% on average without feature are dividable through the maximum RMS of the roll accel-
aggregation. The aggregated feature space and the lines of the eration for the frequency range 15 to 25 m−1 . However, latter
function to classify the events is shown in Figure 5(b). events are often misclassified as segments in good condition
The illustration of the classification shows that road or light damages. Speed bumps and railways crossings are
segments in good condition, with light damages, speed bumps, separable by the value of the peek-to-peek of the pitch rate,
and potholes, can be separated well. This indication is proofed whereas railways crossings are also often misclassified as light
by the quantitative results, listed in Table 2. damages.
The precision and recall for the mentioned classes is above To test the classifier, a data set of more than 200 km of
70%, whereas the performance measures for manhole cover street data is classified and plotted on open street maps. The
and railway crossing is below 62% on average. results are promising and represent the actual street condition
The most important features, determined with in many occasions. A few examples of classified areas are
MANOVA, are shown below.
Journal of Advanced Transportation 7

Table 2: Results of single features classification.

Precision Recall Precision Recall

Material Events
(% ± SD) (% ± SD) (% ± SD) (% ± SD)
smooth surf. 95.8 ± 0.7 97.6 ± 0.7 good 83.9 ± 1.5 92.4 ± 2.7
damaged asph. 97.3 ± 2.0 94.9 ± 1.4 light dam. 78.5 ± 3.8 70.8 ± 3.3
damaged con. 95.6 ± 0.8 92.9 ± 1.1 pothole 85.3 ± 6.6 77.4 ± 7.2
cobblestones 99.1 ± 1.8 100.0 ± 0.0 manhole 40.9 ± 13.6 29.1 ± 15.7
railway cro. 61.6 ± 4.9 57.0 ± 7.9
speed bump 94.9 ± 7.1 91.0 ± 12.2
Average 96.4 ± 0.8 96.4 ± 0.8 74.2 ± 6.3 69.6 ± 8.1
Accuracy 96.1 ± 0.4 81.0 ± 1.7

1
2 1

good potholes
light damages railway crossing

Figure 6: Event classification results of two high speed roads in the

south west of Karlsruhe, Germany.
good potholes
light damages speed bump

The first example shows the event classification results Figure 7: Event classification results for road segments in the city of
on two different high speed roads (Figure 6). The upper Karlsruhe, Germany.
one with Label 1 is a freshly renovated asphalt highway with
close to no damages and the lower one with Label 2 is a
poorly patched asphalt road with a lot of medium and severe
damages. The classification successfully predicted the upper
roadway as good street. Most parts of the lower street were
predicted as light damage and some points even as potholes.
The results represent the road condition very accurate. The
only noticeable misclassification is railway crossing that was
predicted once (Label 3). 1
The second example presents data acquired in an urban
area in Karlsruhe, the predictions are shown in Figure 7. The 2 3
roads in this area are poorly preserved and there is a speed
bump at a pedestrian crossing (Label 1). The classification
model correctly predicts the speed bump (Label 1) for all
overdrives and a pothole (Label 2) on both driving directions.
The third interesting sector is shown in Figure 8. Potholes good speed bump
(Labels 2 and 3), which were at the edge of the driving line, light damages railway crossing
were overdriven multiple times and the classifier predicts the potholes
severe damage accordingly. Sometimes the output at the road
Figure 8: Event classification results of road segments outside of
segments is not pothole but light damages or even good road
Karlsruhe, Germany, for the events potholes and railways crossing.
condition. The reason might be that the pothole was avoided
by the driver.
8 Journal of Advanced Transportation

3
1
2

smooth asphalt damaged concrete

damaged asphalt cobblestones

Figure 10: Material classification of cobblestone and road segments

in urban area.

The material classifier was applied to the same data set

described for event classification. The classifier was able to
reflect the road surface very precisely. The following figures
display the performance on different surfaces. Analog to the
2 event classification shown in Figure 6, the material classifier
could distinguish between both roads and correctly classified
them as smooth surface and damaged asphalt, respectively.
smooth asphalt damaged concrete In contrast, Figure 9 shows a long highway segment with
damaged asphalt cobblestones aged concrete and distinctive concrete joints, which have to
be maintained shortly. Except for one short segment, which
Figure 9: Material classification of an aged concrete highway. was classified as smooth surface, the road state was correctly
predicted.
The classification results of data acquired in the urban
The railway crossing (Label 1) is more elevated than other area of Karlsruhe (Figure 10) show two correctly predicted
crossings and miss-classified as speed bump in few cases. areas of cobblestone (Label 1 and 2). The remaining road
segments are correctly classified as segments with light
4.2. Road Surface Classification. When classifying road sur- damages or in good condition. Especially latter class was
faces, cross-validation yields 96.1% accuracy on average correctly predicted for a road segment, which was recently
without aggregation of features. The aggregated feature space renewed (Label 3). One miss-classification of cobblestone can
is shown in Figure 5(a). The figure indicates, that the be found close to Label 1. However, this road segment is
misclassifications are asphalt classified as damaged asphalt highly damaged with multiple potholes, which have a high
or damaged concrete and vice versa. The illustrated results impact on the vehicle vibration similar to cobblestones.
are underlined by Table 2, where the precision and recall
for cobblestone is above 99.0%, whereas the performance 5. Conclusions and Outlook
measures for asphalt, damaged asphalt, and damaged concrete
are between 92.0 and 97.6% percent. The results show that the system presented in this paper can
The three best individual features for the classification of classify both road materials and events. The features selected
road surface according to MANOVA are by MANOVA are in agreement with the theory of vehicle
excitation. Material classification performs well according to
(i) RMS of the roll acceleration for frequency range from the results of the cross-validation and the test data.
5 to 15 m−1 on average There are multiple miss-classifications of the prediction
of events, especially for structural obstacles, such as manhole
(ii) standard deviation of the pitch rate covers and railway crossing. However, these events might
(iii) stand deviation of the RMS of the vertical acceleration be marked on a map and excluded from classification and
for frequency range 15 to 25 m−1 investigation, as the main objective is to detect road damage.
One reason for misclassifications of the events good
The values of RMS of the roll acceleration and vertical condition, light damages, and pothole might be false manual
acceleration separate the classes smooth surfaces, damaged annotating, since there is sometimes only a fine line between
asphalt and cobbled stone. The values are greatest for cobbled the degree of damages, or the events were not fully over-
stone and low for smooth surface. driven, especially for potholes.
Standard deviation of pitch rate separates the classes As the system is of modular design, the number and
damaged concrete from all other classes. The reason are type of sensors and sensor modality can be varied. When
probably poor and aged concrete joints. adapting feature extraction, also camera recordings might be
Journal of Advanced Transportation 9

useful. Transferability to other vehicles with different chassis Services (MobiSys ’08), pp. 29–39, Breckenridge, CO, USA, June
and dimensions has not been examined so far. Presumably, 2008.
the algorithms of parameters are adapted to the vehicle with [7] C. Koch and I. Brilakis, “Pothole detection in asphalt pavement
which the learning data set was recorded. Here, fusion of images,” Advanced Engineering Informatics, vol. 25, no. 3, pp.
learning data sets from several vehicles and an accordingly 507–515, 2011.
adapted classification routine might help. It can be assumed [8] S. C. Radopoulou and I. Brilakis, “Patch detection for pavement
that the results will be slightly worse. assessment,” Automation in Construction, vol. 53, pp. 95–104,
Generally, the inertial sensor represents a very good 2015.
option to collect information on the tire/road contact at low [9] S. C. Radopoulou and I. Brilakis, “Improving Road Asset
costs and over wide areas. Use of information of several vehi- Condition Monitoring,” pp. 3004–3012.
cles can compensate the drawback of some drivers passing [10] ISO, ISO 8608: Mechanical Vibration - Road Surface Profiles
- Reporting of Measured Data, International Organization for
by safety-relevant damage that, hence, is not measured by the
Standardization, 2016.
sensor. Moreover, obstacles at the roadside are not crossed
[11] K. Chen, G. Tan, M. Lu, and J. Wu, “CRSM: a practical
and, hence, cannot be detected.
crowdsourcing-based road surface monitoring system,” Wire-
Fusion of camera and inertial sensor data probably would less Networks, vol. 22, no. 3, pp. 765–779, 2016.
be the optimum solution for a mobile determination of the [12] M. Perttunen, O. Mazhelis, F. Cong et al., “Distributed road sur-
state of road traffic infrastructure. For road construction face condition monitoring using mobile phones,” in Proceedings
offices, use of a low-cost and computationally efficient system, of the International Conference on Ubiquitous Intelligence and
consisting of an inertial sensor, Raspberry Pi, and simple Computing, pp. 64–78, Springer, 2011.
signal processing, is sufficient and can be recommended. [13] F. Seraj, A. Dilo, T. Luarasi et al., “RoADS: A Road Pave-
ment Monitoring System for Anomaly Detection Using Smart
Phones,” in Big data analytics in the social and ubiquitous
Data Availability context, pp. 128–146, 2016.
The raw data used to support the findings of this study [14] I. Tekin, Economic investigations of the current state of the
have been deposited in http://doi.org/10.5281/zenodo.1461243 road system in order to determine the advantages and areas
of application of a road monitoring system [Bachelors Thesis],
[28]. The data can be processed with the presented toolbox
Karlsruhe Institute of Technology, Institute of Vehicle Systems
available in http://doi.org/10.5281/zenodo.1216187 [21]. Technology.
[15] J. Masino, G. Levasseur, M. Frey, F. Gauterin, R. Mikut, and M.
Conflicts of Interest Reischl, “Charakterisierung der Fahrbahnbeschaffenheit durch
Data Mining von gemessenen kinematischen Fahrzeuggrößen,”
The authors declare that they have no conflicts of interest. Automatisierungstechnik, vol. 65, no. 12, 2017.
[16] J. Masino, J. Pinay, M. Reischl, and F. Gauterin, “Road surface
prediction from acoustical measurements in the tire cavity
Acknowledgments using support vector machine,” Applied Acoustics, vol. 125, pp.
41–48, 2017.
This work was funded by the Federal Ministry for the
[17] J. Masino, M. Frey, F. Gauterin, and R. Sharma, “Development
Environment, Nature Conservation, Building and Nuclear of a highly accurate and low cost measurement device for Field
Safety, Germany, within the Environmental Research Plan Operational Tests,” in Proceedings of the 3rd IEEE International
2014 (Project no. 3714541000). Symposium on Inertial Sensors and Systems, ISS 2016, pp. 74–77,
USA, February 2016.
References [18] M. Sayers, T. Gillespie, and C. Queiroz, “The international
road roughness experiment: Establishing correlation and a
[1] P. De Gonneville and G. Martin, “Le mécanisme d’ accident,” calibration standard for measurements,” World Bank Technical
in Sétra: Service d’ études techniques des routes et autoroutes, Paper, 1986.
Bagneux, 2006. [19] C. C. Ward and K. Iagnemma, “Speed-independent vibration-
[2] “Destatis. Verkehrsunfälle. Destatis, Mar. 2017. Artikelnummer: based terrain classification for passenger vehicles,” Taylor &
2080700161124”. Francis, Vehicle System Dynamics, vol. 47, no. 9, 2009.
[20] M. Sokolova and G. Lapalme, “A systematic analysis of perfor-
[3] P. Mohan, V. N. Padmanabhan, and R. Ramachandran, “Neri-
mance measures for classification tasks,” Information Processing
cell: Rich monitoring of road and traffic conditions using mobile
& Management, vol. 45, no. 4, pp. 427–437, 2009.
smartphones,” in Proceedings of the The ACM Conference on
Embedded Networked Sensor Systems, 2008. [21] J. Thumm and J. Masino, Vehicle Learner Toolbox for Road
Condition Estimation, Matlab files, 2018.
[4] M. Schade, “Der intelligente Löcher-Sucher,” Auto Bild, 2016.
[22] R. Mikut, A. Bartschat, and W. Doneit, “The MATLAB toolbox
[5] E. Esmailzadeh and F. Fahimi, “Optimal adaptive active suspen- SciXMiner: User’s manual and programmer’s guide,” 2017,
sions for a full car model,” Vehicle System Dynamics, vol. 27, no. https://arxiv.org/abs/1704.03298.
2, pp. 89–107, 1997. [23] R. Mikut, O. Burmeister, S. Braun, and M. Reischl, “The open
[6] J. Eriksson, L. Girod, B. Hull, R. Newton, S. Madden, and source MATLAB toolbox Gait-CAD and its application to
H. Balakrishnan, “The pothole patrol: using a mobile sensor bioelectric signal processing,” in Proceedings of the DGBMT-
network for road surface monitoring,” in Proceedings of the 6th Workshop Biosignalverarbeitung, pp. 109–111, Potsdam, Ger-
International Conference on Mobile Systems, Applications, and many, 2008.
10 Journal of Advanced Transportation

[24] C. Halfmann and H. Holzmann, Adaptive Modelle für die

Kraftfahrzeugdynamik, Springer, Berlin, Germany, 2003.
[25] O. J. Woodman, “An introduction to inertial navigation,” Tech.
Rep., University of Cambridge, Computer Laboratory, 2007.
[26] S. J. Orfanidis, Introduction to signal processing, Prentice-Hall,
Inc, 1995.
[27] J. Masino, M.-J. Foitzik, M. Frey, and F. Gauterin, “Pavement
type and wear condition classification from tire cavity acoustic
measurements with artificial neural networks,” The Journal of
the Acoustical Society of America, vol. 141, no. 6, pp. 4220–4229,
2017.
[28] J. Masino, “Characterization of Road Condition with Data
Mining Based on Measured Kinematic Vehicle Parameters,
Data,” 2018.
 International Journal of

Rotating Advances in
Machinery Multimedia

The Scientific
Engineering
Journal of
Journal of

Hindawi
World Journal
Hindawi Publishing Corporation Hindawi
Sensors
Hindawi Hindawi
www.hindawi.com Volume 2018 http://www.hindawi.com
www.hindawi.com Volume 2018
2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Journal of

Control Science
and Engineering

Advances in
Civil Engineering
Hindawi Hindawi
www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Submit your manuscripts at

www.hindawi.com

Journal of
Journal of Electrical and Computer
Robotics
Hindawi
Engineering
Hindawi
www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

VLSI Design
Advances in
OptoElectronics
International Journal of

International Journal of
Modelling &
Simulation
Aerospace
Hindawi Volume 2018
Navigation and
Observation
Hindawi
www.hindawi.com Volume 2018
in Engineering
Hindawi
www.hindawi.com Volume 2018
Engineering
Hindawi
www.hindawi.com Volume 2018
Hindawi
www.hindawi.com www.hindawi.com Volume 2018

International Journal of
International Journal of Antennas and Active and Passive Advances in
Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration
Hindawi Hindawi Hindawi Hindawi Hindawi
www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018