Abstract
Measuring and understanding the end-user browsing Quality of Experience (QoE) is crucial to Mobile Network Operators (MNOs) to retain their customers and increase revenue. MNOs often use traffic traces to detect the bottlenecks and study their end-users experience. Recent studies show that Above The Fold (ATF) time better approximates the user browsing QoE compared to traditional metrics such as Page Load Time (PLT). This work focuses on developing a methodology to measure the web browsing QoE over operational Mobile Broadband (MBB) networks. We implemented a web performance measurement tool WebLAR (it stands for Web Latency And Rendering) that measures web Quality of Service (QoS) such as TCP connect time, and Time To First Byte (TTFB) and web QoE metrics including PLT and ATF time. We deployed WebLAR on 128 MONROE (a European-wide mobile measurement platform) nodes, and conducted two weeks long (May and July 2018) web measurement campaign towards eight websites from six operational MBB networks. The result shows that, in the median case, the TCP connect time and TTFB in Long Term Evolution (LTE) networks are, respectively, 160% and 30% longer than fixed-line networks. The DNS lookup time and TCP connect time of the websites varies significantly across MNOs. Most of the websites do not show a significant difference in PLT and ATF time across operators. However, Yahoo shows longer ATF time in Norwegian operators than that of the Swedish operators. Moreover, user mobility has a small impact on the ATF time of the websites. Furthermore, the website design should be taken into consideration when approximating the ATF time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
ImageMagick: tool to create, edit, compose, or convert bitmap images. https://imagemagick.org. Accessed 12 Oct 2018
WebPageTest. https://www.webpagetest.org. Accessed 09 Jan 2019
WebPagetest Metrics: SpeedIndex. https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index. Accessed 15 Oct 2018
Ahmad, S., Haamid, A.L., Qazi, Z.A., Zhou, Z., Benson, T., Qazi, I.A.: A view from the other side: understanding mobile phone characteristics in the developing world. In: ACM IMC (2016). http://dl.acm.org/citation.cfm?id=2987470
Akamai White Paper: Measuring Real Customer Experiences over Mobile Networks. https://www.akamai.com/jp/ja/multimedia/documents/white-paper/measuring-real-customer-experiences-over-mobile-networks-report.pdf. Accessed 12 Oct 2017
Alay, Ö., et al.: Experience: an open platform for experimentation with commercial mobile broadband networks. In: ACM MobiCom (2017). https://doi.org/10.1145/3117811.3117812
Asrese, A.S.: WebLAR: A Web Performance Measurement Tool (2019). https://github.com/alemnew/weblar
Asrese, A.S., Eravuchira, S.J., Bajpai, V., Sarolahti, P., Ott, J.: Measuring web latency and rendering performance: method, tools & longitudinal dataset. IEEE Trans. Netw. Serv. Manag. (2019, to appear)
Asrese, A.S., Sarolahti, P., Boye, M., Ott, J.: WePR: a tool for automated web performance measurement. In: IEEE Globecom Workshops (2016). https://doi.org/10.1109/GLOCOMW.2016.7849082
Asrese, A.S., Walelgne, E., Bajpai, V., Lutu, A., Alay, Ö., Ott, J.: Measuring web quality of experience in cellular networks (dataset) (2019). https://github.com/alemnew/2019-pam-weblar
Bajpai, V., Kühlewind, M., Ott, J., Schönwälder, J., Sperotto, A., Trammell, B.: Challenges with reproducibility. In: SIGCOMM Reproducibility Workshop, pp. 1–4 (2017). https://doi.org/10.1145/3097766.3097767
Bajpai, V., Schönwälder, J.: A survey on internet performance measurement platforms and related standardization efforts. IEEE Commun. Surv. Tutor. 17(3), 1313–1341 (2015). https://doi.org/10.1109/COMST.2015.2418435
Balachandran, A., et al.: Modeling web quality-of-experience on cellular networks. In: ACM MobiCom (2014). https://doi.org/10.1145/2639108.2639137
Barakovic, S., Skorin-Kapov, L.: Multidimensional modelling of quality of experience for mobile web browsing. Comput. Hum. Behav. 50, 314–332 (2015). https://doi.org/10.1016/j.chb.2015.03.071
Brutlag, J., Abrams, Z., Meenan, P.: Above the Fold Time: Measuring Web Page Performance Visually. https://conferences.oreilly.com/velocity/velocity-mar2011/public/schedule/detail/18692
Cao, Y., Nejati, J., Wajahat, M., Balasubramanian, A., Gandhi, A.: Deconstructing the energy consumption of the mobile page load. Proc. ACM Meas. Anal. Comput. Syst. 1(1), 6 (2017). https://doi.org/10.1145/3084443
Casas, P., Seufert, M., Wamser, F., Gardlo, B., Sackl, A., Schatz, R.: Next to you: monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Netw. Serv. Manag. 13(2), 181–196 (2016). https://doi.org/10.1109/TNSM.2016.2537645
Cecchet, E., Sims, R., He, X., Shenoy, P.J.: mBenchLab: measuring QoE of Web applications using mobile devices. In: International Symposium on Quality of Service, IWQoS (2013). https://doi.org/10.1109/IWQoS.2013.6550259
Chen, Q.A., et al.: QoE doctor: diagnosing mobile app QoE with automated UI control and cross-layer analysis. In: ACM Internet Measurement Conference (2014). https://doi.org/10.1145/2663716.2663726
Dasari, M., Vargas, S., Bhattacharya, A., Balasubramanian, A., Das, S.R., Ferdman, M.: Impact of device performance on mobile internet QoE. In: Internet Measurement Conference, pp. 1–7 (2018). https://doi.org/10.1145/3278532.3278533
DoubleClick: The Need for Mobile Speed: Better User Experiences, Greater Publisher Revenue. https://goo.gl/R4Lmfh. Accessed 26 Feb 2018
Eravuchira, S.J., Bajpai, V., Schönwälder, J., Crawford, S.: Measuring web similarity from dual-stacked hosts. In: Conference on Network and Service Management, pp. 181–187 (2016). https://doi.org/10.1109/CNSM.2016.7818415
FFmpeg: FFmpeg: a complete, cross-platform solution to record, convert and stream audio and video. https://ffmpeg.org. Accessed 12 Oct 2018
Google: Lighthouse: an open-source, automated tool for improving the quality of web pages. https://developers.google.com/web/tools/lighthouse. Accessed 09 Jan 2019
da Hora, D.N., Asrese, A.S., Christophides, V., Teixeira, R., Rossi, D.: Narrowing the gap between QoS metrics and Web QoE using above-the-fold metrics. In: Beverly, R., Smaragdakis, G., Feldmann, A. (eds.) PAM 2018. LNCS, vol. 10771, pp. 31–43. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76481-8_3
Hosek, J., et al.: Mobile web QoE study for smartphones. In: IEEE GLOBECOM Workshop (2013). https://doi.org/10.1109/GLOCOMW.2013.6825149
Hoßfeld, T., Metzger, F., Rossi, D.: Speed index: relating the industrial standard for user perceived web performance to web QoE. In: IEEE International Conference on Quality of Multimedia Experience (2018). https://doi.org/10.1109/QoMEX.2018.8463430
Li, L., et al.: A longitudinal measurement study of TCP performance and behavior in 3G/4G networks over high speed rails. IEEE/ACM Trans. Netw. 25(4), 2195–2208 (2017). https://doi.org/10.1109/TNET.2017.2689824
Mandalari, A.M., et al.: Experience: implications of roaming in Europe. In: MOBICOM, pp. 179–189 (2018). https://doi.org/10.1145/3241539.3241577
Mozilla: Using the Resource Timing API. https://developer.mozilla.org/en-US/docs/Web/API/Resource_Timing_API/Using_the_Resource_Timing_API. Accessed 24 May 2018
Nejati, J., Balasubramanian, A.: An in-depth study of mobile browser performance. In: Conference on World Wide Web, pp. 1305–1315 (2016). https://doi.org/10.1145/2872427.2883014
OpenSignal: Meteor. https://meteor.opensignal.com. Accessed 12 May 2017
Sackl, A., Casas, P., Schatz, R., Janowski, L., Irmer, R.: Quantifying the impact of network bandwidth fluctuations and outages on Web QoE. In: IEEE International Workshop on Quality of Multimedia Experience (2015). https://doi.org/10.1109/QoMEX.2015.7148078
Sonntag, S., Manner, J., Schulte, L.: Netradar - measuring the wireless world. In: IEEE International Symposium and Workshops on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (2013). http://ieeexplore.ieee.org/document/6576402/
Varvello, M., Blackburn, J., Naylor, D., Papagiannaki, K.: EYEORG: a platform for crowdsourcing web quality of experience measurements. In: ACM Conference on emerging Networking EXperiments and Technologies (2016). https://doi.org/10.1145/2999572.2999590
Walelgne, E.A., Kim, S., Bajpai, V., Neumeier, S., Manner, J., Ott, J.: Factors affecting performance of web flows in cellular networks. In: IFIP Networking (2018)
Walelgne, E.A., Manner, J., Bajpai, V., Ott, J.: Analyzing throughput and stability in cellular networks. In: IEEE/IFIP Network Operations and Management Symposium, pp. 1–9 (2018). https://doi.org/10.1109/NOMS.2018.8406261
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix A List and Category of Measured Webpages
The websites are selected from different categories such as social media, news websites, and WIKI pages. Moreover, while selecting these websites, the design of the websites (from simple to media-rich complex webpages) and the purpose of the websites are taken into consideration. Furthermore, for each website we selected a specific webpage that does not require user interaction to show meaningful contents to the user.
-
News websites
-
Wiki websites
-
Social media websites
-
General websites
Appendix B Additional Observations
Although not specific to mobility scenario, Fig. 5(2) also shows that PLT can under- or over-estimate the web QoE. For instance, for Facebook, the onLoad event fires before all the necessary web objects in the above-the-fold area are downloaded. For these types of websites the PLT underestimates the user QoE. On the other hand, for websites like Yahoo and Reddit, the ATF is shorter compared with PLT time, which overestimates the user QoE.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Asrese, A.S., Walelgne, E.A., Bajpai, V., Lutu, A., Alay, Ö., Ott, J. (2019). Measuring Web Quality of Experience in Cellular Networks. In: Choffnes, D., Barcellos, M. (eds) Passive and Active Measurement. PAM 2019. Lecture Notes in Computer Science(), vol 11419. Springer, Cham. https://doi.org/10.1007/978-3-030-15986-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-15986-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-15985-6
Online ISBN: 978-3-030-15986-3
eBook Packages: Computer ScienceComputer Science (R0)