[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Testing and evaluation framework for virtualization technologies

  • Published:
Computing Aims and scope Submit manuscript

Abstract

Virtualization technology has attracted considerable attention in recent years. With the increasing popularity of virtualization technology, how to evaluate and ensure its correctness and quality becomes a critical issue. Several methods are proposed to test and evaluate functionalities, performance and quality of different virtualization technologies. However, the testing and evaluation tasks are still complex and time consuming. In this paper, a testing and evaluation framework for virtualization technologies is introduced. The framework aims to provide an integrated environment for software testers from test environment configuration, usage scenario deployment to automatic test and evaluation execution that helps to achieve the functional and quality evaluation of virtualization technologies efficiently. Furthermore, the testing and evaluation framework is built by several well-known open source software, which decreases the deployment cost and retains certain flexibility for further extension.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. More information can be found at https://sites.google.com/site/virtualizationtestingframework/.

References

  1. Almari FN, Zavarsky P, Ruhl R, Lindskog D, Aljaedi A (2012) Performance analysis of oracle database in virtual environments. In: Proceedings of the 2012 26th international conference on advanced information networking and applications workshops pp 1238–1245

  2. Barker SK, Shenoy P (2010) Empirical evaluation of latency-sensitive application performance in the cloud. In: Proceedings of the first annual ACM SIGMM conference on multimedia systems pp 35–46

  3. Carrozza G, Battaglia L, Manetti V, Marotta A, Canonico R, Avallone S (2014) On the evaluation of vm provisioning time in cloud platforms for mission-critical infrastructures. In: Proceedings of the 2014 14th IEEE/ACM international symposium on cluster, cloud and grid computing pp 802–810

  4. Che J, Yu Y, Shi C, Lin W (2010) A synthetical performance evaluation of OpenVZ, Xen and KVM. In: Proceedings of the 2010 IEEE Asia-Pacific services computing conference pp 587–594

  5. Ghoshal D, Canon RS, Ramakrishnan L (2011) I/O performance of virtualized cloud environments. In: Proceedings of the second international workshop on data intensive computing in the clouds pp 71–80

  6. Hu W, Hicks A, Zhang L, Dow EM, Soni V, Jiang H, Bull R, Matthews JN (2013) A quantitative study of virtual machine live migration. In: Proceedings of the 2013 ACM cloud and autonomic computing conference

  7. Hyper-V. https://technet.microsoft.com/library/hh831531.aspx. Accessed 15 Sept 2016

  8. Incki K, Ari I, Sozer H (2012) A survey of software testing in the cloud. In: Proceedings of the IEEE sixth international conference on software security and reliability companion pp 18–23

  9. Iosup A, Epema D (2006) GrenchMark: a framework for analyzing, testing, and comparing grids. In: Proceedings of the sixth IEEE international symposium on cluster computing and the grid pp 313–320

  10. Jayasinghe D, Swint G, Malkowski S, Li J, Wang Q, Park J, Pu C (2012) Expertus: a generator approach to automate performance testing in IaaS clouds. In: Proceedings of the 2012 IEEE 5th international conference on cloud computing pp 115–122

  11. Jenkins,W, Vilkomir S, Sharma P, Pirocanac G (2011) Framework for testing cloud platforms and infrastructures. In: Proceedings of the 2011 international conference on cloud and service computing pp 134–140

  12. Jia C, Yu YT (2013) Using the 5W+1H model in reporting systematic literature review: a case study on software testing for cloud computing. In: Proceedings of the 13th international conference on quality software pp 222–229

  13. Kao CH, Lin CC, Chen JN (2013) Performance testing framework for REST-based web applications. In: Proceedings of the 2013 13th international conference on quality software pp 349–354

  14. Kao CH, Liu ST (2011) A prototype system for object management in private cloud. In: Proceedings of 2011 international conference on cloud and service computing pp 348–353

  15. Kernel Virtual Machine. http://www.linux-kvm.org/page/Main_Page. Accessed 15 Sept 2016

  16. Linux Virtualization Tests. https://github.com/autotest/virt-test. Accessed 15 Sept 2016

  17. LoadRunner. http://www8.hp.com/us/en/software-solutions/loadrunner-load-testing/. Accessed 15 Sept 2016

  18. Lockwood GK, Tatineni M, Wagner R (2014) SR-IOV: performance benefits for virtualized interconnects. In: Proceedings of the 2014 annual conference on extreme science and engineering discovery environment p 47

  19. McDougall R, Anderson J (2010) Virtualization performance: perspectives and challenges ahead. ACM SIGOPS Oper Syst Rev 44(4):40–56

    Article  Google Scholar 

  20. Nasiri R, Hosseini S (2014) A novel framework for cloud testing. Int J Electron Commun Comput Eng 5(4):850–854

    Google Scholar 

  21. Oracle VM VirtualBox. https://www.virtualbox.org/. Accessed 15 Sept 2016

  22. Padala P, Zhu X, Wang Z, Singhal S, Shin KG (2007) Performance evaluation of virtualization technologies for server consolidation. Tech. rep HP Labs, Palo Alto

    Google Scholar 

  23. Pham C, Chen D, Kalbarczyk Z, Iyer RK (2011) CloudVal: a framework for validation of virtualization environment in cloud infrastructure. In: Proceedings of the IEEE/IFIP 41st international conference on dependable systems and networks pp 189–196

  24. Reddy PVV, Rajamani L (2014) Performance evaluation of operating systems in the private cloud with XenServer hypervisor using SIGAR framework. In: Proceedings of the 9th international conference on computer science and education pp 183–188

  25. Rings T, Grabowski J, Schulz S (2011) A testing framework for assessing grid and cloud infrastructure interoperability. Int J Adv Syst Meas 4(1–2):95–108

    Google Scholar 

  26. Selenium. http://www.seleniumhq.org/. Accessed 15 Sept 2016

  27. Sommerville I (2015) Software engineering. Pearson, New Jersey

    MATH  Google Scholar 

  28. Soundararajan V, Agrawal B, Herndon B, Sethuraman P, Taheri R (2014) Benchmarking a virtualization platform. In: Proceedings of the 2014 IEEE international symposium on workload characterization pp 99–109

  29. SPEC VIRT. https://www.spec.org/virt_sc2013. Accessed 15 Sept 2016

  30. Svard P, Hudzia B, Walsh S, Tordsson J, Elmroth E (2015) Principles and performance characteristics of algorithms for live VM migration. ACM SIGOPS Oper Syst Rev 49(1):142–155

    Article  Google Scholar 

  31. Testing Anything Protocol. https://testanything.org/. Accessed 15 Sept 2016

  32. The Grinder. http://grinder.sourceforge.net/. Accessed 15 Sept 2016

  33. The Xen Project. http://www.xenproject.org. Accessed 15 Sept 2016

  34. Varrette S, Guzek M, Plugaru V, Besseron X, Bouvry P (2013) HPC performance and energy-efficiency of Xen, KVM and VMware hypervisors. In: Proceedings of the 25th international symposium on computer architecture and high performance computing pp 89–96

  35. VMmark. http://www.vmware.com/products/vmmark. Accessed 15 Sept 2016

  36. VMware. http://www.vmware.com. Accessed 15 Sept 2016

  37. Ye K, Wu Z, Zhou BB, Jiang X, Wang C, Zomaya AY (2014) Virt-B: towards performance benchmarking of virtual machine systems. IEEE Internet Comput 18(3):64–72

    Article  Google Scholar 

  38. Yigitbasi N, Losup A, Epema D, Ostermann S (2009) C-Meter: a framework for performance analysis of computing clouds. In: Proceedings of 9th IEEE/ACM international symposium on cluster computing and the grid pp 472–477

  39. Zhang Q, Cheng L, Boutaba R (2010) Cloud computing: state-of-the-art and research challenges. J Internet Serv Appl 1:7–18

    Article  Google Scholar 

  40. Zhao L, Liu A, Keung J (2010) Evaluating cloud platform architecture with the CARE framework. In: Proceedings of the 2010 Asia Pacific software engineering conference pp 60–69

  41. Zhao L, Liu A, Keung J (2012) Evaluation framework of virtualization systems for cloud computing. In: Proceedings of the 2012 IEEE Asia Pacific cloud computing congress pp 48–52

Download references

Acknowledgments

This study is supported by the Ministry of Science and Technology of the Republic of China under grant MOST 105-2218-E-143-001.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, National Taitung University, Taitung, Taiwan

    Chia Hung Kao

Authors
  1. Chia Hung Kao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chia Hung Kao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kao, C.H. Testing and evaluation framework for virtualization technologies. Computing 99, 657–677 (2017). https://doi.org/10.1007/s00607-016-0517-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-016-0517-6

Keywords

Mathematics Subject Classification

Search

Navigation