[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Integer linear programming-based multi-objective scheduling for scientific workflows in multi-cloud environments

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Scientific communities are motivated to schedule the data-intensive scientific workflows in multi-cloud environments, where considerable diverse resources are provided by multiple clouds and resource limitation imposed by individual clouds is overcome. However, this scheduling involves two conflicting objectives: minimizing cost and makespan. In general, dealing with such conflicting criteria is a difficult task. But fortunately recent efficient methods for solving multi-objective optimization problems motivated us to provide a multi-objective model considering minimization of cost and makespan as objectives. For solving this model, we use different scalarization procedures such as weighted-sum, Benson's scalarization and weighted min–max under different scenarios. Moreover, we investigate the stability of obtained solutions and propose a new approach for determining the most stable solution related to weighted-sum and weighted min–max as post-optimality analysis. Results indicate that our proposed weighted-sum approach outperforms the previously developed methods in terms of hypervolume.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://aws.amazon.com/ec2.

  2. https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits#virtual-machines-limits.

  3. The Epigenome workflow with 4000 tasks has eight levels. Assume the execution time of each task is in seconds on VM with 1 CCU. The level 5 has 495 tasks with execution time = 9635.01 s. For executing the tasks in level 5 on VMs of Azure with performance 9 CCU, the 56 VMs are needed in which these tasks are executed in three bags with six tasks and 53 bags with nine tasks.

  4. https://www.rightscale.com/lp/state-of-the-cloud.

  5. CloudHarmony Compute Unit.

  6. http://blog.cloudharmony.com/2010/09/benchmarking-of-ec2s-new-cluster.html.

  7. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator.

  8. ILOG CPLEX. http://www.ilog.com/products/cplex.

References

  1. Abdi S, PourKarimi L, Ahmadi M, Zargari F (2018) Cost minimization for bag-of-tasks workflows in a federation of clouds. J Supercomput 74(6):2801–2822

    Article  Google Scholar 

  2. Abdi S, PourKarimi L, Ahmadi M, Zargari F (2017) Cost minimization for deadline-constrained bag-of-tasks applications in federated hybrid clouds. Future Gener Comput Syst 71:113–128

    Article  Google Scholar 

  3. Nesmachnow S, Iturriaga S, Dorronsoro B (2015) Efficient heuristics for profit optimization of virtual cloud brokers. IEEE Comput Intell Mag 10(1):33–43

    Article  Google Scholar 

  4. Toosi AN, Calheiros RN, Buyya R (2014) Interconnected cloud computing environments: challenges, taxonomy, and survey. ACM Comput Surv (CSUR) 47(1):7

    Article  Google Scholar 

  5. Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 71(9):3373–3418

    Article  Google Scholar 

  6. Schrijver A (1998) Theory of linear and integer programming. Wiley, Hoboken

    MATH  Google Scholar 

  7. Kadioglu S, Malitsky Y, Sellmann M, Tierney K (2010) ISAC-instance-specific algorithm configuration. In: ECAI, vol 215, pp 751–756

  8. Fard HM, Prodan R, Fahringer T (2014) Multi-objective list scheduling of workflow applications in distributed computing infrastructures. J Parallel Distrib Comput 74(3):2152–2165

    Article  Google Scholar 

  9. Hu H, Li Z, Hu H, Chen J, Ge J, Li C, Chang V (2018) Multi-objective scheduling for scientific workflow in multicloud environment. J Netw Comput Appl 114:108–122

    Article  Google Scholar 

  10. Zhu Z, Zhang G, Li M, Liu X (2016) Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans Parallel Distrib Syst 27(5):1344–1357

    Article  Google Scholar 

  11. Jena RK (2015) Multi objective task scheduling in cloud environment using nested PSO framework. Proc Comput Sci 57:1219–1227

    Article  Google Scholar 

  12. Wang X, Yeo CS, Buyya R, Su J (2011) Optimizing the makespan and reliability for workflow applications with reputation and a look-ahead genetic algorithm. Future Gener Comput Syst 27(8):1124–1134

    Article  Google Scholar 

  13. Choudhary A, Gupta I, Singh V, Jana PK (2018) A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Future Gener Comput Syst 83:14–26

    Article  Google Scholar 

  14. Durillo JJ, Prodan R (2014) Multi-objective workflow scheduling in Amazon EC2. Cluster Comput 17(2):169–189

    Article  Google Scholar 

  15. Xu H, Yang B, Qi W, Ahene E (2016) A multi-objective optimization approach to workflow scheduling in clouds considering fault recovery. KSII Trans Internet Inf Syst (TIIS) 10(3):976–995

    Google Scholar 

  16. Qu X, Xiao P, Huang L (2018) Improving the energy efficiency and performance of data-intensive workflows in virtualized clouds. J Supercomput 74(7):2935–2955

    Article  Google Scholar 

  17. Rezaeian A, Naghibzadeh M, Epema DHJ (2019) Fair multiple-workflow scheduling with different quality-of-service goals. J Supercomput 75(2):746–769

    Article  Google Scholar 

  18. Poola D, Ramamohanarao K, Buyya R (2014) Fault-tolerant workflow scheduling using spot instances on clouds. Proc Comput Sci 29:523–533

    Article  Google Scholar 

  19. de Oliveira D, Ocaña KACS, Baião F, Mattoso M (2012) A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds. J Grid Comput 10(3):521–552

    Article  Google Scholar 

  20. Fard H, Prodan R, Barrionuevo JJD, Fahringer T (2012) A multi-objective approach for workflow scheduling in heterogeneous environments. https://doi.org/10.1109/CCGrid.2012.114.

  21. Pietri I, Malawski M, Juve G, Deelman E, Nabrzyski J, Sakellariou R (2013) Energy-constrained provisioning for scientific workflow ensembles. In: Third International Conference on Cloud and Green Computing (CGC), 2013. IEEE, pp 34–41

  22. Zeng L, Veeravalli B, Li X (2015) SABA: a security-aware and budget-aware workflow scheduling strategy in clouds. J Parallel Distrib Comput 75:141–151

    Article  Google Scholar 

  23. Mohammadi S, Pedram H, PourKarimi L (2018) Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments. J Supercomput 74(9):4717–4745

    Article  Google Scholar 

  24. Rehman A, Hussain SS, ur Rehman Z, Zia S, Shamshirband S (2018) Multi-objective approach of energy efficient workflow scheduling in cloud environments. Concurr Comput Pract Exp 31(8):e4949

    Article  Google Scholar 

  25. Arabnejad V, Bubendorfer K, Ng B (2019) Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1):29–44

    Article  Google Scholar 

  26. Bharathi S, Chervenak A, Deelman E, Mehta G, Su M-H, Vahi K (2008) Characterization of scientific workflows. In: Third Workshop on workflows in support of large-scale science, 2008. WORKS 2008. IEEE, pp 1–10

  27. Ehrgott M (2005) Multicriteria optimization, vol 491. Springer, Berlin

    MATH  Google Scholar 

  28. Benson HP (1978) Existence of efficient solutions for vector maximization problems. J Optim Theory Appl 26(4):569–580

    Article  MathSciNet  Google Scholar 

  29. IBMI (2009) CPLEX, V12. 1: user’s manual for CPLEX. International Business Machines Corporation 46(53):157

  30. Topcuoglu H, Hariri S, Wu M-Y (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  31. Zitzler E, Thiele L, Laumanns M, Fonseca CM, Da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132

    Article  Google Scholar 

  32. Bradstreet L (2011) The hypervolume indicator for multi-objective optimisation: calculation and use. University of Western Australia

  33. Sitarz S (2012) Mean value and volume-based sensitivity analysis for Olympic rankings. Eur J Oper Res 216(1):232–238

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Somayeh Mohammadi

  2. Department of Mathematics, Razi University, Kermanshah, Iran

    Latif PourKarimi

  3. Department of Computer Engineering, Amir Kabir University of Technology, Tehran, Iran

    Hossein Pedram

Authors
  1. Somayeh Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Latif PourKarimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hossein Pedram
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Latif PourKarimi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohammadi, S., PourKarimi, L. & Pedram, H. Integer linear programming-based multi-objective scheduling for scientific workflows in multi-cloud environments. J Supercomput 75, 6683–6709 (2019). https://doi.org/10.1007/s11227-019-02877-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-019-02877-8

Keywords

Search

Navigation