Abstract
A new class of algorithms is presented for efficiently carrying out many-to-many parallel data redistribution in a memory-limited environment. Key properties of these algorithms are explored, and their performance is compared using idealized benchmark problems. These algorithms form part of a newly developed MPI-based library MADRE (Memory-Aware Data Redistribution Engine), an open source toolkit designed for easy integration with application codes to improve their performance, portability, and scalability.
This research was supported by the National Science Foundation under Grants CCF-0733035 and CCF-0540948 and used resources of the Argonne Leadership Computing Facility at Argonne National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under contract DE-AC02-06CH11357.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Ricker, P.M., Fryxell, B., Olson, K., Timmes, F.X., Zingale, M., Lamb, D.Q., MacNeice, P., Rosner, R., Tufo, H.: FLASH: A multidimensional hydrodynamics code for modeling astrophysical thermonuclear flashes. Bulletin of the American Astronomical Society 31, 1431 (1999)
de Cougny, H.L., Devine, K.D., Flaherty, J.E., Loy, R.M., Özturan, C., Shephard, M.S.: Load balancing for the parallel adaptive solution of partial differential equations. Appl. Numer. Math. 16(1-2), 157–182 (1994)
Nieter, C., Cary, J.R.: Vorpal: a versatile plasma simulation code. J. Comput. Phys. 196(2), 448–473 (2004)
Catalyurek, U., Boman, E., Devine, K., Bozdag, D., Heaphy, R., Riesen, L.: Hypergraph-based dynamic load balancing for adaptive scientific computations. In: Proc. of 21st International Parallel and Distributed Processing Symposium (IPDPS 2007). IEEE, Los Alamitos (2007)
Pinar, A., Hendrickson, B.: Interprocessor communication with limited memory. IEEE Transactions on Parallel and Distributed Systems 15(7) (July 2004)
Siegel, S.F.: The MADRE web page (2008), http://vsl.cis.udel.edu/madre
Zaki, O., Lusk, E., Gropp, W., Swider, D.: Toward scalable performance visualization with Jumpshot. Int. J. High Perform. Comput. Appl. 13(3), 277–288 (1999)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Siegel, S.F., Siegel, A.R. (2008). MADRE: The Memory-Aware Data Redistribution Engine. In: Lastovetsky, A., Kechadi, T., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2008. Lecture Notes in Computer Science, vol 5205. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87475-1_31
Download citation
DOI: https://doi.org/10.1007/978-3-540-87475-1_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87474-4
Online ISBN: 978-3-540-87475-1
eBook Packages: Computer ScienceComputer Science (R0)