HPDC

Basic data flow patterns which we call idioms, such as stream, transpose, reduction, random access and stencil, are common in scientific numerical applications. We hypothesize that a small number of idioms can cover most programming constructs that dominate the execution time of scientific codes and can be used to approximate the application performance. In this paper, we start with a manual analysis of code coverage on the NAS Parallel Benchmark (NPB) and find that five idioms suffice to cover 100 % of the NPB codes. We then compare the performance of our idiom benchmarks and their corresponding instances in different NPB codes on two different platforms and find that they differ by about 30%. To check the hypotheses with real applications further, we propose an automatic idioms recognition method, implement the method basing on the open source compiler Open64, and verify the prototype system with the previous manual analysis results.

Grid Portals, based on standard Web technologies, are emerging as important and useful user interfaces to computational and data grids. Grid portals enable virtual organizations, comprised of distributed researchers to collaborate and access resources more efficiently and seamlessly. The Astrophysics Simulation Collaboratory (ASC) Grid Portal provides a framework to enable researchers in the field of numerical relativity to study astrophysical phenomenon by making use of the Cactus computational toolkit. We ...

Many high-performance distributed applications use only a small fraction of their available bandwidth. A common cause of this problem is not a flaw in the application design, but rather improperly tuned network settings. Proper tuning techniques, such as setting the correct TCP buffers and using parallel streams, are well-known in the networking community, but outside this community they are infrequently applied. In this paper, we describe a service that makes the task of network tuning trivial for application developers and users. Widespread use of this service should virtually eliminate a common stumbling block for high-performance distributed applications

Online social games are Internet-based games that use the social networks formed by players to extend in-game functionality. For example, gamers participating in the BBO Fans community combine online bridge play with social networking. Despite an increase in the popularity of online social gaming—currently, there exist over one million on-line bridge players—, and of decades of research on social networks, the activity characteristics and the community
structure of online social gaming remain relatively unknown. In this work we investigate and contrast these aspects for two bridge communities, BBO Fans (online) and Locomotiva (face to face). We propose the use of playing relationships instead of traditional social relationships such as friends and
friends-of-friends. Using long-term, large-scale data we have collected from both the online and face to face bridge communities, we analyze user behavior, social network structure, and playing style in bridge communities. We find many similar characteristics in the two studied communities, but we
also find more variation in the activity levels and fewer stable partnerships for the face to face bridge community.

Large-scale distributed computing systems such as grids are serving a growing number of scientists. These environments bring about not only the advantages of an economy of scale, but also the challenges of resource and workload heterogeneity. A consequence of these two forms of heterogeneity is that job runtimes and queue wait times are highly variable, which generally reduces system performance and makes grids difficult to use by the common scientist. Predicting job runtimes and queue wait times have been widely studied for parallel environments. However, there is no detailed investigation on how the proposed prediction methods perform in grids, whose resource structure and workload characteristics are very different from those in parallel systems. In this paper, we assess the performance and benefit of predicting job runtimes and queue wait times in grids based on traces gathered from various research and production grid environments. First, we evaluate the performance of simple yet widely used time series prediction methods and the effect of applying them to different types of job classes (e.g., all jobs submitted by single users or to single sites). Then, we investigate the performance of two kinds of queue wait time prediction methods for grids. Last, we investigate whether prediction-based grid-level scheduling policies can have better performance than policies that do not use predictions.

The Legion project at the University of Virginia is an architecture for designing and building system services that provide the illusion of a single virtual machine to users, a virtual machine that provides secure shared object and shared name spaces, application adjustable fault-tol-erance, ...

In this paper we propose a new method for improv-ing the average response time of Web servers by cooper-atively caching the results of requests for dynamic content. The work is motivated by our recent study of access logs from the Alexandria Digital Library server at UCSB, ...

The Large Hadron Collider (LHC) at CERN, the European Organization for Nuclear Research, needs to produce unprecedented volumes of data when it starts operation in 2007. To provide for its computational needs, the LHC computing grid (LCG) should be deployed as a worldwide computational grid service, providing the middleware upon which the physics analysis for the LHC can be carried out. In 2003, versions of this middleware were deployed which were based on the middleware produced by the European Data Grid project (EDG). In 2004 the LCG-2 release, which consisted of the EDG middleware with some minor modifications, was deployed for use by the LHC experiments. A series of data challenges by these experiments were the first real experiment production use of LCG. During the course of the data challenges, many issues and problems were exposed which had not shown up in more limited tests. The deployment, service and development teams worked closely with the experiments to understand these issues and while some of the problems were solved during the data challenges, others exposed fundamental problems with the middleware as deployed in LCG-2. One of these fundamental problems was the performance under real load of the catalog component provided by EDG, the replica location service. To solve these problems a new component was designed, the LCG file catalog (LFC). The LFC moves away from the replica location service model used in previous LCG releases, towards a hierarchical file system model which is more like a UNIX file system. It also adds missing functionality which was requested by the experiments. This paper presents the architecture and implementation of the LFC and evaluates it in a series of performance tests, with up to forty million entries and one hundred requesting threads from multiple clients. The results show good scalability up to the limits of these tests, and compare favourably with other grid catalog implementations.

Grid technologies enable large-scale sharing of resources within formal or informal consortia of individuals and/or institutions: what are sometimes called virtual organizations. In these settings, the discovery, characterization, and monitoring of resources, services, and computations are challenging problems due to the considerable diversity; large numbers, dynamic behavior, and geographical distribution of the entities in which a user might be interested. Consequently, information services are a vital part of any Grid software ...

The Globus Toolkit Monitoring and Discovery System (MDS4) defines and implements mechanisms for service and resource discovery and monitoring in distributed environments. MDS4 is distinguished from previous similar systems by its extensive use of interfaces and behaviors defined in the new WSResource Framework and WS-Notification specifications, and by its deep integration into essentially every component of the Globus Toolkit. We describe the MDS4 architecture and the Web service interfaces and behaviors that allow users to discover resources and services, monitor resource and service states, receive updates on current status, and visualize monitoring results. We also describe how MDS4 can be used to implement large-scale distributed monitoring and distributed systems, and present experimental results that provide insights into the performance that can be achieved via the use of these mechanisms.

Grid applications can combine the use of computation, storage, network, and other resources. These resources are often geographically distributed, adding to application complexity and thus the difficulty of understanding application performance. We present GridMapper, a tool for monitoring and visualizing the behavior of such distributed systems. GridMapper builds on basic mechanisms for registering, discovering, and accessing performance information sources, as well as for mapping from domain names to physical ...