We present Provenance-to-use (PTU), a tool that minimizes computation time during repeatability t... more We present Provenance-to-use (PTU), a tool that minimizes computation time during repeatability testing. Authors can use PTU to build a package of their software program and include a provenance trace of an initial, reference execution. Testers can perform a partial deterministic replay of the package by choosing a subset of the processes based on the process' compute, memory and I/O utilization obtained during the reference execution. Using the provenance trace, PTU guarantees that events are processed in the same order using the same data from one execution to the next. We show the efficiency of PTU for conducting repeatability testing of workflow-based scientific programs.
Prior to the computational-driven revolution in science, research papers provided the pri- mary m... more Prior to the computational-driven revolution in science, research papers provided the pri- mary mechanism for sharing novel methods and data. Papers described experiments involving small amount of data, derivations on that data, and associated methods and algorithms. Readers reproduced the results by repeating the physical experiment, performing hand calculation, and/or logical argument. The scientific method in this decade has become decisively computational, involving large quantities of data, complex data manipulation tasks, and large, and often distributed, software stacks. The research paper, in its current text form, is only able to summarize the associated data and computation rather than reproduce it computationally. While papers corroborate descriptions through indirect means, such as by building companion websites that share data and software packages, these external websites continue to remain disconnected from the content within the paper, making it difficult to verify claims and reproduce results. There is an critical need for systems that minimize this disconnect. We describe Science Object Linking and Embedding (SOLE), a framework for creating descriptive and interactive publications by linking them with associated science objects, such as source codes, datasets, annotations, workflows, re-playable packages, and virtual machine images. SOLE provides a suite of tools that assist the author to create and host science objects that can then be linked with research papers for the purpose of assessment, repeatability, and verification of research. The framework also creates a linkable representation of the science object with the publication and manages a bibliography-like specification of science objects. In this chapter, we introduce SOLE, and describe its use for augmenting the content of computation-based scientific publications. We present examples from climate science, chemistry, biology, and computer science.
Users can determine the precise origins of their data by collecting detailed provenance records. ... more Users can determine the precise origins of their data by collecting detailed provenance records. However, auditing at a finer grain produces large amounts of metadata. To efficiently manage the collected provenance, several provenance man- agement systems, including SPADE, record provenance on the hosts where it is gen- erated. Distributed provenance raises the issue of efficient reconstruction during the query phase. Recursively querying provenance metadata or computing its transitive closure is known to have limited scalability and cannot be used for large provenance graphs. We present matrix filters, which are novel data structures for representing graph information, and demonstrate their utility for improving query efficiency with experiments on provenance metadata gathered while executing distributed workflow applications.
Science projects are increasingly investing in computational reproducibility. Constructing softwa... more Science projects are increasingly investing in computational reproducibility. Constructing software pipelines to demonstrate reproducibility is also becoming increasingly common. To aid the process of constructing pipelines, science project members often adopt reproducible methods and tools. One such tool is CDE, which is a software packaging tool that encapsulates source code, datasets and environments. However, CDE does not include information about origins of dependencies. Consequently when multiple CDE packages are combined and merged to create a software pipeline, several issues arise requiring an author to manually verify compatibility of distributions, environment variables, software dependencies and compiler options. In this work, we propose software provenance to be included as part of CDE so that resulting provenance-included CDE packages can be easily used for creating software pipelines. We describe provenance attributes that must be included and how they can be efficiently stored in a light-weight CDE package. Furthermore, we show how a provenance in a package can be used for creating software pipelines and maintained as new packages are created. We experimentally evaluate the overhead of auditing and maintaining provenance and compare with heavy weight approaches for reproducibility such as virtualization. Our experiments indicate minimal overheads.
2013 IEEE International Conference on Cluster Computing (CLUSTER), 2013
It has become increasingly important to capture and understand the origins and derivation of data... more It has become increasingly important to capture and understand the origins and derivation of data (its provenance).
2012 IEEE International Geoscience and Remote Sensing Symposium, 2012
Abstract Data and computation are fundamental to advances in geoscience research and discovery. H... more Abstract Data and computation are fundamental to advances in geoscience research and discovery. However, geoscientists currently spend too much time looking for the “right” data, subsequently accessing these data, and then transforming them into a form suitable for analysis. This data management overhead affects a scientists' competitive advantage in making useful contributions. Several cyber-infrastructure (CI) efforts are being undertaken to improve the data management needs of the long-tail geoscientists. In this paper, we ...
Middleware '10 Posters and Demos Track on - Middleware Posters '10, 2010
Abstract Current provenance collection systems typically gather metadata on remote hosts and subm... more Abstract Current provenance collection systems typically gather metadata on remote hosts and submit it to a central server. We describe middleware for managing distributed provenance metadata, where each host maintains an authoritative local repository of the provenance metadata gathered on it. The approach provides several advantages---the system can scale to handle the large amounts of metadata generated when auditing occurs at fine granularity; users retain control over their provenance records; and the middleware ...
Abstract We have developed a prototype distributed query and cross-matching service for the VO co... more Abstract We have developed a prototype distributed query and cross-matching service for the VO community, called SkyQuery, which is implemented with hierarchichal Web Services. SkyQuery enables astronomers to run combined queries on existing distributed heterogeneous astronomy archives. SkyQuery provides a simple, user-friendly interface to run distributed queries over the federation of registered astronomical archives in the VO. The SkyQuery client connects to the portal Web Service, which farms the query out to the ...
Recent evidence has pointed out that an individual pays a positive cost in terms of quality when ... more Recent evidence has pointed out that an individual pays a positive cost in terms of quality when focusing on a broad range of topics. In this paper, we corroborate if scientific researchers also pay a cost in terms of grant money received. For this we measure researcher diversity and correlate it with the amount of money that a researcher has received from federal agencies over the years. Our results reconfirm the negative correlation between wide interests and quality but, on the contrary, indicate that to receive a funded grant it pays to have wide interests.
Research networking platforms, such as VIVO and Profiles Networking provide an information infras... more Research networking platforms, such as VIVO and Profiles Networking provide an information infrastructure for scholarship, representing information about research and researchers—their scholarly works, research interests, and organizational relationships. These platforms are open information infrastructures for scholarship, consisting of linked open data and open- source software tools for managing and visualizing scholarly information. Being RDF based, faceted browsing is a natural technique for navigating such data, partitioning the scholarly information space into orthogonal conceptual dimensions. However, this technique has so far been explored through limited queries in research networking platforms--not allowing for instance full graph based navigation on RDF data. In this paper we present Lens a client-side user interface for faceted navigation of scholarly RDF data. Lens is based on Exhibit, which is a lightweight structured data-publishing framework, but extends Exhibit for expressive SPARQL- like queries and scales it up for navigating amounts of RDF data. Lens consumes data in VIVO ontology, the de facto schema for researcher networking systems. We show how Lens provides better usability over current faceted browsers for research networking platforms.
In this paper, we present GEN an interface generator that takes user-supplied C declarations and ... more In this paper, we present GEN an interface generator that takes user-supplied C declarations and provides the neces- sary interface needed to load and access data from common scientific array databases such as SciDB and Rasdaman. GEN can be used for storing the output of parallel computations directly into the database and automates the previously used inefficient ingestion process which requires development of special database schemas for each computation. Further, GEN requires no modifications to existing C code and can build a working interface in minutes. We show how GEN can be used for Cosmology analysis programs to output data sets in real-time to a database and use for subsequent analysis. We show that GEN introduces modest overhead in program execution but is more efficient than writing to files and then loading. More significantly, it significantly reduces the pro- grammatic overhead of learning new database languages.
We present Provenance-to-use (PTU), a tool that minimizes computation time during repeatability t... more We present Provenance-to-use (PTU), a tool that minimizes computation time during repeatability testing. Authors can use PTU to build a package of their software program and include a provenance trace of an initial, reference execution. Testers can perform a partial deterministic replay of the package by choosing a subset of the processes based on the process' compute, memory and I/O utilization obtained during the reference execution. Using the provenance trace, PTU guarantees that events are processed in the same order using the same data from one execution to the next. We show the efficiency of PTU for conducting repeatability testing of workflow-based scientific programs.
Prior to the computational-driven revolution in science, research papers provided the pri- mary m... more Prior to the computational-driven revolution in science, research papers provided the pri- mary mechanism for sharing novel methods and data. Papers described experiments involving small amount of data, derivations on that data, and associated methods and algorithms. Readers reproduced the results by repeating the physical experiment, performing hand calculation, and/or logical argument. The scientific method in this decade has become decisively computational, involving large quantities of data, complex data manipulation tasks, and large, and often distributed, software stacks. The research paper, in its current text form, is only able to summarize the associated data and computation rather than reproduce it computationally. While papers corroborate descriptions through indirect means, such as by building companion websites that share data and software packages, these external websites continue to remain disconnected from the content within the paper, making it difficult to verify claims and reproduce results. There is an critical need for systems that minimize this disconnect. We describe Science Object Linking and Embedding (SOLE), a framework for creating descriptive and interactive publications by linking them with associated science objects, such as source codes, datasets, annotations, workflows, re-playable packages, and virtual machine images. SOLE provides a suite of tools that assist the author to create and host science objects that can then be linked with research papers for the purpose of assessment, repeatability, and verification of research. The framework also creates a linkable representation of the science object with the publication and manages a bibliography-like specification of science objects. In this chapter, we introduce SOLE, and describe its use for augmenting the content of computation-based scientific publications. We present examples from climate science, chemistry, biology, and computer science.
Users can determine the precise origins of their data by collecting detailed provenance records. ... more Users can determine the precise origins of their data by collecting detailed provenance records. However, auditing at a finer grain produces large amounts of metadata. To efficiently manage the collected provenance, several provenance man- agement systems, including SPADE, record provenance on the hosts where it is gen- erated. Distributed provenance raises the issue of efficient reconstruction during the query phase. Recursively querying provenance metadata or computing its transitive closure is known to have limited scalability and cannot be used for large provenance graphs. We present matrix filters, which are novel data structures for representing graph information, and demonstrate their utility for improving query efficiency with experiments on provenance metadata gathered while executing distributed workflow applications.
Science projects are increasingly investing in computational reproducibility. Constructing softwa... more Science projects are increasingly investing in computational reproducibility. Constructing software pipelines to demonstrate reproducibility is also becoming increasingly common. To aid the process of constructing pipelines, science project members often adopt reproducible methods and tools. One such tool is CDE, which is a software packaging tool that encapsulates source code, datasets and environments. However, CDE does not include information about origins of dependencies. Consequently when multiple CDE packages are combined and merged to create a software pipeline, several issues arise requiring an author to manually verify compatibility of distributions, environment variables, software dependencies and compiler options. In this work, we propose software provenance to be included as part of CDE so that resulting provenance-included CDE packages can be easily used for creating software pipelines. We describe provenance attributes that must be included and how they can be efficiently stored in a light-weight CDE package. Furthermore, we show how a provenance in a package can be used for creating software pipelines and maintained as new packages are created. We experimentally evaluate the overhead of auditing and maintaining provenance and compare with heavy weight approaches for reproducibility such as virtualization. Our experiments indicate minimal overheads.
2013 IEEE International Conference on Cluster Computing (CLUSTER), 2013
It has become increasingly important to capture and understand the origins and derivation of data... more It has become increasingly important to capture and understand the origins and derivation of data (its provenance).
2012 IEEE International Geoscience and Remote Sensing Symposium, 2012
Abstract Data and computation are fundamental to advances in geoscience research and discovery. H... more Abstract Data and computation are fundamental to advances in geoscience research and discovery. However, geoscientists currently spend too much time looking for the “right” data, subsequently accessing these data, and then transforming them into a form suitable for analysis. This data management overhead affects a scientists' competitive advantage in making useful contributions. Several cyber-infrastructure (CI) efforts are being undertaken to improve the data management needs of the long-tail geoscientists. In this paper, we ...
Middleware '10 Posters and Demos Track on - Middleware Posters '10, 2010
Abstract Current provenance collection systems typically gather metadata on remote hosts and subm... more Abstract Current provenance collection systems typically gather metadata on remote hosts and submit it to a central server. We describe middleware for managing distributed provenance metadata, where each host maintains an authoritative local repository of the provenance metadata gathered on it. The approach provides several advantages---the system can scale to handle the large amounts of metadata generated when auditing occurs at fine granularity; users retain control over their provenance records; and the middleware ...
Abstract We have developed a prototype distributed query and cross-matching service for the VO co... more Abstract We have developed a prototype distributed query and cross-matching service for the VO community, called SkyQuery, which is implemented with hierarchichal Web Services. SkyQuery enables astronomers to run combined queries on existing distributed heterogeneous astronomy archives. SkyQuery provides a simple, user-friendly interface to run distributed queries over the federation of registered astronomical archives in the VO. The SkyQuery client connects to the portal Web Service, which farms the query out to the ...
Recent evidence has pointed out that an individual pays a positive cost in terms of quality when ... more Recent evidence has pointed out that an individual pays a positive cost in terms of quality when focusing on a broad range of topics. In this paper, we corroborate if scientific researchers also pay a cost in terms of grant money received. For this we measure researcher diversity and correlate it with the amount of money that a researcher has received from federal agencies over the years. Our results reconfirm the negative correlation between wide interests and quality but, on the contrary, indicate that to receive a funded grant it pays to have wide interests.
Research networking platforms, such as VIVO and Profiles Networking provide an information infras... more Research networking platforms, such as VIVO and Profiles Networking provide an information infrastructure for scholarship, representing information about research and researchers—their scholarly works, research interests, and organizational relationships. These platforms are open information infrastructures for scholarship, consisting of linked open data and open- source software tools for managing and visualizing scholarly information. Being RDF based, faceted browsing is a natural technique for navigating such data, partitioning the scholarly information space into orthogonal conceptual dimensions. However, this technique has so far been explored through limited queries in research networking platforms--not allowing for instance full graph based navigation on RDF data. In this paper we present Lens a client-side user interface for faceted navigation of scholarly RDF data. Lens is based on Exhibit, which is a lightweight structured data-publishing framework, but extends Exhibit for expressive SPARQL- like queries and scales it up for navigating amounts of RDF data. Lens consumes data in VIVO ontology, the de facto schema for researcher networking systems. We show how Lens provides better usability over current faceted browsers for research networking platforms.
In this paper, we present GEN an interface generator that takes user-supplied C declarations and ... more In this paper, we present GEN an interface generator that takes user-supplied C declarations and provides the neces- sary interface needed to load and access data from common scientific array databases such as SciDB and Rasdaman. GEN can be used for storing the output of parallel computations directly into the database and automates the previously used inefficient ingestion process which requires development of special database schemas for each computation. Further, GEN requires no modifications to existing C code and can build a working interface in minutes. We show how GEN can be used for Cosmology analysis programs to output data sets in real-time to a database and use for subsequent analysis. We show that GEN introduces modest overhead in program execution but is more efficient than writing to files and then loading. More significantly, it significantly reduces the pro- grammatic overhead of learning new database languages.
Uploads
Papers by Tanu Malik
The scientific method in this decade has become decisively computational, involving large quantities of data, complex data manipulation tasks, and large, and often distributed, software stacks. The research paper, in its current text form, is only able to summarize the associated data and computation rather than reproduce it computationally. While papers
corroborate descriptions through indirect means, such as by building companion websites that share data and software packages, these external websites continue to remain
disconnected from the content within the paper, making it difficult to verify claims and reproduce results. There is an critical need for systems that minimize this disconnect.
We describe Science Object Linking and Embedding (SOLE), a framework for creating descriptive and interactive publications by linking them with associated science objects, such as source codes, datasets, annotations, workflows, re-playable packages, and virtual machine images. SOLE provides a suite of tools that assist the author to create and host science objects that can then be linked with research papers for the purpose of assessment, repeatability, and verification of research. The framework also creates a linkable representation of the science object with the publication and manages a bibliography-like specification of science objects. In this chapter, we introduce SOLE, and describe its use for augmenting the content of computation-based scientific publications. We present examples from climate science, chemistry, biology, and computer science.
The scientific method in this decade has become decisively computational, involving large quantities of data, complex data manipulation tasks, and large, and often distributed, software stacks. The research paper, in its current text form, is only able to summarize the associated data and computation rather than reproduce it computationally. While papers
corroborate descriptions through indirect means, such as by building companion websites that share data and software packages, these external websites continue to remain
disconnected from the content within the paper, making it difficult to verify claims and reproduce results. There is an critical need for systems that minimize this disconnect.
We describe Science Object Linking and Embedding (SOLE), a framework for creating descriptive and interactive publications by linking them with associated science objects, such as source codes, datasets, annotations, workflows, re-playable packages, and virtual machine images. SOLE provides a suite of tools that assist the author to create and host science objects that can then be linked with research papers for the purpose of assessment, repeatability, and verification of research. The framework also creates a linkable representation of the science object with the publication and manages a bibliography-like specification of science objects. In this chapter, we introduce SOLE, and describe its use for augmenting the content of computation-based scientific publications. We present examples from climate science, chemistry, biology, and computer science.