1.4.5.1. VOSpace

VOSpace is the International Virtual Observatory Alliance (IVOA) interface to distributed storage. It specifies how VO agents and applications can use network attached data stores to persist and exchange data in a standard way.

The Virtual Storage component of Data Lab (DL) consists of both database and physical-file storage on Data Lab servers. This component supports several goals within the DL architecture, it:

• provides intermediate storage near the DL data and compute resources that minimizes network transfers during workflows,

• provides persistent storage for results of asynchronous query/compute jobs,

• allows for sharing of data and results with collaborators or other DL users, and

• serves as the basis for user-defined data access services to user-defined data collections (e.g. an image cutout service in front of images in a user’s storage, a queryable interface to a catalog of sub-selected objects, file-level access to analysis results, etc).

Aside from being able to perform common file operations such as copying, renaming, moving, creating directories, and so on, the Virtual Storage system is required to support I/O from each of the main Data Lab user interfaces:

• web-based tools (e.g. the website Data Discovery Tool)

• command-line tools (e.g. the ‘datalab’ desktop client)

• programmatic APIs (e.g. Jupyter notebooks, user-written tools)

These interfaces are meant to cover a range of expected usage from casual browsing of storage from the website through to scripted analysis on the desktop or in the notebook/compute environment. Because physical hardware resources are allocated, all Virtual Storage access (other than read-only access to public files) requires a DL user account and authenticated access.