Section author: Robert Nikutta <robert.nikutta@noirlab.edu>

Version: 20250303

1.3.3.2. Data Sources panel

The Data Sources panel, located on the left-hand side of the Data Explorer application, displays the three types of data sources you have access to at Data Lab: the catalogs (data tables) hosted by Data Lab, your tables in the MyDB database, and your remote files in VOSpace.

Note

The MyDB and VOSpace sections are only populated when you are logged in.

Data Sources (left panel)

Figure: Data Sources panel showing the Data Lab catalogs, the (logged-in) user’s MyDB tables, and VOSpace files.

Hovering the mouse over one of the “?” symbols brings up additional information about a given data source section (in the figure above, for “Catalogs”).

Each section can be folded or expanded by clicking on the grey area with a section name. The figure below shows the MyDB and VOSpace sections closed.

Data Sources partially folded

Figure: Data Sources panel with the MyDB and VOSpace sections collapsed.

1.3.3.2.1. Catalogs

The Catalogs section lists all astronomical catalogs currently hosted by Data Lab (DL). These comprise surveys conducted on NOIRLab telescopes, additional high-value surveys created by other teams (from the ground and from space), and several large-scale simulated datasets. A catalog can contain multiple tables in a single schema.

All catalogs are stored in DL’s databases and can be queried using SQL/ADQL through the Data Explorer or Python APIs (both in scripts and Jupyter notebooks).

Functionality

  • Hovering over one of the top-level items (a schema) in the “Catalogs” section displays up a tool-tip with a brief description of the survey; in the picture below, the Legacy Surveys Data Release 10 (ls_dr10).

Data Sources - schema

  • Clicking onto an item in “Catalogs” expands the schema, showing the various tables contained in a its release.

Data Sources - table

  • Hovering over the name of a table brings up a tool-tip with the table description (in the figure above, the ls_dr10.tractor table).

  • Clicking on a table name opens Data Explorer’s right-side Context panel with detailed information about the table.

  • Hovering over the ⠿ symbol to the left of a table name brings up a quick context menu with the options to:

    • View Table details (same as clicking a table’s name)

    • Open a new query form with the table’s name pre-populated

    • Open up a new cross-matching form

Data Sources - pop-up menu

1.3.3.2.2. MyDB (user tables)

The MyDB section lists your data tables stored in your relational database at Data Lab. They can be queried like the large survey catalogs hosted at Data Lab, using SQL/ADQL, either through the Data Explorer or Python APIs (including in Jupyter notebooks). However, you have write access to your MyDB tables, meaning you can create new tables, import data into tables, rename tables, and delete them when no longer needed.

Functionality

  • Clicking on either the “Table Name” or “Size” headers sorts the list of MyDB tables. Repeated clicks reverse the sorting order.

  • Clicking on a table name opens Data Explorer’s right-side Context panel with detailed information about the table.

  • Hovering over the six-dots symbol to the left of a table name brings up a quick context menu with the options to:

    • View Table details (same as clicking a table’s name)

    • Open a new query form with the table’s name pre-populated

    • Open a new cross-matching form

    • Delete the table from your MyDB

Data Sources - MyDB

  • Hovering over the table’s name also shows a small pen/edit icon to the right of the name. Click it to rename the table in your MyDB.

  • At the bottom of the MyDB section in the Data Sources panel a blue button labeled “Upload Table” allows you to upload a data table to your MyDB.

1.3.3.2.3. VOSpace (user files)

The VOSpace section of the Data Sources panel lists your data files stored at Data Lab, in your VOSpace remote file storage. (VO stands for “Virtual Observatory”).

Data Sources - VOSpace

You can upload, rename, and delete any files in your remote file storage using Data Lab’s storeClient Python interface, or directly here in the Data Explorer.

Functionality

  • The VOSpace section first lists the directories, then the files.

  • Clicking on either the “Name” or “Size” heading sorts the list accordingly. Repeated clicks reverse the sorting order.

  • Clicking on a directory name opens it and displays its content. The current directory’s path is shown just above the table; clicking on any part of the path navigates back to it.

Data Sources - VOSpace dir

  • Hovering over the six-dots symbol to the left of a file or directory name brings up a quick context menu with the options to:

    • Copy URI - Copies to the clipboard the URI of the file or directory, which can be used in Data Lab’s storeClient functions (see, e.g., this example notebook).

    • Delete - Deletes a file or directory on your VOSpace.

    • Download (only for files) - Downloads a file from your VOSpace to your local computer.

Data Sources - VOSpace pop-up

  • Hovering over a file or directory name also shows a small pen/edit icon. Click on it to rename the file or directory directly in your VOSpace.

  • At the bottom of the VOSpace section in the Data Sources panel, a blue button labeled “Create Folder” allows you to create a new directory in your VOSpace. It will be created inside whatever directory is currently displayed. Simply enter a valid name for the new directory, and click on the “Floppy disk / Save” icon. A confirmation checkmark will appear next to the new directory’s name.

Data Sources - VOSpace new dir

  • Also at the bottom of the VOSpace section is a blue button labeled “Upload file” which allows you to upload a file from your local computer to your VOSpace.