download / vospace permission problems

Question

Good morning, 

I exploring the Gaia, LS, and DESI datasets with Data Lab. In the process, I am creating notebooks, figures, and data samples. I want to download some of these files but there are no clear examples. Ideally, I could push (or pull) from a notebook space folder, e.g.,

$HOME//notebooks/project_demo_figures/

...but it appears that folder is not visible outside machines. 

I also tried to use the vospace/public folder but it seems I do not have permission to write into that folder, i.e., 

dsilva> cd $HOME/vospace/public/

dsilva> cp $HOME/notebooks/project_demo_figures/figure_m50_dynamic_substructure_maybe.pdf .
cp: cannot create regular file './figure_m50_dynamic_substructure_maybe.pdf': Permission denied

I also tried to create folders with "mkdir" and failed. Curiously, I could create subfolders in vospace/public using the web API: https://datalab.noirlab.edu/query.php.

I'm guessing there are two issues here: (a) permissions wrong on my public folder; and (b) I need a howto on downloading.

Thanks,

David

Answer 1

Hi Dave, thanks for reaching out.

So, for interacting with your vospace you need to use Data Lab's storeClient. You can do that from a notebook, but since you said you want to download files to local, it's probably best to install the datalab command line client on your local machine:

pip install astro-datalab

Then, log in once:

datalab login

Now you can list, mv, rm, cp files between your remove vospace and your local machine. For instance:

datalab ls --format="long"
datalab get fr="vos://myfile.csv" to="./myfilelocal.csv"

Note that your files don't have to be in vos://public/. You own your entire vospace.

If you have files in your local NB space, you can either copy them to your vospace (in a notebook on the DL NB server, e.g.:

# log in to get a token if you haven't yet
#from getpass import getpass
#from dl import authClient as ac
#token = ac.login(input("username: "),getpass("password: "))

And then use storeClient to copy files:

from dl import storeClient as sc
sc.cp(fr="./file_in_nb_space.csv",to="vos://file_on_vos.csv")

There are other methods in sc.*, for instance mkdir, etc.

BTW,  you can also just right-click on it in the NB server (left-hand folder view), and select "Download"). This also works for the files in your vospace/ folder (which, on the NB server, is a read-only link).

Let us know if you need more pointers,
Robert

Comments

Thanks Robert, this solved my issues. Two followups for posterity:
* One error was not re-authenticating ('datalab login') when I tried the command-line options.
* I did not know about the File > Download option in JuptyerHub

All the best - Dave