Tried using storeClient for upload but it didn't work

Question

Thanks for clarifying. Sorry for replying so late.

I've tried uploading a .csv file now by using the storeClient. I've renamed one of my recent query results (a .csv file) to 'test.csv' and copied it to my 'tmp' directory ('/tmp/test.csv'). The python notebook code looked like this:

from dl import storeClient as sc
sc.put(fr='/tmp/test.csv',to='vos://catwxvhs/test.csv')

The code returns 'OK' indicating success but if I check the 'catwxvhs' directory in VOspace, it is empty. Am I doing something wrong? I also tested copying from my home directory in case it's a permissions problem but with the same result. 'OK' but file doesn't upload.

Then I tested with adding a token to the code:

from dl import authClient as ac, storeClient as sc
from getpass import getpass

# Authentication
token = ac.login(input("Enter user name: (+ENTER) "),getpass("Enter password: (+ENTER) "))
if not ac.isValidToken(token):
    raise Exception('Token is not valid. Please check your usename/password and execute this cell again.')

sc.put(token, fr='/tmp/test.csv',to='vos://catwxvhs/test.csv')

Also same result.

Answer 1

Hi Martin, just a double-check: you are working on the Data Lab notebook server? Then '/tmp/' is not your directory (mind the leading slash sign). Maybe your file is in your own 'tmp/' directory? (no leading slash). Unfortunately, because of the way that the "terminal" is accessing the underlying file system, the tilde '~' doesn't work to indicate your home directory, and '/home/yourusername/' also doesn't resolve. What works is to walk backwards the path from where your notebook is, to where the 'fr' file is, for instance:

sc.put(fr='../../../../tmp/test.csv',to='vos://test0003.csv',verbose=True)

The verbose=True flag shows you a progress bar for the upload. Hope that helps.
And I confirm that the sc.put() method still prints OK even if the 'fr' file does not exists; we will fix the error messages in a future iteration.

Best,
Robert

Comments

Again sorry for answering late. Just like NOIRlab most recently, I had a hard disk failure and had to deal with a lot of issues.

> … you are working on the Data Lab notebook server? Then '/tmp/' is not your directory (mind the leading slash sign).

I'm not sure what exactly that means or how to answer that. In fact my home OS was Ubuntu 20.04 and now is Manjaro Linux. When 'working on the Data Lab notebook server' means having a Firefox browser session open in JupyterLab (https://gp02.datalab.noirlab.edu/user/martinkb/lab)  then 'yes'. Otherwise I'm not sure what you mean with 'on the Data Lab notebook server'

The leading slash was correct since I really uploaded my local .csv file into the /tmp/ directory on my local harddisk. When it didn't work, I also tried to upload it from my local home directory at '/home/username'. Upload didn't work either.

The code I tried is noted at https://gp02.datalab.noirlab.edu/user/martinkb/lab/tree/catxvhs/misc.ipynb in the section 'Vospace' labeled 'Copy files to VOspace (sc.put)' and basically the one pasted in the first post. When I try to run it with Ctrl+Return it returns 'OK' but doesn't seem to upload.

Let's say a file 'test.csv' is saved in my local home directory (the machine sitting under my desk) at '/home/username'. What would the 'fr=' part have to look like?