Cannot download from particular directory in vospace

Question

I have a handful of directories in my vospace, but there's one particular directory I cannot seem to download from using the datalab get command in my terminal. I get the error "ValueError: No JSON object could be decoded". If I keep the command the exact same but change the path, it works. I re-installed the datalab packages and everything. Sometimes it also runs as if it is downloading but doesn't actually grab any files. My download command looks like this.

datalab get fr=vos://PATHNAME/*

Answer 1

Hi Derrick, thank you for reaching out. Which of your directories specifically is giving you the troubles?

Thanks,

Robert

Comments

I'm actually now realizing all of my directories are not downloading anything. I've stopped getting the error report, but now nothing is being installed. For example, I'm trying this one

datalab get fr=vos://Nov24_2020_z/*

The only other thing I can think of that may be worth mentioning is that I am actively downloading files into another directory in my vospace.

---

Just to make sure: are you logged in with the datalab command line tool? You can check via

datalab whoami

If it says 'anonymous', log in via

datalab login

and then try the 'get' command again.

Please let us know if that didn't solve the problem.
Robert

---

I am logged in, it still seems to fail

---

Thank you for confirming it. We are investigating.
Just one more thing: Does

datalab -v

return 2.18.12? (the latest version)

Thanks,
Robert

---

It does return 2.18.12

---

Thank you. We'll get back to you.

Answer 2

I am unable to reproduce the JSON error, however I can reproduce a problem where the command returns immediately without downloading any files.  Note that with this specific directory there are 4300+ files and the first thing that happens in a wildcard 'get' command is that the directory listing is expanded, for a large directory this can take a while before the downloads begin -- are you sure this isn't what you were seeing?  Adding a "verbose=True" option will have the task print out each file as it's processed and waiting a minute or two should be all that's needed before you see the downloads begin.

OTOH, if the command returns almost immediately without doing anything, it may indicate that there's a problem in your login tokens.  Specifically, that the $HOME/.datalab/dl.conf file specifies one token saying you are logged in, but the $HOME/.datalab/id_token.derrcarr file contains a different token or doesn't exist.  The fix here is to do a "datalab logout" to logout, remove the files in your $HOME/.datalab directory, and then a "datalab login" to login again.  Using your token I was able to download ~2200 files before the disk filled up.

Hope this helps, please let us know if you continue to have problems.