Problems downloading table from myDB

Question

Dear Helpdesk,

I successfully submitted a query yesterday and specified that the outputs were written to myDB. I am able to see the results of my query (via a Jupyter notebook), however I am then unable to write this out to a table (see below).

[3]: qc.mydb_list()

'NSC2_LMC,created:2021-03-23 15:11:25 MST\n'

[4]: tab = qc.query('SELECT * FROM mydb://NSC2_LMC', fmt='table')

tab.write('NSC2_LMC.fits')

---------------------------------------------------------------------------
queryClientError                          Traceback (most recent call last)
<ipython-input-4-2961a19f2bad> in <module>
----> 1 tab = qc.query('SELECT * FROM mydb://NSC2_LMC' class="ansi-blue-fg">, fmt='table')
      2 tab.write('NSC2_LMC.fits')

/data0/sw/anaconda3/lib/python3.7/site-packages/noaodatalab-2.19.0-py3.7.egg/dl/Util.py in __call__(self, *args, **kw)
     80             return function(self.obj, *args, **kw)
     81         else:
---> 82             return function(*args, **kw)
     83 
     84     def __repr__(self):

/data0/sw/anaconda3/lib/python3.7/site-packages/noaodatalab-2.19.0-py3.7.egg/dl/queryClient.py in query(optval, adql, sql, fmt, out, async_, drop, token, profile, **kw)
    399         return qc_client._query (token=def_token(None), adql=adql, sql=optval,
    400                               fmt=fmt, out=out, async_=async_, drop=drop,
--> 401                               profile=profile, **kw)
    402     else:
    403         # optval is (probably) a token

/data0/sw/anaconda3/lib/python3.7/site-packages/noaodatalab-2.19.0-py3.7.egg/dl/queryClient.py in _query(self, token, adql, sql, fmt, out, async_, drop, profile, **kw)
   2028         r = requests.get (dburl, headers=headers, timeout=timeout)
   2029         if r.status_code != 200:
-> 2030             raise queryClientError (r.text)
   2031         resp = qcToString(r.content)
   2032 

queryClientError: Error: QM: MyDB table nsc2_lmc does not exist

So the DB seems to exist, however at the same time, not?! Any help you could provide would be much appreciated.

Regards,

Cameron

Answer 1

Our databases use Postgres which normally allows lower-case only.  Although we've tried to accommodate mixed-case table names there still appears to be a bug in how they are handled with mydb tables.  In the interim I've copied the table as 'nsc2_lmc_lc' which you can access with the query.  We'll fix the problem as soon as we can, until then please try to avoid upper-case in table names and let us know if you continue to have problems.

Comments

Dear Data Lab team,

Thanks for answering my question. I have re-queried using lower case, however this has lead to another problem (it looks like a timeout issue; see the end of the error message below).

ReadTimeout: HTTPSConnectionPool(host='datalab.noao.edu', port=443): Read timed out. (read timeout=300)

The DB has 10,000,000 rows, so I'm assuming I've hit the allowed limit. Is there any chance that such a row limit could be increased and that one can then simply download the whole DB as a FITS file (without the timeout issues)?

Regards,

Cameron