IllegalArgumentException: Table [ … ] is not found in TapSchema

Question

I'm currently trying to code a crossmatch in a Jupyter notebook and keep getting the mentioned error. I have seen it mentioned in two or three other discussions sadly without any solution. The code reads as follows:

from dl import queryClient as qc

query = """
SELECT c.*, v.japermag3
FROM mydb://catwxvhs AS c, vhs_dr5.vhs_cat_v3 AS v
WHERE q3c_radial_query(c.ra, c.dec, v.ra2000, v.dec2000, 0.0014)
AND (v.mergedclass=-1 OR v.mergedclass=-2)
AND v.japermag3 between 18 and 19
AND v.japermag3-c.w2mpro_pm>=2
LIMIT 1
"""

result = qc.query(sql=query, fmt='csv',
                  async_=True, wait=True,
                  timeout=600, poll=5,
                  verbose=1)
print(result)

I've limited the query to just one result (LIMIT 1) for testing purposes but no matter what I do, it always says:

EXECUTING
Status = ERROR; elapsed time: 5, timeout in 595
Retrieving error
Error: IllegalArgumentException: Table [ mydb."catwxvhs_3718" AS c ] is not found in TapSchema

Does anyone know what I'm doing wrong?

Comments

It's been a week. Doesn't anyone maybe have a tip about why this doesn't work? It's basically the same error indicated by the user dpallot back in October of last year at https://datalab.noirlab.edu/help/index.php?qa=1051&qa_1=query-in-async-mode-throws-illegalargumentexception

Answer 1

This problem should now be fixed on the production systems.  I would note however that the specific 'catwxvhs' table still has several active queries running that may be remnants of earlier failed queries and so the table might be in a "funny state" wrt the database.  Please let me know if you'd like me to kill these queries and otherwise cleanup access to that table before trying your query again.

Comments

Hats off to you/the team. Thanks for fixing this bug so quickly.

> Please let me know if you'd like me to kill these queries …

Yes, those were all tests or failed queries. So please kill them all and feel free to clean up. :)

Would you drop me a short note in this discussion once this is done so I know I can start querying again?

---

Hi Martin,  All of your active queries have been killed and the table definitions cleaned up.  Please let us know if you still have problems.

Answer 2

Hi, apologies for the long response time. This is a bug in our ADQL parser. It seems to trip over the "AS c" syntax, but only when used in conjunction with mydb tables (i.e. not when used in conjunction with regular TAP tables).

We think that your query should run as-is but in sync mode (since the ADQL parser is not involved in sync queries). However, the max timeout value for sync queries is timeout=600 (seconds, i.e. 10 minutes). Can you give it a try? Maybe the timeout threshold will suffice? (please note that in unrelated news the remote file storage (vospace) is currently not working, i.e. there might be errors if you try to write out the results to a file on vospace. This should be fixed anytime soon today.)

In the meantime we are looking to fix the parser bug.

Thanks for your patience,

Robert

Comments

Hi Robert,

thanks for answering and taking the time to confirm a bug. I hope it can get fixed easily. :)

I gave the sync query a try. The one with "LIMIT 1" runs successfully but that was planned as test case. I removed the LIMIT for the next run and unfortunately the query times out after 600 seconds.

> ReadTimeoutError: HTTPSConnectionPool(host='datalab.noirlab.edu', port=443): Read timed out. (read timeout=600)

Do you have a rough estimate about when the bug will be fixed?