Query unexpectedly running for two days

Question

About two days ago I ran a query with the following command:

query = 'select * from lsst_sim.simdr2 where random_id between 42 and 42.01 LIMIT 100000'
result = qc.query(sql=query,async_=True, fmt='table', out='mydb://random_100000_lsst_trilegal')

I believe it corresponds to 'qihdc0uu6eb8a7ni' and when I check the qc.status() it says it's executing.

When I try and access the table via: 

df = qc.query("select * from mydb://random_100000_lsst_trilegal", fmt='table')

it's empty.

The command has been running for ~2 days, but that seems suspiciously long. Is there something I've done incorrectly?

Answer 1

Hi Natasha,

The query did completed, in fact only took a few minutes. Unfortunately the job didn't update its status because the underlying process managing the job metadata got restarted losing the previous state. (BTW we are working on an updated system that minimizes these occurrences)

Saying that you can get your results by simply calling the querymanager "results" API with the job id you already have.

E.g.

from dl import authClient               # for Data Lab login

from dl import queryClient as qc        # Query Manager client interface

token = authClient.login ('nsabrams', **your pw**)

job_id = 'qihdc0uu6eb8a7ni'

result = qc.results(token, job_id, fname="job_result.txt")

job_result.txt should have 100k lines which is what your job asked for.

Sorry for the inconvenience,

Igor