0 votes
178 views
Hello,

is there an easy way to download the full nsc_dr2.object table?

I need to perform an all-sky query to then cross-match to Gaia QSOs, but with sql is taking very long and often crashes even if I do sub-queries and perform some cuts.

Thanks a lot,

Elena
asked Jul 13 by esacchi (120 points) | 178 views

1 Answer

0 votes
Hi Elena, thanks for reaching out. Downloading the entire nsc_dr2.object table will be difficult -- it's very big. We have done that on occasion, but for other institutions, and it required that they have access to Globus configured.

For your cross-match, would it help if you could use the pre-crossmatched table nsc_dr2.x1p5__object__gaia_dr2__gaia_source ? It's the xmatch of nsc_dr2.object against gaia_dr2.gaia_source, with a matching radius of 1.5 arcsec, keeping only the nearest match, and not keeping rows without a match. A JOIN operation on this table is much faster than a dedicated cross-match over the entire NSC DR2...

https://datalab.noirlab.edu/query.php?name=nsc_dr2.x1p5__object__gaia_dr2__gaia_source

Other tips on cutting down the size in your queries:

- only request a few columns, not '*'

- experiment while using subqueries (as you do) and LIMIT statements

- go in slices of RA maybe?
 

Also, please feel free to send us a minimal workflow example that shows what you are trying to achieve, and where it fails. We will of course try to help. You can email to datalab@noirlab.edu

Best regards,

Robert
answered Jul 13 by robertdemo (2,340 points)
Thanks a lot for your quick reply.

I imagined that would be too much, so in the end I uploaded my table and used your online cross-match tool, which was surprisingly fast!

I thought about using nsc_dr2.x1p5__object__gaia_dr2__gaia_source but I also need magnitude columns, not only positions.

Thank you again,
Elena
Glad you got it to work.

The point / bonus of the pre-crossmatched tables is that one can then JOIN on either of the ID columns in those tables (which is much faster than cross-matching). So, you could use the pre-crossmatch tables to find the IDs you need, and then join on those IDs with other tables. Some examples are given in this notebook:

https://github.com/noaodatalab/notebooks-latest/blob/master/04_HowTos/CrossmatchTables/How_to_use_pre_crossmatched_tables.ipynb

Best,
Robert

271 questions

273 answers

311 comments

1,836 users

Welcome to Data Lab Help Desk, where you can ask questions and receive answers from other members of the community.