Data Lab Help Desk - Recent questions without answers

I cannot download a table that i queried

Tue, 12 May 2026 14:40:23 +0000

A error appears every time I try to download a result table from a sync query.

The preview table appears to me without any problems, but when i try to download it, this errors appears:

Error: IllegalArgumentException: net.sf.jsqlparser.JSQLParserException JSQLParserException: ParseException: Encountered "(" at line 14, column 27.\r\nWas expecting one of:\r\n "INTO" ...\r\n "FROM" ...\r\n "," ...

Missing catalog for cross-match

Tue, 12 May 2026 11:58:11 +0000

I wish to cross-match a personalized catalogue (already uploaded) to a LS10 catalogue using the legacy tool (https://datalab.noirlab.edu/legacy/xmatch.php).

On this tool, I only find ls_dr10.psc_n, ls_dr10.psc_s, ls_dr10.tractor and ls_dr10.tractor_s. However, I would like the aperture fluxes that are stored in ls_dr10.apflux which is not available while appearing in the datalabs (https://datalab.noirlab.edu/data-explorer?showTable=ls_dr10.apflux).
How can I have access to it for cross-matching? Or should I download it and do the cross-match myself?

Thank you in advance for your help

Install python package Ananke

Tue, 05 May 2026 19:21:57 +0000

Hi,
We needed Ananke package to generate mock catalogs and wanted to request for an installation as installing packages under user environments is not supported.
Here is a link to the package.
https://pypi.org/project/ananke/

local variable 'job' referenced before assignment

Thu, 05 Mar 2026 13:34:29 +0000

Hi,

I was trying to run the python code

from dl import authClient as ac, queryClient as qc

token = ac.login(input("Enter user name: (+ENTER) "),getpass("Enter password: (+ENTER) "))

query = '''SELECT d.*, z.id, z.targetid, xu.id1, xu.id2, xu.distance, u.unwise_objid, u.fluxlbs_w1, u.fluxlbs_w2, u.fracflux_w1, u.fracflux_w2, u.mag_w1_vg, u.mag_w2_vg, u.nm_w1, u.nm_w2, u.qf_w1, u.qf_w2, u.rchi2_w1, u.rchi2_w2
FROM mydb://desi_star AS d
INNER JOIN desi_dr1.zpix AS z
ON z.targetid=d.targetid
INNER JOIN unwise_dr1.x1p5__object__desi_dr1__zpix AS xu
ON xu.id2=z.id
INNER JOIN unwise_dr1.object AS u
ON xu.id1=u.unwise_objid
WHERE xu.distance < 1'''
try:
qc.query(adql=query,async_=True,wait=True,poll=15,verbose=1, timeout=86400, \
out='vos://desixunwise_star.csv', fmt='csv')
except Exception as e:
print(e.message)

And got this:

Error: local variable 'job' referenced before assignment

Is there something wrong with the code?

Why am I unable to upload tables to MyDB?

Fri, 07 Nov 2025 02:32:18 +0000

Hi,

I've tried this three different ways and it doesn't seem to want to let me upload a file to MyDB.

The file is a simple two-column CSV with a proper header and comma-separated values:
```
IRASPSC,TMASS_Designation
00254-3317,00275566-3300257
00428+6854,00460012+6910533
00439+1512,00463294+1528320
01037+1219,01062598+1235530
```

I first tried using the Data Explorer. When I try to upload the file, it says "Unable to process upload. Verify that the csv headers are present and that they match the table schema (for existing tables)". This wasn't an existing table, I just created a new name and started the upload.

I then tried using the command line, and after logging in and checking that I am logged in using `whoAmI`, I used `mydb_create` to create a schema and then tried to `mydb_insert` the table, but I get the response `schema "mydb" does not exist`.

Finally, I tried using the Jupyter notebook option online and retried the `mydb_create` and `mydb_insert`, and met with the same error (it says "OK" when I try `mydb_create`, and then fails to `insert`). The only thing I'm able to do with the notebook is to upload the table to VOSpace. However, when I try to copy it over from VOSpace to MyDB, I get the same error.

I would like to be able to use Galfit. It this possible, and if so, how can I install it and its dependencies?

Wed, 08 Oct 2025 16:39:36 +0000

Dear Astro Datalab support,

I would like to know if it's possible to install Galfit on the server, or if anyone has any advice on how to get a local user installation working within the limited linux environment available. I'd love to be able to work with students with limited computing resources available to them on analyzing host galaxy morphologies on the datalab if it's possible to do so! I got stuck trying to install dependencies (cfitsio; I tried with conda through the conda-forge channel and encountered a number of issues). If anyone has any insights or experience, please let me know!

Best,

Kristina

datalab storeClient public vospace not working

Sun, 28 Sep 2025 03:00:00 +0000

Hello,

I have a set of files stored in my public directory: aspatel://public/di_packets_F475W. However, when I try to access them when signed into another datalab other than aspatel using,

sc.get("aspatel://public/di_packets_F475W", "DI_x21_y81.fits")

I get,

['Error: The user does not have sufficient access privileges for this operation.']

This does not happen when I retrieve files from my other public vospace directory aspatel://public/di_packets_v3 and when signed into a different datalab account (other than aspatel). Any assistance would be helpful.

Datalab Fails Again

Mon, 22 Sep 2025 20:49:30 +0000

Hi, I was using datalab earlier to access Euclid Q1 data. But suddenly it started to fail and unable to load catalog and perform any sql search etc. Cleared cookies, tried other browsers, same issue.

It is down or it's user-end issue? Because the system seems green and online.

Noirlab/datalab is down, cannot access catalog metadata

Wed, 20 Aug 2025 08:01:58 +0000

Dear astro data lab team,

I am trying to find column descriptions to a Euclid_Q1 catalog I downloaded earlier.

On https://datalab.noirlab.edu/data-explorer, on both firefox and epiphany(chrome), I get "Error: unable to load catalogs", whether logged in or not.

The "systems online" status tool says everything is online.

There is no static website on noirlab anymore that has column descriptions. This is unfortunate.

It is not possible to fall back on or create an archive.org copy of the noirlab data documentation pages. https://web.archive.org/web/20250820075851/https://datalab.noirlab.edu/data-explorer?showTable=euclid_q1.object

It would be great if FITS files downloaded with the "datalab" tool had their column descriptions (TCOMM) filled in.

Cheers,
Johannes

csv catalogue

Fri, 25 Jul 2025 06:32:09 +0000

datalab python error httplib2

Thu, 08 May 2025 11:26:45 +0000

Datalab, under Python3, won't start up, becaus dl fails to import module httplib2

I am analyzing strong gravitational lenses from DES Y3A1. I am downloading cut out images and using FITS files specified by RA and Dec. The files include MAGZERO but I cannot find the extinction and airmass for each image. Where can I obtain these?

Sun, 23 Feb 2025 08:35:24 +0000

sql request with "or" is slow

Mon, 02 Dec 2024 20:11:29 +0000

the request that contains OR in the query works very slowly. For example:

---

select top 10
pm, ra, dec
from gaia_dr3.gaia_source
where pm < 1

----

and

---

select top 10
pm, ra, dec
from gaia_dr3.gaia_source
where
pm > 2

---

but the query:
select top 10
pm, ra, dec
from gaia_dr3.gaia_source
where pm < 1 OR pm > 2

works very slowly.

Any suggestions?

Follow-up on MPI question

Wed, 20 Nov 2024 09:55:15 +0000

Thank you for the response. I've re-run my script using the same command mpiexec -n 4 python PHAT_Segment.py and am now receiving the following error message:

Traceback (most recent call last):
  File "/data0/sw/anaconda3/lib/python3.10/site-packages/psutil/_pslinux.py", line 303, in <module>
    set_scputimes_ntuple("/proc")
  File "/data0/sw/anaconda3/lib/python3.10/site-packages/psutil/_common.py", line 399, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
  File "/data0/sw/anaconda3/lib/python3.10/site-packages/psutil/_pslinux.py", line 270, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
  File "/data0/sw/anaconda3/lib/python3.10/site-packages/psutil/_common.py", line 711, in open_binary
    return open(fname, "rb", **kwargs)
FileNotFoundError: [Errno 2] No such file or directory: '/proc/stat'

Please let me know how I should proceed.

Optimizing the cutout download procedure for large machine learning datasets

Mon, 18 Nov 2024 11:33:47 +0000

Hello! I am downloading many thousands of cutouts of galaxies from the DES DR2, and as it stands this is a process that takes multiple days. This is even with parallelization of the downloading procedure by spawning several python processes in a shell script loop. The download process is on the order of a couple of minutes for batches of 20 images, where each image is being downloaded by a separate python process. I have made sure that neither the CPU nor RAM are bottlenecks. Is there some way to optimize this procedure through datalab? I am aware of the existence of pre-created datasets such as through galaxy zoo, however I explicitly must work through FITS files, which is why I have avoided these sets.

Only the main, magnitude, and flux tables visible for des_dr1 crossmatch on website xmatch GUI

Mon, 03 Jun 2024 12:09:53 +0000

Hi, I currently have a catalog of galaxy parameters that I obtained from the des_dr1.galaxies table. I thought that I would be able to use the coadd_object_id to match each object to the des_dr1.photo_z table, but apparently the coadd_objects_id in that table is different since none of my objects match. I am therefore trying to use the website's crossmatch service with the alphawin_J2000 (RA) and deltawin_J2000 (DEC) from des_dr1.galaxies to crossmatch with des_dr1.photo_z to obtain a photometric redshift for each galaxy. For some reason, the des_dr1.photo_z table never appears as an option in the dropdown menu when I select des_dr1 as the target table however. As noted in the title, only the des_dr1.{main, magnitude, flux} tables appear. Am I doing something wrong, or has this changed recently? Additionally, is this the best approach to take to obtain the redshifts here, or is there some other metric I should be using to crossmatch? Thanks!

Sorting the cutout images based on the covered area

Sun, 05 May 2024 22:01:17 +0000

Dear Madam/Sir,

I am trying to download cutout images using a list of RA and Dec positions and following the SIA documentation (https://github.com/astro-datalab/notebooks-latest/blob/master/04_HowTos/SiaService/How_to_use_the_Simple_Image_Access_service.ipynb). The desired cutout size is typically a few arcminutes. I have a couple of queries in this regard:

1- Since the desired cutout sizes are relatively large, I often get cutout images that only partially cover the desired area. So, I would like to determine the available cutout sizes before downloading. The documentation page suggests how to select the image with the largest exposure. I would like to know if it is possible to sort the available cutouts based on their sizes so that I can download the largest cutout image. I am aware of the website https://datalab.noirlab.edu/sia.php where one can visually examine the thumbnails before downloading. However, for a large list of RA and Dec positions, this is inefficient.

2- Is it possible to determine the cutout sizes considering more than one filter at a time before downloading? As per the documentation, we can define only one filter at a time for the search.

3- Are there any plans to produce mosaicked images so that the issue of image gaps can be avoided? This will be useful, particularly in cases where one wants a cutout covering a large (e.g., >10-20 arcmin) sky area.

Thank you in advance.

When are going to be available the dr10 photo-z-sweeps tables on the Astro Data Lab site?

Tue, 10 Oct 2023 07:24:21 +0000

I want to search some photo-z from the dr10 last release, similarly to what it is possible for dr9. However the dr10-photoz table ( listed here) is not listed in the Astro Data Lab Query Interface tool.

Download of the entire DELVE DR2 WIDE catalog

Thu, 19 Jan 2023 10:14:19 +0000

Hi all - we're very interested in the DECam DELVE data products - especially the WIDE survey. Is it possible to get a CSV or FITS extract of your entire 2.5 billion row objects table (all filters) that we can build into our crossmatching algorithm - including any star/galaxy separation metrics? We locally curate catalogs (spatially indexed) onto SSD for crossmatch speed.

Finding the corresponding weightma

Wed, 14 Dec 2022 09:13:46 +0000

Hello! Recently, I've downloaded numerous image cutouts corresponding to archival images from DECam (Stacked, Resampled and/or InstCal), but I now realize that I might have some use for the concominant data, specifically the weight maps.

What's the best way to find the corresponding weight map given a FITS file that I currently have? Best I could do was repeat my original search and modify my queries so that they searched for wtmaps while looking for the same OBSID, but not every file has that keyword in the header it seems. There's other keywords like "WTMAP" or "EXPMAP" which appear to be filenames, but I don't know if there's a way to specifically search for these and retrieve a cutout.

For example, I made a cutout on:

FILENAME= 'DEC13A_20130531_866800f-g-20130602T001044_si'

And I'd be trying to make a cutout on:

WTMAP   = 'DEC13A_20130531_866800f-g-20130602T001044_sw[tile22]'

Thanks in advance!

Could I download any observational proposal?

Tue, 29 Nov 2022 02:37:20 +0000

Hello, I am researching on Bullet cluster. So I downloaded the DECam data of which region covers the Bullet cluster(104.611, -55.948).

But I wonder about two things.

(1) Legacy Imageing Survey DR1 ~ DR9, DECam hasn't taken the Bullet cluster's region(DR10 takes the region but it isn't released yet).
But there is the bullet cluster's obs. data in 2012(Proposal ID: 2012B-0001, 2012B-0003. PI is "Frieman")
Does this mean DECam conduct the other observation except for Legacy Imaging Survey?

(2) I want to download the paper related to "Proposal ID: 2012B-0001, 2012B-0003"
I don't know if the paper exists or not whenever there is a proposal.
If there is no paper, Could I see the proposal? because I want to see the reason why the PI submit the proposal.

Thank you for reading my question.

TAP cone search query fails

Wed, 24 Nov 2021 14:04:42 +0000

Hi! I get an error when trying to use the TAP cone search:

Query (from TAP examples):

SELECT
TOP 1000
*
FROM smash_dr2.source
WHERE
1=CONTAINS(POINT('ICRS', ra, dec),
CIRCLE('ICRS', 79.786, -69.656, 0.05 ))

Exception:

java.io.IOException: PSQLException: ERROR: function point(unknown, double precision, double precision) does not exist
  Hint: No function matches the given name and argument types. You might need to add explicit type casts.
  Position: 974
at uk.ac.starlink.vo.TapQuery.readResultVOTable(TapQuery.java:671)
at uk.ac.starlink.vo.TapQuery.executeSync(TapQuery.java:246)
at uk.ac.starlink.vo.TapTableLoadDialog$6.loadTables(TapTableLoadDialog.java:539)
at uk.ac.starlink.table.gui.TableLoadWorker.run(TableLoadWorker.java:118)

query in async mode throws IllegalArgumentException

Fri, 22 Oct 2021 11:15:36 +0000

Im doing the following xmatch with a mydb table I have uploaded called test.

query = "SELECT aw.designation, xm.id1, xm.distance, " \
              "des.ra, des.dec, des.mag_auto_g, des.mag_auto_r, " \
               "des.mag_auto_i, des.mag_auto_z, des.mag_auto_y " \
               "FROM mydb://test as aw, des_dr1.mag as des, des_dr1.x1p5__main__allwise__source as xm " \
               "WHERE aw.source_id=xm.id2 AND xm.id1=des.coadd_object_id"

qc.query(query, fmt='csv', out="vos://test.csv")

When running this query in sync mode it works. When its run in async mode:

qc.query(query, fmt='csv', out="vos://test.csv", async_=True, wait=True, poll=1, timeout=3000)

I get the following error:

Error: IllegalArgumentException: Table [ mydb."test_2916" AS aw ] is not found in TapSchema

As you can see test_2916 is not the name I specified in the query.

A suggestion for NOAO source catalog

Tue, 07 Aug 2018 13:53:06 +0000

Dear whom it may concern, I have noticed that the NSC gives the mjd (Mean MJD) and deltamjd (range of MJD) for each target. I presume that these two parameters are for ALL filters. It would be great that the the same parameters could provide for each individual filter in case of time-series analysis. Many thanks!

running DES example notebook returns the error "buffer is too small"

Thu, 05 Apr 2018 07:41:14 +0000

I'm trying to obtain image cutouts from the DES. Starting with running the example notebook (https://datalab.noao.edu/desdr1/analysis/DwarfGalaxyDESDR1_20171101.html) on Datalab notebook server with no modification, I get the error:

"TypeError: buffer is too small for requested array"

When it attempts to use the defined function "download_deepest_image" at this entree:

"gimage = download_deepest_image(rac, decc, fov=0.1, band=band)"

I'm wondering if you can help with this problem (I cannot attach the entire error message as it goes over the character limit). Also I'm wondering if this is indeed the easiest way available to obtain image cutouts for DES.

Thanks,

Arash

Can I list the status of all query jobs running under my account?

Sun, 28 May 2017 13:18:48 +0000

I made a mistake when submitting an async query, is there a way to kill the job?

Sun, 28 May 2017 13:18:14 +0000

What is a virtual storage 'capability' and how do I use it?

Sun, 28 May 2017 13:08:02 +0000

How do I save images from an archive query to my virtual storage?

Sun, 28 May 2017 13:03:42 +0000

How do I save a data table computed in my notebook/application to MyDB?

Sun, 28 May 2017 12:49:04 +0000

Is it possible to recover files accidentally deleted from virtual storage?

Sun, 28 May 2017 12:47:32 +0000

What do I do if my query runs too long and times out?

Sun, 28 May 2017 12:41:58 +0000

What indexing scheme is used in the database, i.e. are there special functions I can use to optimize queries?

Sun, 28 May 2017 12:41:09 +0000

How do I share MyDB tables or virtual storage files with collaborators?

Sun, 28 May 2017 12:39:53 +0000

How can I get image cutouts of an object? Of a list of objects?

Sun, 28 May 2017 12:37:41 +0000

How do I query a table for multiple positions?

Sun, 28 May 2017 12:36:57 +0000

Can I retrieve an entire directory and its contents from my virtual storage?

Sun, 28 May 2017 12:31:36 +0000

Can I access external data archives using the Data Lab interfaces? How?

Sun, 28 May 2017 12:29:31 +0000

How do I do a simple crossmatch of a MyDB table with something in Data Lab?

Sun, 28 May 2017 12:26:43 +0000