Data Lab Help Desk - Recent questions

Possible issue with TAP async jobs remaining queued

Tue, 26 May 2026 08:58:25 +0000

Hello Astro Data Lab Support,

I am experiencing an issue with the TAP service at: https://datalab.noirlab.edu/tap

I am submitting asynchronous TAP queries using pyvo. The jobs are successfully created, but they remain indefinitely in the QUEUED phase and never transition to EXECUTING or COMPLETED.

Could you please check whether there is currently an issue with the TAP async job queue or backend execution service?

Thank you for your help.

How to recover the deleted jupyter notebook

Tue, 26 May 2026 05:43:00 +0000

Accidently I deleted jupyter notebook file. How to recover?

Anomalous geometric clustering of high-pmraerr artifacts in NSC DR2

Tue, 19 May 2026 18:22:42 +0000

Hello Astro Data Lab community,

I have been doing some data analysis on the NSC DR2 catalog via ADQL, specifically filtering for deep-space candidates and tracking pipeline calculation errors. My goal was to isolate cross-epoch tracklet linking failures by targeting extreme kinematic flags.

I used a query filtering for parameters like pmraerr > 90.0 (or extreme proper motion values) combined with an observation gap of deltamjd > 30.0 and ndet >= 3.

When I extracted this dataset and processed the coordinates through a 3D Python projection, I noticed a very unusual statistical distribution. Instead of the pipeline noise and artifacts being scattered randomly across the surveyed sky, the rejected candidates (over 270,000 objects) cluster very tightly—within a 3.0° window—along highly specific, symmetrical geometric axes (forming a hexahedral lattice pattern). A Monte Carlo test adjusting for ecliptic survey bias suggests this clustering is highly non-random.

I have uploaded the specific ADQL query, the Python processing scripts, and the resulting CSV dataset to Zenodo for reproducibility:

https://zenodo.org/records/20178610

My question is regarding the data processing pipeline: What algorithmic or instrumental mechanisms could cause pipeline noise and tracklet failures to align in such a highly structured, non-uniform spatial pattern? Could this geometric clustering be a known artifact of the astrometric reduction pipeline, a systemic bias in how the catalog handles extreme cross-epoch deviations, or an anomaly tied to the underlying survey footprint/tessellation?

Any insights into the pipeline architecture that could explain this specific distribution of errors would be greatly appreciated!

SELECT

id, ra, dec, pmra, pmdec, pmraerr, pmdecerr,

gmag, rmag, imag, ndet, mjd, deltamjd, class_star, flags

FROM nsc_dr2.object

WHERE (

(ra BETWEEN 42.0 AND 48.0 AND dec BETWEEN -3.0 AND 3.0) OR

(ra BETWEEN 132.0 AND 138.0 AND dec BETWEEN -3.0 AND 3.0) OR

(ra BETWEEN 222.0 AND 228.0 AND dec BETWEEN -3.0 AND 3.0) OR

(ra BETWEEN 312.0 AND 318.0 AND dec BETWEEN -3.0 AND 3.0) OR

(ra BETWEEN 357.0 AND 360.0 AND dec BETWEEN 42.0 AND 48.0) OR

(ra BETWEEN 0.0 AND 3.0 AND dec BETWEEN 42.0 AND 48.0) OR

(ra BETWEEN 87.0 AND 93.0 AND dec BETWEEN 42.0 AND 48.0) OR

(ra BETWEEN 177.0 AND 183.0 AND dec BETWEEN 42.0 AND 48.0) OR

(ra BETWEEN 267.0 AND 273.0 AND dec BETWEEN 42.0 AND 48.0) OR

(ra BETWEEN 357.0 AND 360.0 AND dec BETWEEN -48.0 AND -42.0) OR

(ra BETWEEN 0.0 AND 3.0 AND dec BETWEEN -48.0 AND -42.0) OR

(ra BETWEEN 87.0 AND 93.0 AND dec BETWEEN -48.0 AND -42.0) OR

(ra BETWEEN 177.0 AND 183.0 AND dec BETWEEN -48.0 AND -42.0) OR

(ra BETWEEN 267.0 AND 273.0 AND dec BETWEEN -48.0 AND -42.0)

)

AND (pmraerr > 90.0 OR pmdecerr > 90.0 OR gmag > 90.0 OR (pmra*pmra + pmdec*pmdec) > 2500.0)

AND deltamjd > 30.0

AND ndet >= 3

Thanks, Victor

MyDB async crossmatch against ls_dr10.tractor fail with 502 or remain queued, and MyDB tables no longer appear in Data Explorer

Fri, 15 May 2026 14:19:42 +0000

Hello Data Lab team,

I am trying to crossmatch three user-uploaded coordinate tables against ls_dr10.tractor using async/MyDB. I would appreciate your advice on whether this workflow is supported, what chunk size/concurrency limits I should use, and whether there may currently be a TAP async or MyDB metadata issue affecting my account.

Input MyDB tables:

mlspecz_north_trainval              395,097 rows
mlspecz_edfn_layered              1,045,139 rows
mlspecz_wide_north_application    9,918,626 rows

Columns: 
object_id, ra, dec, sample_name, xmatch_chunk

Target catalog: ls_dr10.tractor

The full async crossmatch query for the smallest table failed immediately with "502 Bad Gateway". No async job ID was returned. This suggests the failure occurred during request handling, query planning, or async job creation, rather than during normal job execution.

The query was:

SELECT
  u.object_id,
  u.ra AS input_ra,
  u.dec AS input_dec,
  u.xmatch_chunk,
  q3c_dist(u.ra, u.dec, t.ra, t.dec) * 3600.0 AS sep_arcsec,
  t.ls_id,
  t.ra,
  t.dec,
  t.type,
  t.brick_primary,
  t.release,
  t.brickid,
  t.objid,
  t.maskbits,
  t.fitbits,
  t.flux_g,
  t.flux_r,
  t.flux_i,
  t.flux_z,
  t.flux_w1,
  t.flux_w2,
  t.flux_w3,
  t.flux_w4,
  t.flux_ivar_g,
  t.flux_ivar_r,
  t.flux_ivar_i,
  t.flux_ivar_z,
  t.flux_ivar_w1,
  t.flux_ivar_w2,
  t.flux_ivar_w3,
  t.flux_ivar_w4,
  t.wisemask_w1,
  t.wisemask_w2
FROM mydb://mlspecz_north_trainval AS u
JOIN ls_dr10.tractor AS t
  ON q3c_join(u.ra, u.dec, t.ra, t.dec, 0.0002777777777777778)
WHERE t.brick_primary = 1

The crossmatch radius is 1 arcsec. I also tried rewriting the join using a WHERE-style q3c condition, but got the same 502 error.

To reduce query size, I added object-count chunking using xmatch_chunk and attempted to create MyDB indexes with:

qc.mydb_index(table, 'xmatch_chunk', async_=True)
qc.mydb_index(table, q3c='ra,dec', cluster=True, async_=True)

However, my first chunked notebook submitted many async jobs quickly, which may have overloaded or congested the service. I have now aborted all jobs submitted by me and stopped submitting new chunk batches.

I then checked: https://datalab.noirlab.edu/tap/async

There are still many jobs in QUEUED status, but they appear to be mostly from other users. There are currently no jobs with status EXECUTING. Whenever I submit a new job now, it remains in QUEUED.

There is also a separate MyDB/Data Explorer issue.

After I tried to remove/re-upload or clean up my uploaded MyDB tables, the MyDB panel in the Data Explorer web interface became empty. The MyDB section is visible, but no table names are listed. However, I can still access the tables from Jupyter notebooks, so the tables do not appear to be lost.

Also, I previously removed this table:

mlspecz_lsdr10_north_trainval

but this table name still appears in qc.mydb.list(), while the table itself is no longer accessible. This looks like a Data Explorer/MyDB metadata listing issue, stale cache, or possible MyDB registry inconsistency.

Could you please advise on the following?

1. Is a MyDB-to-ls_dr10.tractor Q3C crossmatch of this size, with user tables larger than 10^5 rows, expected to work through Data Lab async?

2. If the full-table query is not feasible, what chunk size and async-job concurrency limit do you recommend?

3. Can you check whether the TAP async queue is currently stalled or not dispatching jobs?

4. Can you check whether my MyDB metadata is inconsistent, given that my tables are accessible from notebooks but no longer appear in the Data Explorer MyDB panel?

For now I have paused the full workflow to avoid submitting more jobs until I know the recommended limits and whether the current queue/MyDB behaviour is normal.

Thank you very much.

I cannot download a table that i queried

Tue, 12 May 2026 14:40:23 +0000

A error appears every time I try to download a result table from a sync query.

The preview table appears to me without any problems, but when i try to download it, this errors appears:

Error: IllegalArgumentException: net.sf.jsqlparser.JSQLParserException JSQLParserException: ParseException: Encountered "(" at line 14, column 27.\r\nWas expecting one of:\r\n "INTO" ...\r\n "FROM" ...\r\n "," ...

Missing catalog for cross-match

Tue, 12 May 2026 11:58:11 +0000

I wish to cross-match a personalized catalogue (already uploaded) to a LS10 catalogue using the legacy tool (https://datalab.noirlab.edu/legacy/xmatch.php).

On this tool, I only find ls_dr10.psc_n, ls_dr10.psc_s, ls_dr10.tractor and ls_dr10.tractor_s. However, I would like the aperture fluxes that are stored in ls_dr10.apflux which is not available while appearing in the datalabs (https://datalab.noirlab.edu/data-explorer?showTable=ls_dr10.apflux).
How can I have access to it for cross-matching? Or should I download it and do the cross-match myself?

Thank you in advance for your help

Install python package Ananke

Tue, 05 May 2026 19:21:57 +0000

Hi,
We needed Ananke package to generate mock catalogs and wanted to request for an installation as installing packages under user environments is not supported.
Here is a link to the package.
https://pypi.org/project/ananke/

Datalab Login trouble

Thu, 26 Mar 2026 16:56:39 +0000

Hello,

I am experiencing trouble logging into my datalab account. I went to enter my correct username and password on the login page and saw that my account was locked. Could you please help me with resetting my account? My username is aspatel.

Thanks so much!

Avi

local variable 'job' referenced before assignment

Thu, 05 Mar 2026 13:34:29 +0000

Hi,

I was trying to run the python code

from dl import authClient as ac, queryClient as qc

token = ac.login(input("Enter user name: (+ENTER) "),getpass("Enter password: (+ENTER) "))

query = '''SELECT d.*, z.id, z.targetid, xu.id1, xu.id2, xu.distance, u.unwise_objid, u.fluxlbs_w1, u.fluxlbs_w2, u.fracflux_w1, u.fracflux_w2, u.mag_w1_vg, u.mag_w2_vg, u.nm_w1, u.nm_w2, u.qf_w1, u.qf_w2, u.rchi2_w1, u.rchi2_w2
FROM mydb://desi_star AS d
INNER JOIN desi_dr1.zpix AS z
ON z.targetid=d.targetid
INNER JOIN unwise_dr1.x1p5__object__desi_dr1__zpix AS xu
ON xu.id2=z.id
INNER JOIN unwise_dr1.object AS u
ON xu.id1=u.unwise_objid
WHERE xu.distance < 1'''
try:
qc.query(adql=query,async_=True,wait=True,poll=15,verbose=1, timeout=86400, \
out='vos://desixunwise_star.csv', fmt='csv')
except Exception as e:
print(e.message)

And got this:

Error: local variable 'job' referenced before assignment

Is there something wrong with the code?

Table not found in TapSchema

Wed, 04 Mar 2026 03:46:41 +0000

Hi,

I was running the query

SELECT d.targetid, d.tileid, d.tilelocid, d.z_rr, d.zerr, d.chi2, d.spectype,

d.deltachi2, d.zwarn_mtl, d.z_qn, d.z_qn_conf, d.is_qso_qn, d.ra, d.dec,

d.dchisq, d.ebv, d.flux_g, d.flux_r, d.flux_z, d.flux_ivar_g, d.flux_ivar_r,

d.flux_ivar_z, d.flux_w1, d.flux_w2, d.flux_ivar_w1, d.flux_ivar_w2,

d.mw_transmission_w1, d.mw_transmission_w2, d.maskbits, d.photsys, d.desi_target,

d.bgs_target, d.z_not4clus, g.source_id, g.ra, g.dec, g.l, g.b,

g.phot_bp_mean_mag, g.phot_g_mean_mag, g.phot_rp_mean_mag, g.in_qso_candidates,

z.id, z.targetid, x.id1, x.id2, x.distance

FROM mydb://desi_quasars AS d

INNER JOIN desi_dr1.zpix AS z

ON z.targetid = d.targetid

INNER JOIN desi_dr1.x1p5__zpix__gaia_dr3__gaia_source AS x

ON x.id1 = z.id

INNER JOIN gaia_dr3.gaia_source AS g

ON x.id2 = g.source_id

WHERE x.distance < 1

AND g.phot_bp_mean_mag > 0

using python.

And I got an error message:

IllegalArgumentException: Table [ mydb.desi_quasars_5974 AS d ] is not found in TapSchema

Why did it add _5674 to the name of the table? Is that the cause of the problem?

I'm unable to launch Jupyter notebook server

Tue, 03 Mar 2026 18:54:11 +0000

I last used it about a month ago and now when trying to launch the notebook, it says
Event log
- Server requested
- Spawning server...
- Spawn failed: Server at http://127.0.0.1:42587/user/account/ didn't respond in 60 seconds

A question about the ID format in the desi_dr1.x1p5zpixsdss_dr17__specobj cross-match table

Tue, 03 Mar 2026 12:40:37 +0000

I have a question concerning the details of the desi_dr1.x1p5__zpix__sdss_dr17__specobj table.

This table appears to be the result of a cross-match between SDSS DR17 and DESI DR1, providing parameters that are crucial for matching these datasets. I truly appreciate the work that went into creating this resource; however, I am having trouble understanding the format of the id1 column.

When comparing id1 with most of the ID fields under the desi_dr1 catalog, I found that they do not match. In fact, there is a difference in the number of digits between them. As a result, I am unable to use the IDs from this table to retrieve subsequent DESI data.

Does anyone know how this id1 is constructed or which specific DESI identifier it corresponds to?

I’m unable to install Ananke because the NOIRLab Data Lab environment does not provide a C/C++ compiler (gcc), which is required to build its Galaxia backend, so could compiler support be enabled or an alternative environment recommended?

Tue, 24 Feb 2026 04:08:16 +0000

Hello,

I am trying to install the Python package Ananke in the NOIRLab Data Lab environment, but the installation fails because a C/C++ compiler (gcc) is not available. The build process for dependencies such as galaxia_ananke stops with:

error: command 'gcc' failed: No such file or directory

Since this pipeline requires compiled C++ extensions (Galaxia backend), I’m currently unable to proceed.

Could you please advise whether compiler support can be enabled, or if there is an alternative compute environment suitable for installing and running such software?

Thank you,
Preksha

I want to run a query of the DESI DRR1 QSOAGN table, so that only sources with coordinates within the eROSITA public footprint are returned. The basic syntax is: SELECT TOP 100 * FROM desi_dr1.agnqso I want to get all sources in the eROSITA footprint

Thu, 19 Feb 2026 22:36:26 +0000

I want to run a query of the DESI DRR1 QSOAGN table, so that only sources with coordinates within the eROSITA public footprint are returned

How to download data with no limit in size

Wed, 18 Feb 2026 08:44:17 +0000

How to get a token?

Fri, 13 Feb 2026 11:40:39 +0000

No catalos and SQL offline (Feb 12)

Thu, 12 Feb 2026 20:12:29 +0000

I can not see the catalogs and SQL offline.

Thank you so much for help!

hpix_4092 is given as float

Tue, 03 Feb 2026 17:16:21 +0000

in des_dr2.coverage, hpix_4092 is provided as float, though it should be bigint as shown in the schema browser.
Any way to retrieve hpix_4092 correctly?

I'd like to query LS DR10 sources that are close to the survey detection limit; can you please show me which flags to use to ensure I get decent photometry?

Mon, 02 Feb 2026 13:10:48 +0000

I'd like to query LS DR10 sources that are close to the survey detection limit; can you please show me which flags to use to ensure I get decent photometry? In addition, I found that if I use conditions to filter data from more than one band at the same time, the search time increases significantly compared to filtering only one band. Is this normal?

Table upload to myDB from Web Explorer does not work

Wed, 14 Jan 2026 18:00:46 +0000

I am trying to upload a table into myDB using the data explorer tab. I get the following error -

"Unable to process upload. Verify that the csv headers are present and that they match the table schema (for existing tables)"

I followed this answer thread to troubleshoot the problem. The table is the LOFAR Value Added Catalog (link to Data release and README file). I made sure the file name, table names and column headers are all lower case and allowed characters only as listed on the CSV file formatting requirements help page here. The total file size is ~ 4 GB but this is under the total size limits as described here. Another interesting thing to note is the file is ~ 4GB on my disk, but during the upload process the total size column actually displays the size as 3.7 GB. To check whether data formatting was the problem, I uploaded a truncated subset of the file using only the first two rows and the same header, this file is 2.4 kB. This gives the same error.

The table was converted from the original fits file into ascii.csv using astropy.table.

EDIT: The header and first row is -

```

source_name,ra,dec,e_ra,e_dec,total_flux,e_total_flux,peak_flux,e_peak_flux,s_code,mosaic_id,maj,min,pa,e_maj,e_min,e_pa,dc_maj,dc_min,dc_pa,isl_rms,flag_workflow,id_flag,prefilter,postfilter,lr_fin,optra,optdec,composite_size,composite_width,composite_pa,assoc,id_qual,assoc_qual,blend_prob,other_prob,created,position_from,renamed_from,id_ra,id_dec,uid_l,unwise_objid,id_name,separation,legacy_id,hpx,release,brickid,objid,maskbits,fracflux_g,fracflux_r,fracflux_z,type,ra,dec,pstar,star,anymask_opt,gmmcomp,zphot,zphot_err,var.density,var.tr.noise,var.in.noise,flag_qual,zspec_sdss,zwarning_sdss,plate_sdss,mjd_sdss,fiberid_sdss,z_hetdex,z_hetdex_conf,hetdex_sourceid,z_desi,z_desi_err,desi_sourceid,2rxs_id,xmmsl2_id,resolved,las,z_best,z_source,size,l_144,field,legacy_coverage,lm_flux,lm_size,bad_lm_flux,bad_lm_image,las_from,mag_g,magerr_g,mag_r,magerr_r,mag_z,magerr_z,mag_w1,magerr_w1,mag_w2,magerr_w2,mag_w3,magerr_w3,mag_w4,magerr_w4,wise_src,mass_median,mass_l68,mass_u68,g_rest,r_rest,z_rest,u_rest,v_rest,j_rest,k_rest,w1_rest,w2_rest,r_50,r_50_err,flag_mass
ILTJ000000.03+195152.1,0.0001397891057877132,19.864494342561894,1.3216566180151654,0.916539605380532,1.2329283979607426,0.43208893097044526,0.4658497899333264,0.12294834612392941,S,P359+21,11.100479930452728,8.584910238521983,101.29414881778887,3.1449812567973736,2.110634486533086,49.723586831447754,9.339157273177563,6.139031440183402,101.29414881778908,0.1155058853328228,8,1,-99,0,0.17094795081591677,,,,,,0,,,,,Create initial sources,LR,--,,,--,--,--,,-1,208,,,,,,,,--,,,,--,--,--,,,,,,,,,,,,,,,,,,,,False,18.678314546355125,,,,,Fall,True,,,False,False,Gaussian,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,False

```

Hi there! I think the jupyter server is offline. I am struggling to access my work :-(

Sat, 03 Jan 2026 15:51:54 +0000

There isn't much more to be said, the jupyterlab server is not loading, and if it does load it shows a blank screen. I suspect that something is wrong with the server, as I have been trying all day to access the server but with no success. Please help!

Download failure

Mon, 15 Dec 2025 18:46:20 +0000

I am trying to match a sample of quasars (4200 objects) to DESI DR1 with the following query;

SELECT a.*, b.flux_r, b.flux_ivar_r, b.flux_z, b.flux_ivar_z
FROM mydb://desi_highz AS a, desi_dr1.photometry AS b
WHERE q3c_join(a.ra, a.declination, b.ra, b.dec, 1./3600.);

The query executes without error and displays the first 100 rows of the table, however, when I try to save the output to mydb I get an error and when I try to download the table as a .csv I get a load of junk. If I open the error message in the jobs tab I don't get anything meaningful - does anyone have any ideas?

Can't upload table to My DB

Wed, 12 Nov 2025 20:03:52 +0000

Hello, i'm currently trying to upload a 2.5MB .csv that is formatted as follows:

id1_1,id1_2

iDR4_3_SPLUS-n19s11_dual_0012382,iDR4_3_SPLUS-n19s11_dual_0010034

iDR4_3_SPLUS-n19s11_dual_0021877,iDR4_3_SPLUS-n19s11_dual_0021851

iDR4_3_SPLUS-n19s07_dual_0006718,iDR4_3_SPLUS-n19s07_dual_0005682

...

but when i try to upload it via qc.mydb_create() i get the error "Connection aborted. ConnectionResetError(10054, "An existing connection was forcibly closed by the remote host", None, 10054, None"

if i try to upload it from the web service i get the error "Unable to process upload. Verify that the csv headers are present and that they match the table schema (for existing tables)".

This is the first table that i try to upload and create to myDB.

Best Regards.

Why am I unable to upload tables to MyDB?

Fri, 07 Nov 2025 02:32:18 +0000

Hi,

I've tried this three different ways and it doesn't seem to want to let me upload a file to MyDB.

The file is a simple two-column CSV with a proper header and comma-separated values:
```
IRASPSC,TMASS_Designation
00254-3317,00275566-3300257
00428+6854,00460012+6910533
00439+1512,00463294+1528320
01037+1219,01062598+1235530
```

I first tried using the Data Explorer. When I try to upload the file, it says "Unable to process upload. Verify that the csv headers are present and that they match the table schema (for existing tables)". This wasn't an existing table, I just created a new name and started the upload.

I then tried using the command line, and after logging in and checking that I am logged in using `whoAmI`, I used `mydb_create` to create a schema and then tried to `mydb_insert` the table, but I get the response `schema "mydb" does not exist`.

Finally, I tried using the Jupyter notebook option online and retried the `mydb_create` and `mydb_insert`, and met with the same error (it says "OK" when I try `mydb_create`, and then fails to `insert`). The only thing I'm able to do with the notebook is to upload the table to VOSpace. However, when I try to copy it over from VOSpace to MyDB, I get the same error.

Installing a custom package on astro data lab

Thu, 06 Nov 2025 03:51:45 +0000

For an ANTARES filter, we're interested in using a microlensing fitting code, BAGLE. It looks like as a user I may not be able to install packages. The version we'd like to install should be pip and conda installable in the next few days. How would I go about getting this package installed?

TLS certificate has expired

Mon, 03 Nov 2025 18:50:31 +0000

The Astro Data Lab TLS certificate expired today.

Unable to make any queries since LS10 outage

Sun, 26 Oct 2025 15:02:04 +0000

Since the outage and recovery last week, I've not been able to make any queries with LS_DR_10 tractor or apflux. Each time I try to make a request, it ends up timing out after 5 minutes. I see that the systems appear to be online, so I am not sure if this issue is isolated to me?

cross-matching query is timing out

Thu, 16 Oct 2025 21:56:09 +0000

Hello,

I am trying to crossmatch a set of observations with the phat_v2.phot_mod table. Below, is my query. I've been able to run this type of query in the past but now datalab is throwing a timeout error after 5 minutes. I've also tried to run it async in python using jobid = qc.query(sql=sql_query, async_=True)

but I get "ERROR" when I check the job status with qc.status(jobid). Any assistance on what I could do next would be greatly appreciated.

dist = 0.00003056 # 0.11 arcsec

sql_query = f"""

SELECT

s.id, s.ra AS alert_ra, s.dec AS alert_dec,

p.ra AS pmod_ra, p.dec AS pmod_dec, p.*,

q3c_dist(s.ra, s.dec, p.ra, p.dec) AS dist_deg

FROM

mydb://alerts_tab_v2 AS s

JOIN

phat_v2.phot_mod AS p

q3c_join(s.ra, s.dec, p.ra, p.dec, {dist})

ORDER BY s.ra, s.dec, p.f814w_vega

"""

result = qc.query(sql=sql_query, fmt='pandas')

I would like to be able to use Galfit. It this possible, and if so, how can I install it and its dependencies?

Wed, 08 Oct 2025 16:39:36 +0000

Dear Astro Datalab support,

I would like to know if it's possible to install Galfit on the server, or if anyone has any advice on how to get a local user installation working within the limited linux environment available. I'd love to be able to work with students with limited computing resources available to them on analyzing host galaxy morphologies on the datalab if it's possible to do so! I got stuck trying to install dependencies (cfitsio; I tried with conda through the conda-forge channel and encountered a number of issues). If anyone has any insights or experience, please let me know!

Best,

Kristina

Cannot access the https://astrosparcl.datalab.noirlab.edu

Wed, 01 Oct 2025 10:59:33 +0000

Dear Support,

Reporting the problem that I cannot access the SPARCL service via the sparclclient python package, receiving the error at the following Jupiter notebook cell:

client = SparclClient(connect_timeout=30, read_timeout=30)

client
Error:
ServerConnectionError Traceback (most recent call last) Cell In[2], line 1 ----> 1 client = SparclClient(connect_timeout=300, read_timeout=5400) 2 client File ~/anaconda3/envs/sparcl_lab/lib/python3.10/site-packages/sparcl/client.py:218, in SparclClient.__init__(self, url, verbose, show_curl, connect_timeout, read_timeout) 216 if urlparse(url).hostname in _pat_hosts: 217 msg += "Did you enable VPN?" --> 218 raise ex.ServerConnectionError(msg) from None # disable chaining 220 self.apiversion = float(verstr) 222 expected_api = SparclClient.KNOWN_GOOD_API_VERSION ServerConnectionError: [SRVCONER] Could not connect to https://astrosparcl.datalab.noirlab.edu/sparc/version/. HTTPSConnectionPool(host='astrosparcl.datalab.noirlab.edu', port=443): Max retries exceeded with url: /sparc/version/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x77bd91d6bf40>: Failed to establish a new connection: [Errno 111] Connection refused'))Did you enable VPN?

I checked the server via https://www.isitdownrightnow.com/, and it seems to be down. Could you please tell me when the server is expected to be up, if it is expected?

datalab storeClient public vospace not working

Sun, 28 Sep 2025 03:00:00 +0000

Hello,

I have a set of files stored in my public directory: aspatel://public/di_packets_F475W. However, when I try to access them when signed into another datalab other than aspatel using,

sc.get("aspatel://public/di_packets_F475W", "DI_x21_y81.fits")

I get,

['Error: The user does not have sufficient access privileges for this operation.']

This does not happen when I retrieve files from my other public vospace directory aspatel://public/di_packets_v3 and when signed into a different datalab account (other than aspatel). Any assistance would be helpful.

DESI DR1 VAC

Wed, 24 Sep 2025 10:04:28 +0000

Would it be possible to add the DESI DR1 value-added catalogs to the database?

Datalab Fails Again

Mon, 22 Sep 2025 20:49:30 +0000

Hi, I was using datalab earlier to access Euclid Q1 data. But suddenly it started to fail and unable to load catalog and perform any sql search etc. Cleared cookies, tried other browsers, same issue.

It is down or it's user-end issue? Because the system seems green and online.

It's still not working

Fri, 12 Sep 2025 20:02:44 +0000

Hi, it is still failing. The catalog metadata is not unreachable.

And the Xmatch, either on the data explorer page or the legacy Xmatch tool, non is working. Warning messages disappear, but uploading table results in stuck page and become unresponsive.

Clear cache and rerun, no improvement. Change browser does not work (Chrome and Safari).

Data Explorer Down

Fri, 12 Sep 2025 18:22:39 +0000

It is down. Cannot access any catalog or metadata.

Cannot upload table nor do cross-match.

Attempted on legacy X-match fails, with warning "502 bad gateway, ngix/1.28.0".

Similar issue with https://datalab.noirlab.edu/help/index.php?qa=2446&qa_1=noirlab-datalab-is-down-cannot-access-catalog-metadata

Noirlab/datalab is down, cannot access catalog metadata

Wed, 20 Aug 2025 08:01:58 +0000

Dear astro data lab team,

I am trying to find column descriptions to a Euclid_Q1 catalog I downloaded earlier.

On https://datalab.noirlab.edu/data-explorer, on both firefox and epiphany(chrome), I get "Error: unable to load catalogs", whether logged in or not.

The "systems online" status tool says everything is online.

There is no static website on noirlab anymore that has column descriptions. This is unfortunate.

It is not possible to fall back on or create an archive.org copy of the noirlab data documentation pages. https://web.archive.org/web/20250820075851/https://datalab.noirlab.edu/data-explorer?showTable=euclid_q1.object

It would be great if FITS files downloaded with the "datalab" tool had their column descriptions (TCOMM) filled in.

Cheers,
Johannes

i was trying to login in my account in notebook, it was giving the output anonymous.

Fri, 15 Aug 2025 18:26:51 +0000

i was trying to login in my account in notebook, it was giving the output anonymous. However i was able to login using the same credential in astrodatalab but Now if i am trying to login in astrodata lab its saying your account has been locked and the status of my account is showing suspended. What is the problem here can you please resolve it ASAP i need to download data from legacy survey dr10 catalogue.

LSST SIM reference needs updating

Thu, 14 Aug 2025 23:43:56 +0000

This page: https://datalab.noirlab.edu/data/lsst-sim indicates that users of the LSST-SIM dataset should visit the Rubin citations page for info on how to cite. But since this simulation is not distributed as part of the rubin_sim package, there is no reference to it. There is now instead a published paper: https://iopscience.iop.org/article/10.3847/1538-4365/ac7be6 . Can the page be updated with this new reference? I believe I created this page originally, but with the new web pages am not sure how I would change it myself.

Looks like the stale DB problem is back!

Wed, 13 Aug 2025 18:18:01 +0000

Hi! Mydb appears to be down again, just thought I'd let you know. Thanks!

SIA server down?

Mon, 11 Aug 2025 17:44:02 +0000

Hi,

I'm trying to request an image cutout using the sia package but got this error: DALServiceError: 504 Server Error: Gateway Time-out for url: https://datalab.noirlab.edu/sia/des_dr1?POS=13.19%2C-26.59&SIZE=0.2422934725574616%2C0.21666666666666667&FORMAT=ALL&INTERSECT=OVERLAPS&VERB=2.

Probably it's because the server is down?

Best,

Yize

Can't access personal tables in myDB

Thu, 07 Aug 2025 19:34:18 +0000

Hello!

I've been working quite a bit with some tables previously uploaded to myDB. Historically this has been working quite well. Today, however I noticed that I can no longer run queries with QueryClient that match against tables in myDB

(e.g. I get an error that looks like this:

dl.queryClient.queryClientError: Error: relation "mydb.snr10_matched225" does not exist

LINE 2: FROM mydb."snr10_matched225" AS d,

)

I've also checked the Data Explorer itself to see if these tables accidentally got deleted or something; they do seem to still be there. However, I can't query them from inside the Data Explorer either; I get the message One or more tables provided in the query were not found with any query involving these tables. Interestingly, querying against these tables worked successfully as recent as Aug 5; re-running queries that worked then no longer run. I also don't seem to be able to successfully re-upload identical copies of these tables to myDB. Is there a way to fix this?

unable to load a csv catalogue

Fri, 25 Jul 2025 06:44:04 +0000

Hi, I'm trying to upload a new csv catalogue in myDB (less than 500 KB) but I continue to get an error: "Unable to process upload. Verify that the csv headers are present and that they match the table schema (for existing tables).". I can not understant what the problem is. Thanks.

The first rows of my file are:

ra,dec,name

135.5244181209099,-1.95698313545906,1

csv catalogue

Fri, 25 Jul 2025 06:32:09 +0000

how do I see my user space

Fri, 18 Jul 2025 10:03:07 +0000

502 Bad Gateway

Wed, 09 Jul 2025 19:28:56 +0000

'Bad Gateway' error return when trying to run an SQL query

Thu, 26 Jun 2025 21:04:20 +0000

I am trying to run an SQL query using qc.query with a successfully logged-in token via the python astro_datalab package, which I have previously run successfully. This now returns a queryClient.queryClientError with a 502 Bad Gateway HTML message. It appears that the backends are up according to the website - is there another reason I might be hitting this sort of error?

Jupyter server down?

Thu, 26 Jun 2025 15:02:16 +0000

Hi team - it seems the Jupyter notebook server is down?

Dave

Installing celerite and jplephem

Wed, 25 Jun 2025 00:16:36 +0000

Hi! I'm working on an ANTARES filter and am installing a package that requires celerite and jplephem. I tried just conda installing celerite in a terminal, but I got the below error. Is there some preferred way to install packages on datalab?

WARNING conda.exception_handler:print_unexpected_error_report(167): KeyError('active_prefix_name')

Traceback (most recent call last):

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__

    return func(*args, **kwargs)

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main.py", line 83, in main_subshell

    exit_code = do_call(args, parser)

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/conda_argparse.py", line 199, in do_call

    result = getattr(module, func_name)(args, parser)

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/notices/core.py", line 131, in wrapper

    return func(*args, **kwargs)

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main_install.py", line 140, in execute

    return install(args, parser, "install")

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/install.py", line 372, in install

    unlink_link_transaction = solver.solve_for_transaction(

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/core/solve.py", line 148, in solve_for_transaction

    unlink_precs, link_precs = self.solve_for_diff(

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/core/solve.py", line 217, in solve_for_diff

    final_precs = self.solve_final_state(

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda_libmamba_solver/solver.py", line 144, in solve_final_state

    in_state = SolverInputState(

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda_libmamba_solver/state.py", line 186, in __init__

    _supplement_index_with_system(virtual)

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/core/index.py", line 172, in _supplement_index_with_system

    for package in context.plugin_manager.get_virtual_packages():

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/manager.py", line 346, in get_virtual_packages

    return tuple(self.get_hook_results("virtual_packages"))

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/manager.py", line 212, in get_hook_results

    plugins = [item for items in hook() for item in items]

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/manager.py", line 212, in <listcomp>

    plugins = [item for items in hook() for item in items]

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/virtual_packages/cuda.py", line 65, in conda_virtual_packages

    cuda_version = cached_cuda_version()

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/virtual_packages/cuda.py", line 60, in cached_cuda_version

    return cuda_version()

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/virtual_packages/cuda.py", line 35, in cuda_version

    queue = context.SimpleQueue()

  File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/context.py", line 113, in SimpleQueue

    return SimpleQueue(ctx=self.get_context())

  File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/queues.py", line 341, in __init__

    self._rlock = ctx.Lock()

  File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/context.py", line 68, in Lock

    return Lock(ctx=self.get_context())

  File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__

    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)

  File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__

    sl = self._semlock = _multiprocessing.SemLock(

FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/exception_handler.py", line 164, in print_unexpected_error_report

    get_main_info_str(error_report["conda_info"])

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main_info.py", line 398, in get_main_info_str

    return "\n".join(("", *(f"{key:>23} : {value}" for key, value in builder()), ""))

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main_info.py", line 398, in <genexpr>

    return "\n".join(("", *(f"{key:>23} : {value}" for key, value in builder()), ""))

  File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main_info.py", line 358, in builder

    if info_dict["active_prefix_name"]:

KeyError: 'active_prefix_name'

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__

        return func(*args, **kwargs)

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main.py", line 83, in main_subshell

        exit_code = do_call(args, parser)

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/conda_argparse.py", line 199, in do_call

        result = getattr(module, func_name)(args, parser)

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/notices/core.py", line 131, in wrapper

        return func(*args, **kwargs)

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/main_install.py", line 140, in execute

        return install(args, parser, "install")

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/cli/install.py", line 372, in install

        unlink_link_transaction = solver.solve_for_transaction(

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/core/solve.py", line 148, in solve_for_transaction

        unlink_precs, link_precs = self.solve_for_diff(

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/core/solve.py", line 217, in solve_for_diff

        final_precs = self.solve_final_state(

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda_libmamba_solver/solver.py", line 144, in solve_final_state

        in_state = SolverInputState(

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda_libmamba_solver/state.py", line 186, in __init__

        _supplement_index_with_system(virtual)

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/core/index.py", line 172, in _supplement_index_with_system

        for package in context.plugin_manager.get_virtual_packages():

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/manager.py", line 346, in get_virtual_packages

        return tuple(self.get_hook_results("virtual_packages"))

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/manager.py", line 212, in get_hook_results

        plugins = [item for items in hook() for item in items]

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/manager.py", line 212, in <listcomp>

        plugins = [item for items in hook() for item in items]

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/virtual_packages/cuda.py", line 65, in conda_virtual_packages

        cuda_version = cached_cuda_version()

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/virtual_packages/cuda.py", line 60, in cached_cuda_version

        return cuda_version()

      File "/data0/sw/anaconda3/lib/python3.10/site-packages/conda/plugins/virtual_packages/cuda.py", line 35, in cuda_version

        queue = context.SimpleQueue()

      File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/context.py", line 113, in SimpleQueue

        return SimpleQueue(ctx=self.get_context())

      File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/queues.py", line 341, in __init__

        self._rlock = ctx.Lock()

      File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/context.py", line 68, in Lock

        return Lock(ctx=self.get_context())

      File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__

        SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)

      File "/data0/sw/anaconda3/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__

        sl = self._semlock = _multiprocessing.SemLock(

    FileNotFoundError: [Errno 2] No such file or directory

`$ /data0/sw/anaconda3/bin/conda install celerite`

  environment variables:

conda info could not be constructed.

KeyError('active_prefix_name')

An unexpected error has occurred. Conda has prepared the above report.

If you suspect this error is being caused by a malfunctioning plugin,

consider using the --no-plugins option to turn off plugins.

Example: conda --no-plugins install <package>

Alternatively, you can set the CONDA_NO_PLUGINS environment variable on

the command line to run the command without plugins enabled.

Example: CONDA_NO_PLUGINS=true conda install <package>

If submitted, this report will be used by core maintainers to improve

future releases of conda.

Would you like conda to send this report to the core maintainers? [y/N]: n

No report sent. To permanently opt-out, use

    $ conda config --set report_errors false

SIA search using a box in galactic coordinates

Sat, 21 Jun 2025 19:07:30 +0000

From the examples available I see how to query the list of images available in the NOIRLab archive within a box defined by its center in (RA,Dec) and (width,height) using sia.SIAService.

But I need to do it for a box defined in galactic coordinates (l,b). Is there a function to do that?

A full example would be welcome.

Querying ls_dr9.apflux on ls_id returns many blank rows

Wed, 11 Jun 2025 21:02:46 +0000

Hi!

I'm working with a set of galaxies derived from DESI DR1. I've uploaded a table containing their DESI TARGETIDs to a workspace, and I'm attempting to retrieve their forced photometry (i.e. https://datalab.noirlab.edu/data-explorer?showTable=ls_dr9.apflux) from LS DR9. As the DESI targets were identified from LS DR9, it is my understanding that there should be measurements in this table for each DESI target. However, when I run my query (pasted below), I tend to get many empty rows. Am I doing something wrong here, or is it expected that some DESI targets have no associated apflux measurement?

sql = '''SELECT m.TargetID_DESI, a.apflux_g_1, a.apflux_g_2, a.apflux_g_3, a.apflux_g_4, a.apflux_g_5, a.apflux_g_6, a.apflux_g_7, a.apflux_g_8, a.apflux_ivar_g_1, a.apflux_ivar_g_2, a.apflux_ivar_g_3, a.apflux_ivar_g_4, a.apflux_ivar_g_5, a.apflux_ivar_g_6, a.apflux_ivar_g_7, a.apflux_ivar_g_8

FROM mydb://snr5_matched225 AS m

LEFT JOIN desi_dr1.photometry as p

ON m.TargetID_DESI = p.targetid

LEFT JOIN ls_dr9.apflux as a

ON p.ls_id = a.ls_id '''

qu = qc.query(sql=sql,async_=True,wait=True,poll=15,verbose=1,timeout=1800)

df_out = convert(qu)

df_out.to_csv('../apflux_catalogs/apflux_matched_snr5_225.csv')

Data Lab Help Desk - Recent questions

Possible issue with TAP async jobs remaining queued

How to recover the deleted jupyter notebook

Anomalous geometric clustering of high-pmraerr artifacts in NSC DR2

MyDB async crossmatch against ls_dr10.tractor fail with 502 or remain queued, and MyDB tables no longer appear in Data Explorer

I cannot download a table that i queried

Missing catalog for cross-match

Install python package Ananke

Datalab Login trouble

local variable 'job' referenced before assignment

Table not found in TapSchema

I'm unable to launch Jupyter notebook server

A question about the ID format in the desi_dr1.x1p5__zpix__sdss_dr17__specobj cross-match table

I’m unable to install Ananke because the NOIRLab Data Lab environment does not provide a C/C++ compiler (gcc), which is required to build its Galaxia backend, so could compiler support be enabled or an alternative environment recommended?

I want to run a query of the DESI DRR1 QSOAGN table, so that only sources with coordinates within the eROSITA public footprint are returned. The basic syntax is: SELECT TOP 100 * FROM desi_dr1.agnqso I want to get all sources in the eROSITA footprint

How to download data with no limit in size

How to get a token?

No catalos and SQL offline (Feb 12)

hpix_4092 is given as float

I'd like to query LS DR10 sources that are close to the survey detection limit; can you please show me which flags to use to ensure I get decent photometry?

Table upload to myDB from Web Explorer does not work

Hi there! I think the jupyter server is offline. I am struggling to access my work :-(

Download failure

Can't upload table to My DB

Why am I unable to upload tables to MyDB?

Installing a custom package on astro data lab

TLS certificate has expired

Unable to make any queries since LS10 outage

cross-matching query is timing out

I would like to be able to use Galfit. It this possible, and if so, how can I install it and its dependencies?

Cannot access the https://astrosparcl.datalab.noirlab.edu

datalab storeClient public vospace not working

DESI DR1 VAC

Datalab Fails Again

It's still not working

Data Explorer Down

Noirlab/datalab is down, cannot access catalog metadata

i was trying to login in my account in notebook, it was giving the output anonymous.

LSST SIM reference needs updating

Looks like the stale DB problem is back!

SIA server down?

Can't access personal tables in myDB

unable to load a csv catalogue

csv catalogue

how do I see my user space

502 Bad Gateway

'Bad Gateway' error return when trying to run an SQL query

Jupyter server down?

Installing celerite and jplephem

SIA search using a box in galactic coordinates

Querying ls_dr9.apflux on ls_id returns many blank rows

A question about the ID format in the desi_dr1.x1p5zpixsdss_dr17__specobj cross-match table