HTTPError: HTTP Error 500: Internal Server Error using SIA on NSC DR2

Question

Hello Datalab Team,

I'm attempting to download a cutout centered on a measurement from the NSC DR2 however I'm getting an HTTP 500 error code. The specific URL that generates the error is:

http://datalab.noirlab.edu/svc/cutout?col=&siaRef=c4d_130903_045146_ooi_g_ls9.fits.fz&extn=59&POS=325.802,-1.154&SIZE=0.1,0.1

Code Snippet (run from inside a Jupyter notebook):

---

import numpy as np
from pyvo.dal import sia
from astropy.utils.data import download_file

CATALOG_URL = "https://datalab.noirlab.edu/sia/nsc_dr2"
sia_nsc = sia.SIAService(CATALOG_URL)

center = (325.802, -1.154)
mjd = 56538.20262243832
result = sia_nsc.search(center, size=0.1).to_table()
mjds = np.array(result["mjd_obs"]).astype(float)

row = result[(mjds <= mjd + 1e-8) & (mjds >= mjd - 1e-8) & (result["prodtype"] == "image")]
url = row["access_url"][0]
print(url)
filename = download_file(url,cache=True,show_progress=False,timeout=120)

---

Output:

---

http://datalab.noirlab.edu/svc/cutout?col=&siaRef=c4d_130903_045146_ooi_g_ls9.fits.fz&extn=59&POS=325.802,-1.154&SIZE=0.1,0.1

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
/tmp/ipykernel_112098/1098418106.py in <module>
     14 url = row["access_url"][0]
     15 print(url)
---> 16 filename = download_file(url,cache=True,show_progress=False,timeout=120)

/epyc/projects/thor/envs/thor_py39/lib/python3.9/site-packages/astropy/utils/data.py in download_file(remote_url, cache, show_progress, timeout, sources, pkgname, http_headers, ssl_context, allow_insecure)
   1394                 f"included as a valid source.")
   1395         elif len(sources) == 1:
-> 1396             raise errors[sources[0]]
   1397         else:
   1398             raise urllib.error.URLError(
----------- truncated -----------
/epyc/projects/thor/envs/thor_py39/lib/python3.9/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    492         for handler in handlers:
    493             func = getattr(handler, meth_name)
--> 494             result = func(*args)
    495             if result is not None:
    496                 return result

/epyc/projects/thor/envs/thor_py39/lib/python3.9/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    639 class HTTPDefaultErrorHandler(BaseHandler):
    640     def http_error_default(self, req, fp, code, msg, hdrs):
--> 641         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    642 
    643 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 500: Internal Server Error

---

Thanks so much in advance, 

- Joachim

Answer 1

Please try running your script again -- the access_urls for NSC DR2 were missing the collection name meaning the filepath to the parent FITS file was not being found.  This has been fixed and I was able to run your script to retrieve the cutout.

Please let us know if the problem continues.

Comments

The code snippet worked as intended. Thanks for the super fast turnaround!

---

Hi @datalab,

It looks like there may be a new issue with the NSC DR2 access URLs. I am not able to run the code snippet above and now get another HTTP Error 500: Internal Server Error.

---

Please try again.  We recently had to switch to a backup database that still had the old (incorrect) URLs.  That backup DB has been updated and the example code now runs for me.