Data Surveys
Data Surveys
Astro Data Lab hosts over 30 astronomical surveys, both images and catalogs, published across the astronomical spectrum, ranging from nearby asteroids and local universe to far, extragalactic sources, generally hosted with the approval and/or help of their authors. For some datasets, we host more than one data release.
Survey Sky Maps
Click on a map below to learn more about the dataset.
Catalogs
Reference Catalogs
From these surveys, we have considered 5 as the most important and used them as reference to crossmatch with all other surveys. They are AllWISE, Gaia DR3 (for astrometry), NSC DR2, SDSS DR17 (for spectroscopy), and unWISE DR1 (for photometry), and vice versa. All main object tables have been crossmatched against our default reference datasets within a 1.5 arcsec radius, nearest neighbor only. These tables will appear with x1p5 in their name in our table browser. Example: phat_v3.x1p5__phot_mod__gaia_dr3__gaia_source.
Added Columns
For convenience to our users, we have added some of the following columns to several of the catalogs when they did not have them originally:
- elon - Ecliptic Longitude (deg)
- elat - Ecliptic Latitude (deg)
- glon - Galactic Longitude (deg)
- glat- Galactic Latitude (deg)
- htm9 - Hierarchical Triangular Mesh (HTM) index (order 9 => ~10 arcmin size)
- ring256 - HEALPIX index (Nsides 256, Ring scheme => ~14 arcmin size)
- nest4096 - HEALPIX index (Nsides 4096, Nest scheme => ~52 arcsec size)
- random_id - Random ID assigned for random sampling, uniformly spread across all catalog rows, ranging from 0 to 100
Some names for these columns may be different in some catalogs.
In some photometric catalogs that have fluxes, we have also calculated AB magnitudes, using the formula
mag = 22.5 - 2.5log10(flux)
where the flux is the flux of the object in nanomaggies. We added them, generally following the standard nomenclature, depending on the band used. For example:
- mag_g
- mag_r
- mag_i
- mag_z
If the inverse flux variance is provided, we usually calculated signal-to-noise ratio values based on the formula
snr = flux * sqrt(flux_ivar)
where flux_ivar is the inverse flux variance. We added them with the usual nomenclature. For example:
- snr_g
- snr_r
- snr_i
- snr_z
Some catalogs have other columns calculated and added as well, like de-reddend fluxes and magnitudes, colors, etc.
Caveat: Data Type Discrepancies
Some data types in our TapSchema may not exactly match how they are stored in the underlying database. These discrepancies primarily affect columns labeled as DOUBLE, which in certain cases are implemented differently behind the scenes. While these differences generally do not impact standard usage, they may be relevant for advanced queries or data handling. The following subsections outline the specific cases where such mismatches occur and how they are represented.
NUMERIC
Be aware that some columns are labeled as DOUBLE in our TapSchema but are actually stored as NUMERIC in our database. If there is a mismatch, it will be noted in the column description. Column descriptions for each table are available on our Data Explorer page. For convenience, each column with the NUMERIC datatype is also listed below:
ARRAY
Be aware that some columns are labeled as DOUBLE in our TapSchema but are actually stored as ARRAY (or double precision[]) in our database. If there is a mismatch, it will be noted in the column description. Column descriptions for each table are available on our Data Explorer page. For convenience, each column with the ARRAY (or double precision[]) datatype is also listed below:
| Table Name | Column Name | TapSchema Datatype | Database Datatype |
|---|---|---|---|
| desi_dr1.mws | covar | DOUBLE | double precision[] |
| desi_dr1.mws | elem | DOUBLE | double precision[] |
| desi_dr1.mws | elem_err | DOUBLE | double precision[] |
| desi_dr1.mws | param | DOUBLE | double precision[] |
See the example code below on how to query and convert these columns into a matrix value:
=================================
from dl import queryClient as qc
import numpy as np
import pandas as pd
from io import StringIO
query = '''SELECT * FROM desi_dr1.mws
LIMIT 10
'''
# Save the query in default format (string with comma-separated values).
res = qc.query(sql=query)
# Starting from 'res' as the result from the qc.query() call in default
# format (csv string), the following properly formats the array columns.
# Those need to be listed manually.
# Define the converter to split the array values and convert to float.
# Note: the x[1:-1] gets rid of the opening and closing curly brackets
# in the CSV string.
float_converter = lambda x: np.array(x[1:-1].split(','), dtype='float')
# Define the converters for all array columns.
converters = {'covar': float_converter,
'elem': float_converter,
'elem_err': float_converter,
'param': float_converter}
# Read into a pandas DataFrame.
df = pd.read_csv(StringIO(res), converters=converters)
=================================
