We are happy to share with you the latest developments at Astro Data Lab in this January 2023 newsletter, AAS241 edition!
In this newsletter
- Release of SPARCL spectroscopic data access service
- New datasets at Data Lab
- New Jupyter notebooks
- Data Lab at the AAS241 meeting in Seattle
Release of SPARCL spectroscopic data access service
SPectra Analysis Retrievable Catalog Lab (SPARCL) is our new online service for discovery and retrieval of one-dimensional optical/infrared spectra. SPARCL currently serves 1D spectroscopic data from the Sloan Digital Sky Survey (SDSS), from both the original SDSS spectrograph and the upgraded instrument of the Baryon Oscillation Spectroscopic Survey (BOSS). SPARCL has been designed and tested to support spectra from the Dark Energy Spectroscopic Instrument (DESI), which will be included in the database with the first DESI public data release later this spring. Major elements of SPARCL include:
- Capabilities to discover and query for spectra based on parameters of interest
- A fast service that delivers desired spectra either individually or in bulk
- Documentation and example Jupyter notebooks to help users learn to make use of these elements in their research
The SPARCL Python client can be easily installed locally:
pip install sparclclient
It is also installed by default on Astro Data Lab's Jupyter notebook server. The client can be loaded within a Python session or program via:
from sparcl.client import SparclClient
client = SparclClient()
Additionally, Astro Data Lab's database now holds the sparcl.main table, which enables discovery of spectra using SQL queries. Use the retrieved spectrum IDs to fetch the corresponding spectra with SPARCL. See the "New Jupyter notebooks" section of this newsletter for details on our how-to-use SPARCL notebook.
New datasets at Data Lab
Astro Data Lab has recently incorporated several new datasets for use by the community: Buzzard DR1, DECaPS DR2, DELVE DR2 Mixture Density Network (MDN) photometric redshifts Value-Added Catalog (VAC), Legacy Surveys DR10 (LS DR10), Siena Galaxy Atlas 2020 (SGA-2020), Dark Energy Survey DR1 VACs, and Dark Energy Survey Year 3 cosmology VACs.
| Recently added catalogs at Astro Data Lab. | |||
|---|---|---|---|
| Dataset |
Number of objects |
Survey area (deg2) |
Bands |
| Buzzard DR1 | 3.1 B | 10,316 | DECam ugrizY, Rubin ugrizY, VISTA zYJHKs, Roman YJHK, WISE W1 & W2 |
| DECaPS DR2 | 3.3 B | 2,664 | grizY |
| DELVE DR2 MDN photometric redshifts VAC | 2.5 B | 25,951 | griz |
| DES DR1 VACs | up to 2.3 B | as DES | grizY |
| DES Y3 Cosmo VACs | 399 M | 5,240 | grizY |
| LS DR10 | 2.8 B | 22,645 | griz, W1, W2, W3, W4 |
| SGA-2020 | 383 K | not contiguous | grz, W1, W2, W3, W4 |
The Astro Data Lab team has already crossmatched the DECaPS DR2 and SGA-2020 catalogs with our reference datasets: Gaia DR3 (for astrometry), AllWISE, NSC DR2, unWISE DR1 (for photometry), and SDSS DR17 (for spectroscopy), and vice versa. LS DR10 will be crossmatched very soon. We have also added a few other useful columns such as nest4096, ring256, and htm9 for Healpix-based and Hierarchical Triangular Mesh (HTM)-based sky tessellation use cases. These pre-crossmatched tables are accessible in the schema browser, and through standard TAP/SQL/ADQL queries, like all other catalogs at Data Lab.
The Astro Data Lab team evaluates periodically which external survey datasets we should source, ingest, and serve. We appreciate requests and suggestions from our users. Please contact us at datalab@noirlab.edu to send your request and, if possible, mention an example scientific use case.
Buzzard DR1
Buzzard is a simulated galaxy catalog from a large N-body lightcone simulation. It is the second large-scale simulated datasets available at Astro Data Lab, after LSST SIM DR2 (TRILEGAL). The Buzzard catalog spans one quarter of the sky extending to z=2.3, and is complete to r=27. Each galaxy is assigned a position, velocity, SED, half-light radius, ellipticity, and broad band photometry in a number of relevant passbands via the ADDGALS algorithm. Furthermore, weak lensing shear and convergence are computed with the CALCLENS algorithm and used to lens the broad-band photometry and angular positions. The Buzzard simulation has been used in a large number of cosmological analyses and is well-suited to simulated studies of large scale structure. We are publicly releasing one of a set of nine catalogs. The unreleased catalogs are the same in every respect, but are built on lightcones simulated with different initial seeds. Access to these additional catalogs is available upon request from the catalog creators (Joe DeRose: jderose@lbl.gov). More information about Buzzard DR1 can be found on the Buzzard landing page.
DECaPS DR2
Dark Energy Camera Plane Survey DR2 is a five-band optical and near-infrared survey of the southern Galactic plane with the Dark Energy Camera (DECam) on the 4.0m Blanco telescope at the Cerro Tololo Inter-American Observatory in Chile. The survey is designed to reach past the main-sequence turn-off at the distance of the Galactic center through a reddening E(B-V) of 1.5 mag and with average seeing around 1''. The footprint covers |b| ≤ 10 deg and 6 deg > l > −124 deg, complementary to coverage by Pan-STARRS1. DECaPS DR2 simultaneously solves for the positions and fluxes of all the sources in each image, delivering positions and fluxes of 3.32 billion stars with up to 5 mmag repeatability. Most of these objects are highly reddened and deep in the Galactic disk, probing the structure and properties of the Milky Way and its interstellar medium. More information about DECaPS DR2 can be found on the DECaPS DR2 landing page.
DELVE DR2 Mixture Density Network (MDN) photometric redshifts VAC
Derived sample of photometric redshift estimates for objects in the DELVE DR2 catalog using a Mixture Density Network (MDN) and trained on griz magnitudes and colors. This network outputs the weights, means, and standard deviations of 20 Gaussian distributions, which are then combined into a single probability density function (PDF) from which samples of the photo-z can be drawn. The resulting photo-z information can be accessed using the interfaces described on the Data Access page. For column names, descriptions and data types, please see delve_dr2.photoz. More information about the DELVE DR2 photo-z VAC can be found on the DELVE DR2 landing page.
DES DR1 VACs
We ingested a variety of value-added catalogs derived and made available from the DES DR1 sample by various groups, including:
BAO sample: group of galaxies optimized to measure Baryon Acoustic Oscillations. IM3SHAPE & SHAPE METACAL: galaxy shape catalogs. MOCK: reproduces angular footprint, redshift distributions and uncertainties, abundance of galaxies, and clustering amplitude as a function of redshift of the BAO sample. MORPH: structural and morphological catalog for 45 million objects. PHOTO_Z: includes derivation and validation of redshift distribution estimates for the populations of galaxies used as weak-lensing sources. PSF: estimation used for weak lensing using PSFEx with somewhat modified parameters. SGSEP: star-galaxy separation catalogs; several catalogs containing different star-galaxy classifications using broad-band photometric data.
| DES DR1 Value-Added Catalogs at Astro Data Lab. | |||
|---|---|---|---|
| Data table |
Number of objects |
Bands |
|
| baosample | 1.3 M | - | |
| im3shape | 443 K | - | |
| mock | 2.3 B | - | |
| mof | 137 M | griz | |
| morph | 55 M | gri | |
| photo_z | 685 M | - | |
| psf | 14 M | grizY | |
| sgsep_cosmos_tests_v2 | 116 K | grizY | |
| sgsep_hsc_tests_v2 | 13 K | grizY | |
| sgsep_stripe82_tests_v2 | 47 K | grizY | |
| sgsep_validation_masked_v3 | 98 M | griz | |
| sgsep_vvds_tests_v2 | 4 K | grizY | |
| shape_metacal_flux_griz | 137 M | griz | |
| shape_metacal_riz_unblind | 137 M | riz | |
DES Y3 cosmology VACs
The Dark Energy Survey Year 3 (Y3) data products used for cosmological analyses, which includes additional information with respect to DR1, and provides the necessary information to reproduce and reanalyze this data set. Data Lab now hosts the y3_gold cosmo catalog with 399M rows, plus the y3_gold_footprint and y3_gold_surveyconditions tables. For more information, please visit DES Y3 Cosmology Data Release.
LS DR10
The latest release from the Dark Energy Spectroscopic Instrument (DESI) Legacy Imaging Surveys team, LS DR10 significantly expands its photometric observations to >20,000 square degrees, and includes entirely new imaging in the i-band, in addition to grz imaging and four infrared bands from NEOWISE. The imaging includes DECaLS imaging taken with the Dark Energy Camera (DECam), as well as imaging associated with the Dark Energy Survey (DES). However, imaging of the Northern sky from BASS and MzLS are not included in LS DR10 and should be obtained from LS DR9. The tractor, apflux, and wise tables contain 2.8 B rows each, and will become fully available at Data Lab during the AAS241 meeting.
SGA-2020
Published by the Dark Energy Spectroscopic Instrument (DESI) Legacy Imaging Surveys team, the Siena Galaxy Atlas 2020 (SGA-2020) is an atlas of 383,620 nearby galaxies primarily selected from the HyperLeda extragalactic database of known large angular-diameter galaxies. The catalog contains detailed information for these galaxies based on the deep, wide-field grz imaging from LS DR9 and the all-sky infrared imaging at 3.4-22 microns from unWISE, spanning nearly 20,000 square degrees. More information about SGA-2020 can be found on the SGA-2020 landing page.
New Jupyter Notebooks
Data Lab curates an extensive collection of notebooks for our user community, which range from introductory, over technical, educational, to entire science use cases. Several new notebooks were recently added to the suite:
1. How to use SPARCL
Author(s): Alice Jacques, Steve Pothier, Adam Bolton, and the Astro Data Lab Spectro Team
This notebook provides a basic introduction to using the SPARCL client
to find and retrieve spectroscopic data within a Python Jupyter
notebook context, giving example uses of each method. The
sparclclient connects to the SPARCL server at NSF’s NOIRLab and provides access
to the contents of the SPARCL database.
2. Science with the LSST simulated catalog LSST SIM DR2 (TRILEGAL)
Author(s): Giada Pastorelli
Color-magnitude and color-color diagrams of the South Galactic Pole
This notebook shows how to retrieve data from the lsst_sim.simdr2 table for the South Galactic Pole (SGP), i.e. sources with galactic longitude b<=−85 degrees. It plots the sources in terms of both color-magnitude and color-color diagrams in several LSST filters. The notebook also shows the corresponding Hess diagrams, color-coded according to stellar and population parameters such as surface gravity, effective temperature, galactic component, etc.
Combining single and binary models: the South Galactic Pole
This notebook shows how to combine the single and binary stars catalogs from LSST SIM DR2 to provide a more realistic description of a real stellar population. The present computation of binary systems represents just one-tenth of the binaries that should be in a "binaries-only" simulation. After combinations, the notebook plots the resulting luminosity functions, CMDs, and color-color diagrams. It again uses stars in the South Galactic Pole.
Data Lab at the AAS241 meeting in Seattle
Astro Data Lab team members will be at the NOIRLab booth, which is part of the NSF Pavilion (Booth #202). Stop by during the following times to see a demo of the Data Lab science platform and collect Data Lab and SPARCL stickers and postcards!
- Mon Jan 9: 9:30-10am, 12-12:30pm, 5:30-6pm
- Tues Jan 10: 9-9:30am, 12-12:30pm
- Wed Jan 11: 5:30-6pm
- Thur Jan 12: 9-9:30am
Contact Us
You can visit our website, use the Helpdesk, reach us via email, and follow us on Twitter.
Currently registered and active users have been subscribed to this Newsletter mailing list. To unsubscribe, send an empty email to datalab-newsletter-unsubscribe@taurus.tuc.noirlab.edu and follow the instructions in the verification email to confirm.
