ADASS 2023 Tutorial: Exploring big data efficiently with the Astro Data Lab science platform
Please join us either in-person or vitually at the ADASS 2023 conference in Tucson, Arizona. We will host a tutorial session about the Data Lab and how it connects with other data-oriented services at NOIRLab on Sunday November 5, 2023 at 3:10pm MST. The full tutorial description is listed below.
In the era of data-intensive astronomy the community needs to acquire skills to handle increasingly larger and more complex datasets, and to access high-performance computing and analysis tools. In this tutorial we will teach participants how to use data-proximate science platforms to conduct astronomy research. Using the Astro Data Lab science platform and the new SPARCL (SPectra Analysis and Retrievable Catalog Lab) service for spectroscopy, participants will first learn how to find documentation, information about all of Astro Data Lab’s data holdings of over 100 TB of wide-field survey catalogs, 2.5 PB of imagery, and millions of spectra from DESI and SDSS, and how to access help from the Astro Data Lab team. We will then teach the group in an interactive mode how to use various data services and analysis tools at Data Lab, including how to crossmatch tables, build and submit catalog queries, search for images and create cutouts, search for and download spectra, and how to use the Astro Data Lab Jupyter notebook server. The participants will execute and modify a number of science-case example notebooks from various domains of astronomy focusing on data analysis. The tutorial will also make use of some amenities on science platforms, including remote file storage and remote user-owned database tables.
Primary learning objectives:
- Construct SQL queries to query large datasets through a dedicated Jupyter Notebook server, web-interface, and command-line interface
- Discover and query for Sloan Digital Sky Survey (SDSS) and Dark Energy Spectroscopic Instrument (DESI) spectra with the new SPARCL tool
- Create plots with the data obtained to realize the graphical and visualization capabilities within our notebooks
- Get image cutouts for a set of objects using an image discovery service and a cutout tool
Participants will need:
- A laptop
- Connection to the internet
- A Data Lab account (sign up here: https://datalab.noirlab.edu/account/register.html)
- Optionally: participants can work with their own data by preparing a CSV data table with at least RA, Dec coordinates for crossmatching (the tutorial will show how to generate such a table)
Tutorial notebooks are available on GitHub here
For pre-tutorial questions/support, please email datalab@noirlab.edu
