Database Challenges In The Pan-STARRS Project

Pan-STARRS is an all-sky telescope survey system now under development by the Institute for Astronomy, University of Hawaii. The first telescope in this system, PS1, will become operational in 2008. The PS1 system will support multiple survey programs of the Pan-STARRS PS1 Science Consortium which will operate the PS1 telescope for a 3.5 years.

The 1.8-m PS1 telescope on Haleakala, Maui uses a CCD camera with 1.4 billion pixels to image a 7 square degree piece of sky every thirty seconds. The system will generate an average of 1.4 TBytes of image data per night.

The images taken at the observatory will be transfered to the Maui High Performance Computing Center where they will be processed by the system's Image Processing Pipeline (IPP). The IPP's catalogs of source positions, fluxes, and other attributes will be made available to the consortium scientists through the Published Science Products Subsystem (PSPS). The major component of the PSPS is the Object Data Manager (ODM) which stores the source attributes published by the IPP in a relational database system. Over the course of the PS1 observing, the all-sky photometric and astrometric survey program will record over 140 billion detections through 5 filters from about 5.5 billion distinct astronomical objects in the stationary sky. By the end of the survey, the ODM will occupy over 80 TBytes of disk storage.

The ODM is being developed in a shared-nothing architecture using Microsoft SQL Server. The design leverages the database development work done during the Sloan Digital Sky Survey by the database team in the Physics and Astronomy Department at Johns Hopkins University.

The ODM incorporates two distinct database subsystems, the first for data ingest, detection-to-object correlation and logging, while the second provides SQL support for arbitrary user queries against the object and detection attributes and supporting observational metadata.