Working draft

.gitignore

@@ -8,4 +8,6 @@
 *.xdv
 *.dvi
 *.pdf
+*.tdo
 authors.tex
+RTN-114.synctex.gz

RTN-114.tex

@@ -3,13 +3,53 @@
 % Author guide:
 %   https://journals.aas.org/aastexguide/
 
-\documentclass[modern]{aastex7}
+% dvipsnames passed to xcolor
+\documentclass[twocolumn,twocolappendix, linenumbers, twocolappendix,trackchanges, dvipsnames]{aastex7}
 
 % Package imports go here.
 
+
+\usepackage{amsmath}
+\usepackage{textcomp}
+\usepackage{gensymb}
+\usepackage{graphicx}
+\usepackage{mathtools, nccmath}
+\usepackage{hyperref}
+\usepackage[htt]{hyphenat} % Hyphenate texttt blocks
+\usepackage{xspace} % grids of plots
+\usepackage{subcaption}
+\captionsetup{compatibility=false}
+%\usepackage[dvipsnames]{xcolor} % already loaded in by aaxtex7, creates option class otherwise
+
+% TODO notes 
+\usepackage[colorinlistoftodos]{todonotes}
+\let\Oldtodo\todo
+\renewcommand{\todo}[1]{\Oldtodo[inline]{#1}}
+\newcommand{\eric}[2][]{\Oldtodo[inline,color=cyan, #1]{Eric: #2}}
+\newcommand{\ian}[2][]{\Oldtodo[inline,color=red, #1]{Ian: #2}}
+\newcommand{\bruno}[2][]{\Oldtodo[inline,color=green, #1]{Bruno: #2}}
+\newcommand{\bri}[2][]{\Oldtodo[inline,color=purple, #1]{Bri: #2}}
+\newcommand{\jake}[2][]{\Oldtodo[inline,color=yellow, #1]{Jake: #2}}
+\newcommand{\gregory}[2][]{\Oldtodo[inline,color=lime, #1]{Gregory: #2}}
+\newcommand{\melissa}[2][]{\Oldtodo[inline,color=Orchid, #1]{Melissa: #2}}
+\newcommand{\TODO}[1]{\textcolor{red}{\textbf{TODO:} #1}}
+
 % Local commands go here.
 \newcommand{\docRef}{RTN-114}
 \newcommand{\docUpstreamLocation}{\url{https://github.com/lsst/rtn-114}}
+\newcommand{\degsq}{$\rm deg^{2}$}
+
+%\providecommand{\secref}[1]{\hyperref[#1]{\autoref{#1}}}
+\providecommand{\secref}[1]{\hyperref[#1]{\S\ref{#1}}}
+\providecommand{\appref}[1]{\hyperref[#1]{Appendix \ref{#1}}}
+\providecommand{\tabref}[1]{\hyperref[#1]{\autoref{#1}}}
+\providecommand{\figref}[1]{\hyperref[#1]{\autoref{#1}}}
+\providecommand{\eqnref}[1]{\hyperref[#1]{Eq.~\ref{#1}}}
+\providecommand{\recref}[1]{\hyperref[#1]{REC-\ref{#1}}}
+
+
+% All graphics paths here 
+\graphicspath{./}
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %%
@@ -44,19 +84,22 @@
 %%\graphicspath{{./}{figures/}}
 
 
+
+% Include the parameters file with the data
+\input{parameters}
+
+
 \begin{document}
 \doi{10.71929/rubin/3019817}
 \input{authors}
+\correspondingauthor{Eric C. Bellm, Ian Sullivan}
+
 \date{\today}
 \title{Alert Production with the Vera C. Rubin Observatory}
 
 % This can write metadata into the PDF.
 % Update keywords and author information as necessary.
-\hypersetup{
-    pdftitle={Alert Production with the Vera C. Rubin Observatory},
-    pdfauthor={RubinBuilderPaper},
-    pdfkeywords={}
-}
+\hypersetup{pdftitle={\@title}, pdfauthor={\@author}, pdfkeywords={\@keywords}}
 
 \input{abstract}
 
@@ -66,8 +109,24 @@
 %% You can use the \uat command to link your UAT concepts back its source.
 %% \keywords{\uat{Galaxies}{573} --- \uat{Cosmology}{343}}
 
-%% Main content as individual inputs.
-\input{body}
+\listoftodos
+
+\input{intro}
+\input{observations}
+
+\input{data_products}
+\input{pipeline}
+
+\input{processing}
+
+\input{performance}
+
+\input{data_services}
+
+\input{support}
+
+\input{conclusion}
+
 
 
 %% Modify acknowledgments as needed.
@@ -87,13 +146,18 @@
 %% parentheses, after the keyword, but they are not verified.
 
 \vspace{5mm}
-\facility{Rubin}
+\facility{Rubin:Simonyi (LSSTCam), Rubin:USDAC}
 
 %% Similar to \facility{}, there is the optional \software command to allow
 %% authors a place to specify which programs were used during the creation of
 %% the manuscript. Authors should list each code and include either a
 %% citation or url to the code inside ()s when available.
-\software{LSST Science Pipelines}
+\eric{review and update citations to use current DOIs}
+\software{Rubin Data Butler \citep{2022SPIE12189E..11J},
+          LSST Science Pipelines \citep{10.71929/rubin/2570545},
+          LSST Feature Based Scheduler v3.0  \citep{peter_yoachim_2024_13985198, Naghib_2019}
+          Astropy \citep{2013A&A...558A..33A, 2018AJ....156..123A, 2022ApJ...935..167A}
+          }
 
 %% Appendix material should be preceded with a single \appendix command.
 %% There should be a \section command for each appendix. Mark appendix
@@ -106,6 +170,7 @@
 
 %% \appendix
 
+%\printglossaries
 
 % Include all the relevant bib files so that references can be found from
 % lsst-texmf.

abstract.tex

@@ -1,5 +1,7 @@
 
 \begin{abstract}
-Describes the LSST Alert Production pipelines, data products, execution environment, and early performance.
+
+Describes the LSST Alert Production pipelines, data products, execution environment, and early performance. 
+
 \end{abstract}
 

conclusion.tex

@@ -0,0 +1,4 @@
+%\section{Discussion}
+\section{Conclusion} \label{sec:conclusion}
+
+\eric{summarize}

data_products.tex

@@ -0,0 +1,50 @@
+\section{Prompt Data Products} \label{sec:data-products}
+
+
+\eric{
+Summary of relevant aspects of the DPDD \citep{LSE-163}, including latency considerations and user access.
+Describe key requirements that drive technical decisions.
+}
+
+
+
+
+\subsection{Alerts} \label{sec:alerts}
+
+Alert packets are the primary data product for time-critical LSST science.
+Each alert packet is triggered by a single positive or negative detection on a difference image.
+This measurement is packaged with a variety of contextual information in order to enable downstream users to quickly identify events of interest without requiring followup queries to Rubin databases.
+
+Each alert packet contains
+
+\begin{itemize}
+    \item The triggering DIASource record
+    \item The corresponding DIAObject or SSObject record
+    \item Any DIASource and DIAForcedSource records that exist, and difference image noise estimates where they do not, taken from the previous 12 months.
+    \item Cutout images of the science, template, and difference images.
+\end{itemize}
+\eric{confirm contents}
+
+The cutout images are packaged in the FITS format using the astropy \texttt{CCDData} class and include mask and variance planes, an estimate of the PSF, a World Coordinate System, and other metadata.
+Most cutouts are 30$\times$30 pixels but are expanded to a larger size for extended sources.
+The maximimum cutout size of 102$\times$102 pixels allows cutouts to contain trailed Near Earth Asteroids traveling slower than ten degrees per day while avoiding the performance impacts that would result from larger cutouts.
+
+Alerts are transmitted and stored in Apache Avro\footnote{\url{https://avro.apache.org/}} format, a strongly-schemaed compact binary serialization.
+However, the Avro standard includes its schemas as uncompressed ASCII.
+To conserve bandwidth, we therefore send alerts without including the schemas required to deserialize them.
+Instead, the schemas are stored in a separate schema registry and are referenced by a unique identifier in the alert packet.
+\citep{DMTN-093} describes the alert packaging and schema versioning used in the alerts.
+Avro alert schemas are programatically generated from the database schema definitions stored in the \texttt{lsst.sdm.schemas} package and can be found in the \texttt{lsst.alert.packet} repository\footnote{\url{https://github.com/lsst/alert_packet/}}.
+
+\subsection{Images} \label{sec:images}
+
+\subsection{Prompt Catalogs} \label{sec:catalogs}
+
+\eric{point to schema browser.  forward reference the PPDB.}
+
+\subsection{Data Rights} \label{sec:data-rights}
+
+\citet{RDO-013} describes the Rubin Data Rights policies.
+In brief, alert packets are world-public and can be freely shared with anyone.
+Similarly, the contents of the PPDB are freely sharable, although direct access to the PPDB is restricted to Data Rights holders.
+All other data products are proprietary.

data_services.tex

@@ -0,0 +1,46 @@
+\section{Data Access Services} \label{sec:services}
+
+\subsection{Community Alert Brokers} \label{sec:brokers}
+
+Near real-time access to alert packets is available through services provided by community alert brokers.
+Due to limited bandwidth from the USDF, seven brokers were selected through a proposal process \citep{LDM-612} to receive the alert stream directly from the alert distribution system at the USDF (\S \ref{sec:alertdist}):  
+
+\begin{itemize}
+    \item ALeRCE: Automatic Learning for the Rapid Classification of Events \citep{Forster:21:ALeRCE}
+    \item AMPEL: Alert Management, Photometry, and Evaluation of Light Curves \citep{Nordin:19:AMPEL}
+    \item ANTARES: Arizona-NOIRLab Temporal Analysis and Response to Events System \citep{Matheson:21:ANTARES}
+    \item Babamul  \citep{2025arXiv251100164J}
+    \item Fink \citep{Moller:21:FINK}
+    \item Lasair \citep{Smith:19:Lasair}
+    \item Pitt-Google
+\end{itemize}
+
+As alert packets are world-public and freely sharable (\S \ref{sec:data-rights}), additional brokers and other services may operate ``downstream'' by connecting to full or filtered streams provided by the upstream brokers.
+As of this writing, downstream brokers include
+
+\begin{itemize}
+    \item Point of Interest Broker
+    \item SNAPS: Solar System Notification Alert Processing System 
+\end{itemize}
+
+\eric{check broker sites if there are newer citations}
+
+\subsection{Alert Filtering Service}
+
+\melissa{ANTARES system, community alert filters}
+
+\subsection{Rubin Science Platform}
+
+\gregory{todo}
+
+\subsubsection{Alert Archive}
+
+\subsubsection{Image Services}
+
+\subsubsection{Butler Data Products}
+
+\subsubsection{Prompt Products Database} \label{sec:ppdb}
+
+\subsection{IDACs}
+
+\todo{confirm IDAC section}

intro.tex

@@ -0,0 +1,38 @@
+\section{Introduction}
+
+Time-domain measurements are among of the foundational techniques of astronomical science:
+repeated observations of the night sky enable discovery of transient, variable, and moving objects.
+The cadence of the observations, the intrinsic luminosities and timescales of the phenomena, and the spatial volume probed by the observations set the rate of discovery \citep{Bellm:16:Cadences}. 
+In recent years, scientists have conducted dedicated optical time-domain surveys with large CCD mosaic cameras (e.g., PanSTARRS, DECam, ZTF, HSC, Gaia, Kepler, TESS) as well as distributed telescope networks (e.g., ASAS-SN, ATLAS, KMT-Net, \eric{more}).
+These have yielded large new samples of supernovae, variable stars, active galactic nuclei, and solar system objects.
+Additionally, they have uncovered the first exemplars of rare new classes of objects \eric{add some examples: FBOTS, ISOs, asteroids interior to venus, ...}.
+
+Frequently, time-critical followup observations are necessary to classify and characterize objects discovered in time-domain surveys.
+Historically, human-composed circulars or telegrams were used to dissemenate discoveries to the wider community.
+The increasing rate of transient candidates as well as the desire for rapid automated followup motivated machine-readable alternatives \eric{VOEvent, GCN, SciMMA/HopSkotch, TNS}.
+These trends culminated in the public ``alert stream'' paradigm employed by ZTF \citep{Patterson:19:ZTFAlertDistribution}, in which hundreds of thousands of unfiltered difference image sources are shipped along with historical lightcurves and image cutouts to third-party alert brokers for classification, filtering, and followup.
+This approach has enabled fully automated identification, reporting, and followup of supernova candidates \eric{cites}.
+
+The Legacy Survey of Space and Time (LSST) to be conducted by the Vera C.\ Rubin Observatory promises an order of magnitude increase in transient discovery.
+Rubin's large collecting area, wide field of view, and fast readout and slew will deliver nearly a thousand \eric{confirm} exposures across a wide swath of the Southern Hemisphere sky to unprecedented depths.
+This capability motivated the development of a rapid data processing pipeline to identify and publicize time-variable phenomena in LSST images: the Rubin Alert Production System (AP).
+Along with the annual Data Release Processing (DRP), these productions make use of the Rubin Science Pipelines software as well as the larger systems and infrastructure of Rubin Data Management (DM).
+\eric{cites throughout}
+
+In this paper, we describe the design, implementation, and initial performance of the Rubin Alert Production system.
+In \S \ref{sec:observations} we detail the alert-producing observations taken to date.
+\S \ref{sec:data-products} presents the science data products resulting from alert production and \S \ref{sec:pipeline} describes the data processing pipelines which produce them\footnote{
+Discovery of new solar system objects from daytime processing of the prompt data products is not described here but is presented in \todo{appropriate citation}.}.
+We summarize the data processing environment and infrastructure in \S \ref{sec:data-processing}.
+We characterize the early scientific and technical performance of alert production in \S \ref{sec:performance}.
+In \S \ref{sec:services} and \S \ref{sec:support} we describe how data are served to scientists and the support channels available to users.
+We conclude in \S \ref{sec:conclusion}.
+
+
+%\note{See discussion on common material and timing considerations in \cite{PSTN-020}.}
+%\note{This outline proceeds assuming this paper is being produced at the end of commissioning/early operations, and some form of routine alert production is happening in operations based on commissioning-derived or incremental templates.
+%If only a small amount of LSSTCam commissioning data is available and is being released, only a minimal paper decribing the pipelines used to process that data would useful, and a more complete AP reference should come later.}
+%\note{Information about the processing environment is presumed to be covered in \cite{PSTN-018}.
+%}
+%\note{SS Processing falls under Prompt Processing but detailed discussion is deferred to \cite{PSTN-025}.}
+