Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12574

clarify DM's capabilities for processing non-standard visit images from special programs



      The purpose of this ticket is to ensure that DM's capability for processing non-standard visit images, and required by LSR-REQ-0111, will meet the needs of Special Programs data (which might be considerably more diverse than data for the wide-fast-deep main survey). For now we have adopted a broad interpretation of "processing" to mean both the initial ISR stage and also the Level 1 Alert Production pipeline, because allowing as many LSST images as possible to contribute to the Alert Stream is a benefit to time domain science.

      After comparing the anticipated potential diversity in Special Programs data with the currently planned DM pipelines, we have identified several open questions that need to be resolved in advance of the call for Special Programs white papers.

      (1) What is the anticipated minimum (and maximum, if applicable) exposure time for an image to be successfully processed by ISR and DIA? In the latter case, this will probably be limited by the time needed to generate a well-formed PSF.

      (2) Presently, OSS-REQ-0291 requires that even short exposures be spaced by 15 seconds. Might this requirement be slackened to accommodate a higher cadence Special Program? If so, and if Level 1 processing is required for the science goals, then DM will need to assess options to elastically increase NCSA computational resources and/or the policy for dropping vs. delaying Alerts during observations with a fast ingest cadence. These issues apply also to Special Programs in the Galactic plane, where crowded fields will increase the volume of Alerts. Alternatively, we could formally require that only 30 second visits will be allowed to contribute to Level 1 Alert Production

      (3) Long sequences of images on a single field are a likely Special Programs mode (e.g., DDF). We need to confirm that the processing pipeline has the ability to re-parse a scheduled N-exposure visit into standard 2-exposure visits. Consequential images of the same field may lead to unassociated DIASources of the same DIAObject, and it should be confirmed that the DM pipelines can recognize this when it happens, and flag the resulting Alerts appropriately.

      The deadline has been set to 1.5 months prior to the expected call for Special Programs white paper proposals. Additional context for this ticket can be found in Section 3.1 of the DMTN study on DM and Special Programs, available at https://dmtn-065.lsst.io/


          Issue Links



              • Assignee:
                mgraham Melissa Graham
                mgraham Melissa Graham
                John Swinbank, Kian-Tat Lim, Leanne Guy, Melissa Graham
              • Votes:
                0 Vote for this issue
                4 Start watching this issue


                • Due:

                  Summary Panel