Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16191

Best seeing selector does not work with visit+ccd dataIds

    XMLWordPrintable

    Details

    • Team:
      Alert Production

      Description

      Let's pretend you have many sets of decam full focal plane images which visit the same region of the sky. You would like to use BestSeeingWcsSelectImagesTask to select only 5 which have the best seeing, and make a coadd with them. So you configure nImagesMax=5, proceed to create warps and coadds... and get very sad when there are giant gaps of empty sky all over your coadds.

      This appears to be happening because a decam dataId is an amalgamation of visit+ccdnum. So the image selector is naively choosing the 5 very best visit+ccdnum combinations to include in the warps, which will then be coadded.

      This ticket is to fix the BestSeeingWcsSelectImagesTask (and perhaps the WcsSelectImagesTask too, if applicable) so it knows which dataIds actually overlap on the sky when making its selections.

        Attachments

          Issue Links

            Activity

            Hide
            mrawls Meredith Rawls added a comment -

            It's worth noting this issue is a bit more subtle than this. Even if the selector was modified to function on a per-visit basis (rather than a per-visit+ccd basis), it would be constrained to HiTS-type surveys where each visit revisits (ha) the same exact region of sky. This will most often not be the case. There is a fundamental inconsistency with operating on patches vs. operating on visits that needs to be addressed to fully solve the problem.

            Show
            mrawls Meredith Rawls added a comment - It's worth noting this issue is a bit more subtle than this. Even if the selector was modified to function on a per-visit basis (rather than a per-visit+ccd basis), it would be constrained to HiTS-type surveys where each visit revisits (ha) the same exact region of sky. This will most often not be the case. There is a fundamental inconsistency with operating on patches vs. operating on visits that needs to be addressed to fully solve the problem.
            Hide
            ebellm Eric Bellm added a comment - - edited

            I'd rephrase the problem statement slightly further: the superclass WcsSelectImagesTask identifies images which overlap a given patch anywhere. Any subsequent cuts by subclasses on seeing BestSeeingWcsSelectImagesTask or PSF properties PsfWcsSelectImagesTask may accordingly lead to areas of the patch where no data appears in the coadd, even if there is input data there.

            This behavior is not an issue if the applied cuts are viewed as thresholds--i.e. no data should appear in the coadd if it has properties outside the allowed envelope (missing coverage in the coadd is preferred to coadding "bad" data). However, it does not give the desired behavior for selecting a "good seeing" subset for the purposes of image differencing. In that case we would prefer to achieve maximal spatial coverage but allow the seeing to vary as needed across the patch.

            As a concrete illustration, consider two disjoint fixed pointings, both of which overlap half of a patch. One pointing (A) was visited ten times on the first night, with 1.0" seeing, and the second (B) was visited ten times on a second night with 2.0" seeing. As presently configured, calling BestSeeingWcsSelectImagesTask with nImagesMax = 5 on this data will select five visits from pointing A and none from pointing B, leaving a gap in the coadd. The desired behavior would be to select the 5 best seeing images from both A and B. Overlaps between A and B would further complicate this since we'd prefer to select only the portions of the B input images not covered by A for coaddition.

            A temporary workaround that improves spatial coverage is to set nImagesMax to a large number and set maxPsfFwhm to a value that selects data everywhere. As the above case illustrates, though, we would need to set maxPsfFwhm == 2.0, which is substantially higher than is needed for the A pointing and so in a real example with other data would lead to a worse seeing coadd at that position.

            Show
            ebellm Eric Bellm added a comment - - edited I'd rephrase the problem statement slightly further: the superclass WcsSelectImagesTask identifies images which overlap a given patch anywhere. Any subsequent cuts by subclasses on seeing BestSeeingWcsSelectImagesTask or PSF properties PsfWcsSelectImagesTask may accordingly lead to areas of the patch where no data appears in the coadd, even if there is input data there. This behavior is not an issue if the applied cuts are viewed as thresholds--i.e. no data should appear in the coadd if it has properties outside the allowed envelope (missing coverage in the coadd is preferred to coadding "bad" data). However, it does not give the desired behavior for selecting a "good seeing" subset for the purposes of image differencing. In that case we would prefer to achieve maximal spatial coverage but allow the seeing to vary as needed across the patch . As a concrete illustration, consider two disjoint fixed pointings, both of which overlap half of a patch. One pointing (A) was visited ten times on the first night, with 1.0" seeing, and the second (B) was visited ten times on a second night with 2.0" seeing. As presently configured, calling BestSeeingWcsSelectImagesTask with nImagesMax = 5 on this data will select five visits from pointing A and none from pointing B, leaving a gap in the coadd. The desired behavior would be to select the 5 best seeing images from both A and B. Overlaps between A and B would further complicate this since we'd prefer to select only the portions of the B input images not covered by A for coaddition. A temporary workaround that improves spatial coverage is to set nImagesMax to a large number and set maxPsfFwhm to a value that selects data everywhere. As the above case illustrates, though, we would need to set maxPsfFwhm == 2.0 , which is substantially higher than is needed for the A pointing and so in a real example with other data would lead to a worse seeing coadd at that position.

              People

              Assignee:
              ebellm Eric Bellm
              Reporter:
              mrawls Meredith Rawls
              Watchers:
              Eric Bellm, Meredith Rawls
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.